transformer models Archives - insideAI News https://insideainews.com/tag/transformer-models/ Illuminating AI's Frontiers: Your Go-To News Destination. Tue, 25 Jun 2024 21:07:13 +0000 en-US hourly 1 https://wordpress.org/?v=6.6.1 https://insideainews.com/wp-content/uploads/2024/06/iain-favicon.png transformer models Archives - insideAI News https://insideainews.com/tag/transformer-models/ 32 32 136462205 Video Highlights: Fine Tune GPT-J 6B in Under 3 Hours on IPUs https://insideainews.com/2023/06/03/video-highlights-fine-tune-gpt-j-6b-in-under-3-hours-on-ipus/ https://insideainews.com/2023/06/03/video-highlights-fine-tune-gpt-j-6b-in-under-3-hours-on-ipus/#respond Sat, 03 Jun 2023 13:00:00 +0000 https://insidebigdata.com/?p=32512 Did you know you can run GPT-J 6B on Graphcore IPU in the cloud? Following the now infamous leaked Google memo, there's been a real storm in the AI world recently around smaller, open source language models, like GPT-J, that are cheaper and faster to fine-tune, run and perform just as well as larger models for many language tasks. ]]> https://insideainews.com/2023/06/03/video-highlights-fine-tune-gpt-j-6b-in-under-3-hours-on-ipus/feed/ 0 32512 Research Highlights: SparseGPT: Prune LLMs Accurately in One-Shot https://insideainews.com/2023/03/03/research-highlights-sparsegpt-prune-llms-accurately-in-one-shot/ https://insideainews.com/2023/03/03/research-highlights-sparsegpt-prune-llms-accurately-in-one-shot/#respond Fri, 03 Mar 2023 14:00:00 +0000 https://insidebigdata.com/?p=31764 A new research paper shows that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. ]]> https://insideainews.com/2023/03/03/research-highlights-sparsegpt-prune-llms-accurately-in-one-shot/feed/ 0 31764 Video Highlights: Attention Is All You Need – Paper Explained https://insideainews.com/2023/02/15/video-highlights-attention-is-all-you-need-paper-explained/ https://insideainews.com/2023/02/15/video-highlights-attention-is-all-you-need-paper-explained/#respond Wed, 15 Feb 2023 14:00:00 +0000 https://insidebigdata.com/?p=31639 In this video presentation, Mohammad Namvarpour presents a comprehensive study on Ashish Vaswani and his coauthors' renowned paper, “Attention Is All You Need.” This paper is a major turning point in deep learning research. The transformer architecture, which was introduced in this paper, is now used in a variety of state-of-the-art models in natural language processing and beyond. Transformers are the basis of the large language models (LLMs) we're seeing today. ]]> https://insideainews.com/2023/02/15/video-highlights-attention-is-all-you-need-paper-explained/feed/ 0 31639 The Move Toward Green Machine Learning https://insideainews.com/2022/11/06/the-move-toward-green-machine-learning/ https://insideainews.com/2022/11/06/the-move-toward-green-machine-learning/#respond Sun, 06 Nov 2022 14:00:00 +0000 https://insidebigdata.com/?p=26201 A new study suggests tactics for machine learning engineers to cut their carbon emissions. Led by David Patterson, researchers at Google and UC Berkeley found that AI developers can shrink a model’s carbon footprint a thousand-fold by streamlining architecture, upgrading hardware, and using efficient data centers. ]]> https://insideainews.com/2022/11/06/the-move-toward-green-machine-learning/feed/ 0 26201 Research Highlights: Transformer Feed-Forward Layers Are Key-Value Memories https://insideainews.com/2022/05/06/research-highlights-transformer-feed-forward-layers-are-key-value-memories/ https://insideainews.com/2022/05/06/research-highlights-transformer-feed-forward-layers-are-key-value-memories/#respond Fri, 06 May 2022 13:00:00 +0000 https://insidebigdata.com/?p=29230 In this regular column, we take a look at highlights for important research topics of the day for big data, data science, machine learning, AI and deep learning. It’s important to keep connected with the research arm of the field in order to see where we’re headed. In this edition, if you (like me) have wondered what the feed-forward layers in transformer models are actually doing, this is a pretty interesting paper on that topic. Enjoy!]]> https://insideainews.com/2022/05/06/research-highlights-transformer-feed-forward-layers-are-key-value-memories/feed/ 0 29230 Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training https://insideainews.com/2022/04/13/habana-labs-and-hugging-face-partner-to-accelerate-transformer-model-training/ https://insideainews.com/2022/04/13/habana-labs-and-hugging-face-partner-to-accelerate-transformer-model-training/#respond Wed, 13 Apr 2022 14:00:00 +0000 https://insidebigdata.com/?p=29034 Habana® Labs, a pioneer in high-efficiency, purpose-built deep learning processors, and Hugging Face, the home of Transformer models, announced that they’re joining forces to make it easier and quicker to train high-quality transformer models. Thanks to the integration of Habana’s SynapseAI software suite with the Hugging Face Optimum open-source library, data scientists and machine learning engineers can now accelerate their Transformer training jobs on Habana processors with just a few lines of code and enjoy greater productivity as well as lower training cost.]]> https://insideainews.com/2022/04/13/habana-labs-and-hugging-face-partner-to-accelerate-transformer-model-training/feed/ 0 29034