Uniqtech Guide to GPT-3, ChatGPT and Large Language Models (LLM) 💬
Deep dive on GPT-3, ChatGPT, GPT-4
This comprehensive guide to GPT models explains the basics, compare GPT-3 to GPT-4, ChatGPT, discusses the core architecture that enabled GPT-3 like models — transformer and attention, and shares the trend and future of generative AI.
Welcome to our newsletter (Issue Number 052023-01) and digital flash cards. Our contents save time for students and developers. Our notes will help you learn AI faster - harnessing power of AI 🦾. Take your GPT skills to the next level: become a power user, become a developer. Never be obsolete.
By the way, we got permission to build, test developer plugins (alpha/beta) for GPT, what should we build? Let us know what you want to see. We have something in mind that will turbo charge learners’ ability to learn faster.
Our newsletter and digital flash cards save you time so you can learn modern AI skills fast and go out to do great things.
Our newsletter and digital flash cards save you time so you can learn modern AI skills fast and go out to do great things.
In the upcoming weeks, we will release dedicated newsletters for Hugging Face, Developer Lifestyle, Advanced Prompt Engineer and more. Stay tuned. 👩🏻💻
[Subscribe to our newsletters today!]
Disclaimers, Warnings, Policies
Read our disclaimers first, attached in the next section. If you disagree with any of our disclaimers, warnings, terms, legal terms and policies please immediately stop using all our contents. By reading on, you agree to all our terms. Pro cards are high quality flash cards for paid members only. Become a paid member today and turbo charge your AI learning journey. Visit here. Our contents are exclusively hosted on our substack, medium and proprietary sites. Other sites do not have our permission to post our content. Never trust any links from those sites. There are websites that scrape our articles that may replace our links with malicious links. Please only consume our contents on our official websites and domains, directly from us.
https://ml.learn-to-code.co
http://www.uniqtech.co/
https://uniqtech.medium.com
For all policies, legal docs and terms visit https://ml.learn-to-code.co/payments.html
Disclaimer 3rd party. Before clicking on any external URL, be sure to read this disclaimer, and do your own research. Warning 3rd party links can change any time, may contain malicious contents. Disclaimer, warning: 3rd party resources https://ml.learn-to-code.co/skillView.html?skill=NNNfU9HNQwliDvzx55WI
GPT-3 & Generative AI model basics
GPT-3 is a part of the trending generative AI systems, general artificial intelligence (AGI) which are able to generate unique original-appearing content, data, natural language texts. These systems are able to achieve original appearing results by learning (being trained on) a large amount of data on the internet.
GPT-3 in OpenAI’s Own Words
What is the mission of OpenAI? : "OpenAI is an AI research and deployment company. Our mission is to ensure that artificial general intelligence benefits all of humanity. (Two pillars: ) startup, research lab, safety and policy." - OpenAI presentation.
What does GPT-3 really do? How GPT-3 technical product manager describes GPT-3: How GPT-3 like models really work? "It doesn't have a goal / objective in mind other than predicitng the next word." ... "based on all the previous text". It is not programmed to do a specific task. The single API can be used for all these tasks because "at the text level it understands the text”
Use cases: tasks that GPT-3 can help with: include classification, data parsing, text generation, summarization, and search. It can handle general NLP language tasks in English. 'Unlike most Al systems which are designed for one use-case, the API today provides a general-purpose "text in, text out" interface, allowing users to try it on virtually any English language task.’ - direct quote by Ashley Pilipiszyn.
OpenAI API is the first commercial product that allows developers to access the platform's AI models.
[Subscribe to our newsletters today!]
More GPT-3 Basics
ChatGPT made history as the fastest growing app in terms of daily active user count in a short period of time. Yes, its growth curve was better than TikTok!
For text-in, text-out models like GPT-3, its sole purpose is to predict the next or next sequence of words.
At its core, GPT-3 is a text completion, text generation tool. That’s its No.1 goal.
GPT-3 and similar models are known as large language models. What is a large language model (LLM)? What do we mean by large language models? GPT-3 has 175 billion parameters. Its predecessor GPT-2 only had 1.5 billion parameters. (DALLE image generation model has 12 billion parameters.)
Not task specific : What do we mean when we say GPT-3 is task agnostic, not task specific? GPT-3 is a general purpose natural language model. Because GPT-3 has the capability of few shot learning and generalizability, GPT-3 can be used for a wide range of natural language tasks without being trained for specific tasks. The opposite of task agnostic is to be trained for a specific task - used for binary classification only, or trained for sentiment analysis only (NLP example).
What is multi-modal: right now famous models are mostly text-in text-out. Eventually they may become more multi-modal like DALLE, generate beyond text results - image, video, audio etc.
Competitions, LLM Competitive Landscape. GPT-3 versus competitor(s): Google Bard competes with OpenAI GPT-3. Google internally issued red alert indicating that GPT-3 threatens its search business. Bing has integrated ChatGPT into its search bar (I already see it in my edge mobile browser) and wow it is a whole lot smarter than its predecessors. Robinhood newsletter : “Even the Bard demo made embarrassing mistakes. May be Google felt pressure to introduce Bard too early due to OpenAI competition? … Google's been in the Al game for years (it invented the "transformer" tech that powers ChatGPT). But it's historically been more cautious with rolling out these lightly tested tools to the public” Google reportedly issued an internal warning about OpenAI’s competition https://finance.yahoo.com/news/googles-management-reportedly-issued-code-190131705.html.
Google Bard: What would you like to ask Bard? We also have access to Bard. With the new update, Bard can help you code : "Code and debug with Bard 🐞 This update introduces software development support for over 20 programming languages, including JavaScript, Python, SQL, TypeScript, and even Google Sheets functions. Collaborate with Bard to help generate, explain, and debug your code. Google Colab users can also export Python code generated by Bard directly to Google Colab to seamlessly continue programming tasks.”
Few shot learning : When provided with just a few good examples at inference, GPT-3 can successfully “learn” and complete language tasks.
Interfacing with GPT Models: There are several ways to interact with GPT-3, GPT-4, ChatGPT. You can do it via Playground a text area on the OpenAI website, via https://platform.openai.com/playground, via the Python API client using command line and Jupyter Notebook using the API key, and ChatGPT https://chat.openai.com/auth/login . Each has its advantages. For example, the ChatGPT chat interface is easy to use, requires the least amount of technical knowledge.
Creative use cases of GPT-3 models
Changing style of writing : Companies have been using GPT-3 to write poems, rap songs about their products, write social media posts.
GPT-3 can generate writings that are virtually indistinguishable from human writing. A college kid’s fake, AI-generated blog fooled tens of thousands. This is how he made it. “It was super easy actually,” he says, “which was the scary part.” - by Karen Haoarchive page "At the start of the week, Liam Porr had only heard of GPT-3. By the end, the college student had used the AI model to produce an entirely fake blog under a fake name. It was meant as a fun experiment. But then one of his posts reached the number-one spot on Hacker News.”
More use cases: 👩🏼💻 Use ChatGPT 💬 for UIUX design. Use GPT models to auto complete spreadsheet with data with accuracy (right now it is lacks accuracy). More plug-ins: "The first plugins have been created by Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier." Source https://openai.com/blog/chatgpt-plugins
Auto completion
Next, let’s have an in-depth discussion about GPT-3.
[Subscribe to our newsletters today!]
Transformer
The core technology behind fancy large language models (LLM) is Transformer - a neural network layer introduced by Google researchers. Also see advanced GPT-3 usage section below. Transformers are powered by Attentions also a concept introduced by Google Researchers in 2017.
Our previous flash cards about transformers
Transformer Basics: https://ml.learn-to-code.co/skillView.html?skill=14Gs13fzNShpGTbATuG5
Best explanation of Transformers https://ml.learn-to-code.co/skillView.html?skill=KZBv9d1rtSVkerCS5D6O
Transformer models you should know.
Getting started with Hugging Face state of art NLP models, transformers https://ml.learn-to-code.co/skillView.html?skill=0KAxS5PW1JQEtH8xCZ19
Easter egg: NLP state-of-art transformers tutorial https://ml.learn-to-code.co/skillView.html?skill=fgMl8Ag2RWJZEm3S9YBB
Attention is All You Need Explained in a Twitter Thread [pro, paid member] https://ml.learn-to-code.co/skillView.html?skill=jicELImWdtoUthrtN7DZ
ChatGPT versus GPT-3, GPT-3.5, GPT-4
The conversation-based interface of ChatGPT is perhaps more natural, easier-to-use for early adopters and non-coders (non-developers). This characteristic may have propelled its massive adoption and population. In fact, ChatGPT saw the fastest adoption rate in software history.
ChatGPT users can select GPT-4 model as its compute engine instead of GPT-3. There’s a limit "GPT-4 currently has a cap of 25 messages every 3 hours. Expect significantly lower caps, as we adjust for demand.” (displayed near the ChatGPT prompt input box).
In OpenAI’s own words “ChatGPT is powered by gpt-3.5-turbo and gpt-4, OpenAI's most advanced models.” - OpenAI Cookbook
ChatGPT is a fine tuned version of GPT-3.5 (OpenAI & MIT), which is the next release after GPT-3 (2020 timeline).
The original GPT-3 playground seems easy-to-use, but requires a bit more formatting in its free form text area when providing examples, prompts, questions or answers to the model, selecting models (e.g. davinci) and or set hyperparameters.
GPT-4 is the current state of art (SOA) generative language model by OpenAI. It demonstrated human-level competencies on advanced exams and tasks.
GPT-4 has exhibited extraordinary test-taking capabilities: it has out-performed humans on academic and professionals exams during simulation, including AP tests, graduate tests and the Bar exam for lawyers.
GPT-4 performance on hard exams, AP exams, professional medical legal exams
GPT-4 Performance on Professional and Graduate Level Exams https://ml.learn-to-code.co/skillView.html?skill=B6paQ7Hm93psJaisYABA
GPT-4 passed US medical licensing exam https://ml.learn-to-code.co/skillView.html?skill=ncrK8JeORADs2P0GBL2O
GPT-4 can solve difficult problems with greater accuracy, thanks to its broader general knowledge and advanced reasoning capabilities." (Source: OpenAI newsletter)
OpenAI also thinks GPT-4 is less likely to provide unsafe answers, generate results that are controversial or poses risks on sensitive or illegal topics. It is also more like to answer with some thing like “I don’t know the answer”. Warning: despite recent advances, there are still biases, unsafe answers, and issues with GPT responses. Do your own research. Use caution.
Dear readers, what would you like to ask GPT-4? Let us know. We can try it out for you. Use the message tab.
How many parameters? The number of parameters in GPT-4 is yet to be confirmed. GPT-3 number of parameters was published in the paper. GPT-3 has 175 billion parameters. Its predecessor GPT-2 only had 1.5 billion parameters.
Every new GPT model is more costly to train and serve than its predecessor: "Sam Altman stated that the cost of training GPT-4 was more than $100 million." Source: Wikipedia.
The interaction with ChatGPT looks more like zero-shot learning. We simply ask the model to perform, without giving examples or instructions and hope for the best result. And thanks to ChatGPT being fairly smart, we usually get reasonable results. ChatGPT can also use legacy GPT-3 model, GPT-3.5 (default) and GPT-4. When interact with GPT-3 playground, we will want to provide it with more example prompts and example answers / labels - effectively few-shot learning. We give it just a few high quality demonstrations of what we want it to achieve. The reality is you can use GPT models however you want. People are very creative.
Few shot learning steers the model to do tasks without explicitly program it or update its weights. To steer GPT-3 to do classification tasks or summarization tasks, we just need to change our prompts (prompt engineering): we can say summarize the paragraph below, be sure to note keywords like this and that.
OpenAI says: “Bing (timeline Jan 2023) is powered by one of our next-generation models that Microsoft customized specifically for search. It incorporates advancements from ChatGPT and GPT-3.5.”
Easter Egg: ChatGPT explained by experts [pro] - insightful lightening talk about ChatGPT and the Advancement of Generative AI https://ml.learn-to-code.co/skillView.html?skill=womGbCP1LlWSMXAcHh0w
[Subscribe to our newsletters today!]
Deep Dive : Uniqtech Guide to Prompt Engineering and Advanced Prompt Engineering Practice.
Advanced usage of GPT-3 and ChatGPT [pro resource, paid members only]
Explained by OpenAI engineers / product managers :
Easter Egg: How to use OpenAI and GPT-3 like models (intermediate, advanced) [pro, ebook] 🦾
An analogy that explains how to program GPT-3 using prompt engineering [pro] 🦾
OpenAI PM explains how to write better prompt for GPT-3 (prompt engineering): OpenAI PM explains how to write better prompt for GPT-3 (prompt engineering) [pro] https://ml.learn-to-code.co/skillView.html?skill=I9eDrbm6nWJGXnf5io1F
Easter egg (premium resource for paid subscribers only)best course on prompt engineering.
Advanced GPT-4 prompt engineering [pro, easter egg resource] - Free high quality ChatGPT, GPT-4 prompt engineering course https://ml.learn-to-code.co/skillView.html?skill=nNF6oeN3ZssmpYTrDpQG
Customize your GPT-3
Hyper parameter tuning for GPT-3 [pro, paid members, best practice, pro tip] https://ml.learn-to-code.co/skillView.html?skill=Y5bm83ybAHpVUbilbelG
Advanced Usage of OpenAI API, GPT-3 Model [pro, paid members] https://ml.learn-to-code.co/skillView.html?skill=GDEfrEZ1ayhu2cF4vQ7n
Build your on GPT, Advanced GPT-3
What if GPT is banned or it goes away behind a more formidable paywall? Learn to build your own GPT like systems from scratch using machine learning and AI skills.
You can use LlaMA model and llamaindex. What is the LLaMA and Stanford Alpaca Model 🦙 https://ml.learn-to-code.co/skillView.html?skill=W8kp8JrfTGQoZ8zYus3L
Meet the Llamas and Alpaca … models: Hugging Face hired real life llamas to attend its AI party 🦙 https://ml.learn-to-code.co/skillView.html?skill=juzQSGgbErp6TFhinsMw
Use open sourced options and alternatives like Hugging Face. Our next newsletter is on Hugging Face.
Advanced Usage of OpenAI API, GPT-3 Model [pro]
Building advanced apps with GPT-3
GPT-3 powered apps: Here are some examples of how GPT-3 is used by OpenAI clients https://openai.com/blog/gpt-3-apps GPT-3 is also integrated into Bing Search to provide better search results. GPT-3 powered applications: possible features using GPT-3 text generation, data parsing, summarizing, search, classification, topic modeling.
Other Generative AI Apps
OpenAI Text-to-Image AI Generation DALLE : "DALL-E¹ is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text-image pairs. We've found that it has a diverse set of capabilitiesincluding creating anthropomorphized versions of animals and objects, combining unrelated concepts in plausible ways, rendering text, and applying transformations to existing images … We decided to name our model using a portmanteau of the artist Salvador Dalí and Pixar's WALL.E” - Source: OpenAI website.
How does AI powered video generation work? How digital avatars are created? Voice generation at work.
Meta’s new AI can turn text prompts into videos
https://makeavideo.studio
Stability AI plugin for photoshop: adding AI power to traditional photo editing apps.
⭐ A particularly interesting generative model you should know is the llama and the alpaca model 🦙 Read our flash card here about the llama and alpaca model and how Hugging Face hosted a real life AI parties and invited three live llamas to attend and celebrating the models and the unleashing of its weights.
What is the LLaMA and Stanford Alpaca Model 🦙 https://ml.learn-to-code.co/skillView.html?skill=W8kp8JrfTGQoZ8zYus3L
Meet the Llamas and Alpaca … models: Hugging Face hired real life llamas to attend its AI party 🦙 https://ml.learn-to-code.co/skillView.html?skill=juzQSGgbErp6TFhinsMw