Название: What Are ChatGPT and Its Friends? Opportunities, Costs, and Risks for Large Language Models
Автор: Mike Loukides
Издательство: O’Reilly Media, Inc.
Год: 2023-03-24
Язык: английский
Формат: pdf, azw3, epub, mobi
Размер: 10.2 MB

ChatGPT is not your run-of-the-mill automated chat server. It's much more. But if you want to understand the possibilities that ChatGPT (and similar models) offers your organization, it's important to move past the hype. Large language models like GPT-3 and GPT-4 represent one of the biggest technological leaps we've seen in our lifetime. In this concise report, Mike Loukides zeros in on the essential things you should understand to take advantage of ChatGPT.

ChatGPT, or something built on ChatGPT, or something that’s like ChatGPT, has been in the news almost constantly since ChatGPT was opened to the public in November 2022. What is it, how does it work, what can it do, and what are the risks of using it? A quick scan of the web will show you lots of things that ChatGPT can do. Many of these are unsurprising: you can ask it to write a letter, you can ask it to make up a story, you can ask it to write descriptive entries for products in a catalog. Many of these go slightly (but not very far) beyond your initial expectations: you can ask it to generate a list of terms for search engine optimization, you can ask it to generate a reading list on topics that you’re interested in. It has helped to write a book. Maybe it’s surprising that ChatGPT can write software, maybe it isn’t; we’ve had over a year to get used to GitHub Copilot, which was based on an earlier version of GPT. And some of these things are mind blowing. It can explain code that you don’t understand, including code that has been intentionally obfuscated. It can pretend to be an operating system. Or a text adventure game. It’s clear that ChatGPT is not your run-of-the-mill automated chat server. It’s much more.

First, let’s make some distinctions. We all know that ChatGPT is some kind of an AI bot that has conversations (chats). It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, which has received some specialized training. GPT-3.5 is one of a class of language models that are sometimes called “large language models” (LLMs)—though that term isn’t very helpful. The GPT-series LLMs are also called “foundation models.” Foundation models are a class of very powerful AI models that can be used as the basis for other models: they can be specialized, or retrained, or otherwise modified for specific applications. While most of the foundation models people are talking about are LLMs, foundation models aren’t limited to language: a generative art model like Stable Diffusion incorporates the ability to process language, but the ability to generate images belongs to an entirely different branch of AI.

Models like ChatGPT will play an important role in the future of programming. We are already seeing widespread use of GitHub Copilot, which is based on GPT-3. While the code Copilot generates is often sloppy or buggy, many have said that its knowledge of language details and programming libraries far outweighs the error rate, particularly if you need to work in a programming environment that you’re unfamiliar with. ChatGPT adds the ability to explain code, even code that has been intentionally obfuscated. It can be used to analyze human code for security flaws. It seems likely that future versions, with larger context windows, will be able to understand large software systems with millions of lines, and serve as a dynamic index to humans who need to work on the codebase. The only real question is how much further we can go: can we build systems that can write complete software systems based on a human-language specification, as Matt Welsh has argued? That doesn’t eliminate the role of the programmer, but it changes it: understanding the problem that has to be solved, and creating tests to ensure that the problem has actually been solved.

Contents: