Recently, Andrej Karpathy, a prominent figure in the world of AI, delivered a riveting talk on the potential of large language models, specifically focusing on OpenAI’s latest model, GPT-4. His presentation unraveled the intricacies of these AI powerhouses and their application, as well as the strategy needed to extract the best performance from them.
Karpathy’s opening remarks drew parallels between Google’s AlphaGo and GPT-4, referring to both as “Tree of Thought” systems. They generate possibilities (branches), evaluate them, and prune unnecessary ones. He further elaborated that the optimal utilization of these models isn’t restricted to simple question-answer prompts. Instead, it involves a combination of multiple prompts woven together with Python glue code, essentially redefining the concept of ‘prompt engineering’.
He cited two enlightening examples of advanced prompt engineering: The ‘React’ paper and Auto GPT. In ‘React’, the model’s responses are a sequence of thoughts, actions, and observations that mimic a thinking process. This mechanism allows the model to use tools in its actions. Auto GPT, on the other hand, equips language models with task lists, facilitating a more organized and efficient breakdown of tasks.
One of the crucial insights from Karpathy’s talk was the inherent imitative nature of language models. They aim to generate responses that are statistically similar to their training data, irrespective of the quality of the solution. Thus, explicitly asking for high-quality, detailed responses is necessary to get the best out of these models.
He also emphasized the importance of augmenting models with tools like calculators or code interpreters to enable them to solve problems that are inherently difficult for them. Equally vital is informing these models when and how to use these tools. He touched upon the concept of retrieval-augmented models, which load the working memory of the model with contextually relevant information, improving their response quality and coherence.
Another fascinating technique Karpathy discussed was constraint prompting. This involves directing the output of language models to fit specific forms or templates, using a method called “Guidance”, developed by Microsoft. By imposing a template (like JSON) on the output, developers can create structured responses that are more predictable and useful.
While prompt engineering is essential, Karpathy also explored the idea of finetuning models, which involves changing the model’s weights. While finetuning has become more accessible due to recent techniques like LoRA, which modifies only small, sparse pieces of the model, it is important to note that it is technically involved, requires a higher degree of expertise, and can slow the iteration cycle.
As for recommendations on using large language models (LLMs), Karpathy proposes a two-step process. The first is achieving top performance using a GPT-4 model, with very detailed prompts full of task context and relevant instructions. The second step involves optimizing this performance, potentially through finetuning.
However, these strategies come with a caveat. Karpathy cautions about the limitations of LLMs, including biases, potential hallucinations, reasoning errors, and susceptibility to various types of attacks. His advice is to deploy LLMs in low-stakes applications and always pair them with human oversight.
He concluded the talk on an optimistic note by highlighting GPT-4’s impressive knowledge across various domains. His demonstration of the model’s capabilities with a Python example showed how it can be used to generate inspiring, human-like messages.
In a nutshell, Andrej Karpathy’s talk underscores the promise and potential of large language models, and while they have their limitations, they’re an extraordinary testament to the progress made in artificial intelligence.