Have LLMs reaching the dreaded "Peak of Inflated Expectations"?
The "Peak of Inflated Expectations" proves premature for LLMs as real-world applications consistently exceed anticipated capabilities. From transforming software development workflows to enabling unprecedented creative augmentation, these tools demonstrate practical value that transcends the typical hype cycle, suggesting we've only begun to scratch the surface of their potential.
Photo Credit: Rob GrzywinskiOriginally posted on February 13, 2023 on LinkedIn. Edited from original version.
The AGI Distraction
Much of the AI media coverage throughout 2022 was focused on the push towards Artificial General Intelligence (AGI). I believe that this has muddied the waters with respect to LLMs. LLMs are simply one step along the road toward AGI. It would be as if you were constantly talking about your new home and then only showing everyone a smart lock. Of course the smart lock is less interesting than a whole house. But that doesn't mean that the smart lock itself isn't useful, interesting, innovative or provide efficiencies that you never expected.
Setting Realistic Expectations
Given what I've personally experienced with LLMs, I believe we simply set the bar too high with AGI. AGI is a fantastic goal to work towards but is certainly not necessary to achieve once-in-a-millennia impact. (There has been much talk about the fact that LLMs and the hype around them are likely going to change the course of research of AGI simply because the money tends to follow the hype. The scientist in me absolutely believes that that is true. Fundamental research will be redirected towards more near-term goals because of all of the media around LLMs. While I know that all of the researchers would love it to be otherwise, it's unfortunately "How it Goes"™️. Industry will focus on the nearer term profit-making solutions while governments, etc will have to fund the core research. The fact that this modern-day Manhattan Project has been carried out (mostly) in public right in front of our eyes has been exhilarating!)
Continuous Breakthroughs
We don't know what we don't know about LLMs. For me, the most exciting thing that happened last year was when the paper "Large Language Models are Zero-Shot Reasoners" came out in May (of 2022). The authors discovered that you could augment your prompt with the statement "Let's think step by step" and take its reasoning skills from basically zero to near-human levels (in one case from 17.7% to 78.7% -- Inconceivable!). (Watching this whole field collectively poop its pants was an experience like no other.) And this is just one of many such discoveries that was made. How can one take statements about "overinflated expectations" seriously when you've seen this happen?
A Long Time Coming
The recent hype around ChatGPT ha given the impression that LLMs just popped onto the scene. The reality is that GPT-2 came out in February of 2019. GPT-3 was released in mid-2020, GPT-3.5 in early 2022 and ChatGPT in last 2022. And this is only OpenAI. Cohere, Google, BLOOM, etc all have similar models with similar timelines. The original Transformer ("Attention Is All You Need") paper was published mid-2017!
The Bottom Line: Over-Delivering
The bottom line is that the answer is a firm and definitive No. LLMs are far from being at the peak of inflated expectations. If there’s a single take away that one can have over the past year is that LLMs are over delivering on expectations. If you asked me 12 months ago if any appreciable fraction of my software development work would be augmented by any form of AI, I would have laughed myself silly. The reality is that today ~40% of my development work is being augmented or completely replaced by LLMs. When you factor in all of the other ways I use LLMs on a daily basis -- from spec writing to ideation -- over half of my daily output is augmented by LLMs. Inconceivable!(725 tokens)
The "smart colleague with dementia" metaphor illuminates both the potential and limitations of Large Language Models. By understanding LLMs through this lens of brilliant but context-challenged collaborators, we gain practical insights into maximizing their capabilities while navigating their inherent constraints.10 January 2025
A decade-old vision of AI companionship finds its moment as language barriers dissolve through LLMs. The personal journey from dusty pitch decks to daily AI integration reveals how shared context and understanding transform theoretical possibilities into practical realities.8 January 2025
