The intensifying limelight on Generative AI (Gen AI), which started with the launch of ChatGPT towards the end of 2022, is not due to a recent breakthrough overnight. It is the accumulation of hundreds of years of algorithm development, hardware advancement, and data proliferation. You may have heard of other AI ‘milestones’ in the past - DeepBlue, IBM Watson, and AlphaGo, to name a few. The difference is that this time it is expected to stay, mature, and disrupt.
In 2017, Google published a paper named Attention Is All You Need, which introduced a new type of neural network: Transformers. This new architecture has improved parallelisation processing capability and is suited for tasks such as natural language processing. It marked the start of the acceleration of a new breed of AI - Generative AI - which can take on the challenges of creative work that traditionally could only be accomplished by humans, such as writing, programming, art, and architectural design. Since then, organisations have started to build larger and larger Gen AI models on the back of advancements in hardware and cloud computing.
Gen AI models are often called Foundational Models.
Size, Productisation and Applications
With Generative AI models, size matters. There is a roughly linear relationship between the performance of these large, general-purpose models and their size (as of OpenAI GPT-4). AI companies fine-tune their general-purpose models, package specific capabilities into user-friendly products such as ChatGPT and DALL-E, and then ship them to the general public. Developers can also build their own applications on top of these LLMs through APIs for tasks specific to their needs, such as analysing customer sentiment, generating charts, or writing code in a particular style. The application of Generative AI can be either incremental - assisting creative work or improving personal recommendations - or disruptive, for example, replacing contracted newsletter copywriters.
Discovery
Specific Gen AI models, tools and services are tagged with tech/ai/gen-ai (and more specific children such as tech/ai/llm, tech/ai/agentic). Use the tag pane or a Dataview query to browse.
References
- Akosner (2022). Generative AI: A Creative New World. Sequoia Capital US/Europe. https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/.