By Anita Srinivasan
14th March, 2024
Artificial Intelligence (AI) has come a long way from its early days of rule-based systems and narrow applications. In recent years, there has been a shift in the AI landscape, moving away from the traditional dependence on individual models towards the integration of Compound AI Systems. This transition marks a significant step forward, encouraging the collaboration between several AI components and creating more robust and versatile solutions.
Today, we will explore the evolution of compound AI systems, looking at what it means for developers. We will also look at some of the key challenges presented by Compound AI along with overcoming strategies to mitigate these challenges.
EMERGENCE OF COMPOUND AI SYSTEMS
Recognizing the need for more versatile and adaptive solutions, the AI community has shifted towards Compound AI Systems. These systems integrate multiple AI models, each specialized in different domains, to work together synergistically. The goal is to combine the strengths of individual models, compensating for their weaknesses and creating a more comprehensive and intelligent system.
An example of how Compound AI is already making significant strides in bringing about innovation in various sectors is the latest antibiotic drug discovered by a group of students.
A group of MIT students made a groundbreaking discovery using AI to find a new antibiotic in December 2023. Their innovative approach, powered by advanced algorithms, analyzed large datasets to identify molecules with strong antimicrobial properties.
This AI-driven method outperformed traditional techniques and represents a significant step in addressing antibiotic resistance. The students’ work not only highlights the potential of AI in drug discovery but also offers hope for future advancements in precision medicine and global health solutions.
We believe that compound AI systems are likely to be the most effective way to maximize AI results in the future, making them one of the most impactful trends in AI for 2024.
UNDERSTANDING COMPOUND AI SYSTEMS AND HOW ARE THEY USEFUL
A Compound AI System refers to an advanced artificial intelligence architecture that integrates multiple specialized AI models, each designed for specific tasks or domains.
Integrating compound AI systems into projects offers enhanced versatility, adaptability, and performance. These systems combine specialized models, adapting dynamically to changing environments and effectively addressing complex challenges across various sectors. By collaboratively utilizing the strengths of each model and optimizing resource utilization, they reduce development costs while providing comprehensive solutions. Hence, developing compound AI systems requires a strategic approach to seamlessly integrate all AI models.
As AI continues to mature, the impact of Compound AI Systems’ versatility will become more evident in our daily lives, enhancing problem-solving capabilities, and opening new avenues for innovation. A study by Berkeley Artificial Intelligence Research reveals that Google’s AlphaCode 2 achieves remarkable programming results by using a combination of LLMs to generate a million potential solutions and then narrowing down the set.
Similarly, AlphaGeometry solves Olympiad problems by combining an LLM with a traditional solver. In the business world, Databricks found that 60% of LLM applications use retrieval-augmented generation (RAG), and 30% use multi-step chains.
This shift to compound systems raises interesting design questions but is also exciting because it shows that superior AI outcomes can be achieved through clever engineering, not just scaling up training.
The BAIR researchers present numerous AI systems in the table below, offering a comprehensive analysis of the challenges along with detailing recent research conducted in these domains.
| AI SYSTEM | COMPONENTS | DESIGN | RESULTS |
| AlphaCode 2 | Fine-tuned LLMs for sampling and scoring programs Code execution module Clustering model | Generates up to 1 million solutions for a coding problem then filters and scores them. | Matches 85th percentile of humans on coding contests |
| AlphaGeometry | Fine-tuned LLM Symbolic math engine | Iteratively suggests constructions in a geometry problem via LLM and checks deduced facts produced by symbolic engine. | Between silver and gold International Math Olympiad medallists on timed test |
| Medprompt | GPT-4 LLM Nearest-neighbour search in database of correct examples LLM-generated chain-of-thought examples Multiple samples and ensembling | Answers medical questions by searching for similar examples to construct a few-shot prompt, adding model-generated chain-of-thought for each example, and generating and judging up to 11 solutions. | Outperforms specialized medical models like Med-PaLM used with simpler prompting strategies |
| Gemini on MMLU | Gemini LLM Custom inference logic | Gemini’s CoT@32 inference strategy for the MMLU benchmark samples 32 chain-of-thought answers from the model and returns the top choice if enough of them agree or uses generation without chain-of-thought if not. | 90.04% on MMLU, compared to 86.4% for GPT-4 with 5-shot prompting or 83.7% for Gemini with 5-shot prompting |
| ChatGPT Plus | LLM Web Browser plugin for retrieving timely content Code Interpreter plugin for executing Python DALL-E image generator | The ChatGPT Plus offering can call tools such as web browsing to answer questions; the LLM determines when and how to call each tool as it responds. | Popular consumer AI product with millions of paid subscribers |
| RAG, ORQA, Bing, Baleen, etc | LLM (sometimes called multiple times) Retrieval system | Combine LLMs with retrieval systems in various ways, e.g., asking an LLM to generate a search query, or directly searching for the current context. | Widely used technique in search engines and enterprise apps. |
CHALLENGES TO EXPECT WHILE WORKING WITH COMPOUND AI SYSTEMS
Building, improving, and operating Compound AI Systems is notably more challenging than working with a single AI model, involving careful planning for model collaboration and system structure. Addressing challenges such as operational issues, data handling, quality assurance, and cost management is complicated due to the multiple models being involved. Unexpected security risks arise, demanding robust safety measures. Continuous learning, real-time updates, and managing increased complexity are also challenging.
To overcome these hindrances, collaboration among experts in machine learning, software engineering, and specific domains is essential. Developers must prioritize comprehensive system design, integrating specialized models for optimal collaboration. Transparent model collaboration with explainable AI enhances trust. A focus on security-aware model development addresses vulnerabilities in multiple model integration from design to training.
FUTURE OUTLOOK
In closing, we believe the future outlook for Compound AI Systems is promising, with ongoing advancements in comprehensive system design, automated tuning tools, and transparent model collaboration through explainable AI. As these technologies mature, we anticipate a more seamless integration of AI models, greater adaptability to dynamic tasks, and enhanced trust in AI systems. The journey ahead involves not only overcoming current hurdles but also embracing continuous innovation and collaboration to unlock the full potential of Compound AI Systems in shaping a more intelligent and interconnected future.
We at InovarTech can help you unlock new possibilities in the tech space. Visit us to explore our resources or book a free strategic consultation with our experts for a tech-driven future and business success.
References:
