In late June 2023, Microsoft quietly annouced their new AI language model Orca — a new open-sourced development in the field of Al that can imitate the reasoning of large foundation models in a much smaller footprint, which may revolutionise development of generative AI technology…..
This new AI model is designed to solve many of the limitations of using smaller language models and revolutionise the Al industry.
Orca will do this by “imitating” the reasoning processes of GPT-4 into smaller language models making it more efficient and effective is specific and niche use cases and in more localised instances.
Built in Microsoft and OpenAI partnership
The development of Orca is part of the ongoing collaboration between Microsoft and OpenAI, which has led to the creation of a new AI model that can replicate the intricate reasoning and complex instructional processes of GPT-4, and able to house billions of parameters.
Orca is a 13-billion parameter model that learns to imitate the reasoning process of LFMs
Microsoft
Microsoft say that with the help of GPT-4, Orca is capable of learning explanations, step-by-step thought processes, and other complex instructions.
What makes Orca AI different?
What makes Orca AI different from other AI models is its unique learning approach. Orca will learn by emulating the reasoning processes of large-scale language models.
This approach will enables Orca AI to understand and replicate the reasoning steps of these models, thereby enhancing its own reasoning capabilities. This learning mechanism is big change from the conventional AI models, and since Orca is much smaller in size, it will not need as much computational resources to operate and is said to be equal in performance with large foundation models (LFM) and Large Language Models (LLM) such as #GPT-4.
The video below explains this in more detail.
Orca is the first small parameter model to replicate, not only the patterns and output created by large parameter modules, but also to imitate the reasoning process of these models, all while maintaining a much smaller overhead.
Orca can also be optimised for specific tasks and trained using large language models like GPT-4.
Microsoft is investing heavily in AI and are working on multiple AI projects and tools to further enhance, innovate and differentiate their products and services, and Orca is another step in that direction.