The generative technology landscape is evolving at a rapid pace. Oracle is committed to offering the latest capabilities to accelerate your AI journey in the cloud. So, Oracle announced a host of new features for the Oracle Cloud Infrastructure (OCI) Generative AI service. With these latest additions, the platform now offers a new generation of large language models (LLMs), supported through a new chat API.
- New Cohere Command R and R+ models
Oracle announces the general availability of Cohere Command R and Cohere Command R + on OCI Generative AI for on-demand and dedicated hosting.
According to Cohere, Command R targets the emerging scalable category of models that balance high efficiency with strong accuracy, enabling companies to move beyond proof of concept and into production. It’s designed for RAG applications and enterprise use cases. The Cohere Command R model has the following advantages over the original Cohere model on the Generative AI service:
- Lower latency and higher throughput
- Longer 16k context window
- Improved capabilities across 10 key languages
The context window is a textual range that the LLM can process when generating information. The increase allows a broader array of use cases and improved contextual accuracy. This enhancement and the key languages supported—English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Simplified Chinese, and Arabic—make Cohere’s models more enterprise-focused than ever. Use the Command R model for general-purpose conversations and a wide range of applications, such as customer service, content generation, and language translation. Fine-tuning for Command R will be available in an upcoming release of OCI Generative AI.
OCI Generative AI also now supports the Cohere Command R + model. Like Command R, Command R+ features a longer 128k context window and supports the same 10 key languages. Command R+ is an enhanced version of the model, with a deeper understanding of language, and the ability to generate more nuanced and contextually appropriate responses. Command R+ is designed for more complex and specialized use cases that require an in-depth understanding of language, such as generating long-form content, summarization, question-answering, and language generation for specific domains or industries. It is best suited for complex RAG workflows and multi-step tool use.
- New Meta Llama 3 model with fine-tuning
The OCI Generative AI service now also supports the Meta Llama 3 70-billion parameter model in general availability. This model offers improved capabilities of reasoning, code generation, and instruction compared to Llama 2.
In addition to the Cohere Command models, you now can fine-tune the Llama 3 models with your own training data. The Llama 3 pretrained model supports the Low-Rank Adaptation (LoRA) fine-tuning method, which makes fine-tuning large models more efficient. When fine-tuning a model with the LoRA method, you don’t need to update all the original parameters of the model. Instead, LoRA adds smaller matrices to the model that transform the inputs and outputs. Only these smaller matrices are updated during fine-tuning, which helps improve accuracy while using less memory. The original model parameters stay frozen, and you can merge the small, adapted matrices back to the original model or keep them separate if you want to fine-tune them for other use cases without forgetting previous tasks.
- OCI Generative AI chat API and playground
OCI Generative AI is also introducing a new chat experience, enabling seamless integration into Cohere Command R, Cohere Command R +, and the Meta Llama 3 70-billion parameter models. We’ve added chat capabilities to the OCI Generative AI API that match what our model partners provide out of the box today, including Tool Use when working with the Cohere Command R and R + models. With Tool Use, you can now create API payloads (JSONs with specific parameters) based on user interactions and chat conversational history. You can use these payloads to instruct other applications or tools. For example, you can instruct an application to automatically categorize and route support tickets to the appropriate individual.
The generative landscape is changing at a rapid pace. We’re committed to delivering the latest features to help ensure our customers have the latest capabilities. You can now use an OCI Generative AI chat API to more easily create a chatbot-like experience with AI models like the new Command R, R +, and Llama 3 models. You can also now fine-tune Llama 3 models with your training data, enabling even greater model customization to address your business challenges.
Enchant Apps has a team of Oracle Cloud experts across all modules. If you are interested in learning more or have a project where you need Oracle cloud expertise, feel free to reach out to us!