Red Hat adds AI support for IBM Granite and Intel Gaudi
RHEL AI is meant for developing, testing and running GenAI models for enterprise applications
Red Hat’s latest release of its Enterprise Linux AI foundation model platform, RHEL AI, adds IBM Granite large language model support and comes with previews for code generation and function calling capabilities as well as Intel Gaudi 3 accelerator support.
The open-source tools vendor and IBM subsidiary’s latest version of RHEL AI—now generally available—is a prime area for Red Hat services partners and systems integrators “helping customers implement and integrate that into their use cases and business,” Joe Fernandes, Red Hat’s AI business unit vice president and general manager, told CRN.
“AI in general and generative AI in particular is complicated, so our services partners and systems integrators are the ones helping companies explore and integrate different use cases in a cost-effective way,” Fernandes said in an email.
“That maps directly to how Red Hat aims to reduce costs (with smaller models), eliminate complexity (around integrating those models with customer data and use cases) and enable flexibility (to deploy those models wherever they need to run across a hybrid environment).”
RHEL AI 1.3
Red Hat has been working to increase the overall percentage of company revenue that comes through the channel and improve partner profitability.
RHEL AI is meant for developing, testing and running GenAI models for enterprise applications, the vendor said in a statement Thursday.
Version 1.3 adds support for the latest innovations in parent IBM’s open-source licensed Granite LLMs and leverages open-source technology for data preparation.
RHEL AI brings together Granite with the InstructLab model alignment project created by Red Hat and IBM. Users can leverage the components for a packaged, bootable RHEL image for individual server deployments across hybrid clouds, according to Red Hat.
Version 1.3 supports Granite 3.0 8b English language use cases. This version has a developer preview for using this model’s non-English language features, plus code generation and function calling capabilities, with full support coming in a future RHEL AI release.
The new RHEL AI version also supports the Docling IBM Research open source project for converting common document formats into Markdown, JSON and other formats for GenAI applications and training. The new version can do context-aware chunking, accounting for the structure and semantic elements of documents used for GenAI training with the goal of improving GenAI responses.
RHEL AI added a technology preview for Intel Gaudi 3 accelerator support with this release and now supports parallelised serving across multiple nodes for multiple requests in real time, according to the vendor. Users can dynamically alter LLM parameters during serving.
Other capabilities slated for future RHEL AI releases will support additional document formats for Docling, integration for retrieval-augmented generation pipelines and InstructLab knowledge tuning.
This article first appeared on Computing’s sister site CRN