Nvidia's Open-Source Agent Framework Aims to Connect Enterprise Data Without Competing With Platforms
At GTC Berlin, Nvidia unveiled Agent Knowledge Foundation, a collection of open-source components designed to help AI agents understand enterprise data while leaving control in the hands of existing data platforms like Snowflake and Databricks.
Nvidia introduced Agent Knowledge Foundation on Wednesday at its GTC Berlin conference, positioning the toolkit as a way for enterprises to deploy AI agents that can work across their data infrastructure without displacing the platforms already managing it. The announcement reflects a broader challenge facing large organizations: connecting artificial intelligence tools to company data in ways that respect existing security and access controls.
The toolkit bundles several components, including Nvidia's Nemotron Retriever models for search, a new Auto-Ontology library for mapping data to business concepts, the Kumo family of models for structured data analysis, and CUDA-X data processing libraries. According to Adel El Hallak, Nvidia's vice president of product for agentic AI, the system allows data to remain in place under existing access rules rather than requiring migration or centralization.
"I wish it was as easy as connected to my Snowflake system, connected to my email exchange, connected to my knowledge base, and then all of a sudden, magically, the AI understands all the data. But the reality is different," El Hallak tells The New Stack. Large language models struggle with context when faced with complex enterprise data structures, he explains, even when connected to individual data sources.
Better retrieval, fewer lookups
Nvidia's approach prioritizes embedding and reranking models that improve search accuracy, even if they introduce slight latency. El Hallak argues that more precise retrieval reduces the overall number of lookups an agent must perform, ultimately lowering end-to-end latency. In testing on OpenSearch, Nvidia's Nemotron Retriever raised search relevance by 65%, requiring fewer search steps with minimal latency overhead. SAP's fine-tuned version of the reranker achieved nearly three times faster performance, while ServiceNow reported better reranking quality with close to 25% less latency.
From RAG to ontologies
Retrieval-augmented generation, or RAG, retrieves text chunks based on similarity, but Nvidia argues that knowledge graphs better represent concepts and their relationships. The Auto-Ontology library constructs these graphs from a company's tables, mapping raw data to business concepts such as customers, products, and orders. The library includes a REST API and a Model Context Protocol server that agents can query.
El Hallak notes that language models often conflate business terms that humans easily distinguish. "A tier is the same thing as a plan when you're looking at a mobile carrier," he says, "but LLMs mess those up." Nvidia acquired Illumex, an Israeli startup specializing in semantic layers and ontologies for enterprise data, for a reported $75 million in February. An Nvidia spokesperson told Globes at the time that the team's expertise would support the adoption of agentic AI in organizations.
Predictions from tables
Kumo, acquired by Nvidia in June in a deal valued by The Information at more than $400 million, developed foundation models for relational data. Nvidia has released several Kumo models as open weights on Hugging Face, including Kumo Tabular in September. El Hallak describes these models as replacements for the custom pipelines data teams historically built for individual tasks like classification, churn prediction, or sentiment analysis.
"The whole point of the ontology … is to bring harmonization as you create those knowledge graphs, so there's [consistency] across the tables, and then you could use Kumo to drive predictive analytics," El Hallak says.
Partners, not rivals
Rather than positioning itself as a competitor to data platforms, Nvidia is framing partnerships as central to its strategy. "I'm not looking to compete with, for example, a Snowflake. On the contrary, I want to partner with them," El Hallak says. Nvidia's own supply chain team uses Agent Knowledge Foundation through Palantir's Foundry platform, building on prior collaboration between the companies. Databricks is integrating a Kumo anomaly detection model with Genie Ontology, its context layer introduced in June. Snowflake customers can run Kumo models in Snowflake Notebooks.
"I'm not thinking about ACLs and data access and all those things. That's why I need the partners," El Hallak says, referring to access control lists and other security features that enterprise partners provide.
Owning the stack
Not all vendors are pursuing this collaborative approach. SAP completed its acquisition of Prior Labs, the company behind TabPFN tabular models, in July and committed more than €1 billion to the investment over four years. SAP now offers TabPFN-3.5 Plus in SAP AI Core alongside its own RPT models, consolidating tabular AI capabilities within its platform.
Nvidia's strategy differs markedly. The company distributes its models through Hugging Face, which it agreed to acquire for $12.9 billion in September. "We just build little weights and models, and we publish them on Hugging Face," El Hallak says. This approach leaves Nvidia focused on model development while allowing Snowflake, Databricks, and other partners to maintain control over data and associated permissions.