Based on NVIDIA CEO Jesen Huang’s commentary on the Role of Databases for the Agentic Era in his GTC 2026 keynote. The diagram below is a readable version of Jensen's "Best Slide"; the content is created using LLM from the talk's transcript and then edited.
Summary of the Talk [wrt Databases]
For a database audience, the keynote underscores a fundamental shift: data is no longer just stored and queried—it is continuously activated to power agentic systems. The talk highlights that the center of gravity is moving from traditional transactional and analytical databases toward AI-driven data platforms that unify structured, unstructured, and real-time data streams into a single operational fabric. Massive growth in AI infrastructure—driven by data center expansion and trillion-dollar-scale compute demand—signals that data systems must scale not just for queries, but for continuous inference and agent workflows. A key theme is the rise of “agentic architectures,” where data is embedded, indexed, and retrieved dynamically to support reasoning systems, making vector search, hybrid retrieval, and multimodal indexing first-class primitives. The keynote also implicitly reframes databases as part of an end-to-end AI stack—tightly coupled with compute, models, and orchestration layers—rather than standalone systems. This creates pressure on databases to evolve toward low-latency, context-aware, and semantically rich data access patterns. Finally, the emerging paradigm suggests that competitive advantage will hinge on how effectively platforms can turn raw data into actionable intelligence for autonomous agents, blending transactions, analytics, and AI into a unified system of execution.
Intro
Back in the early 90s, when databases were ruling the world. Client-Server architecture was the mainframe modernization approach. Oracle, Informix, Sybase SQL Server(!) were ruling the database world. The big war between the big three was TPC-C and TPC-H – T for Transactions. Today, it’s T for Tokens! The goal of every enterprise application is to get the job done for customers and consumers. The consumers have embraced the Token computing, aka Generative AI. One of the big themes of Jensen Huang’s 2026 GTC keynote is that the five-layer cake of computing is to enable applications - both consumer and enterprise applications. Jensen said, behind all these applications are modern databases and the unreasonably effective SQL. While SQL has been extended for semi-structured data (with SQL++), SQL and the databases have to go beyond their roots and handle unstructured data - pure text, PDFs, images, and more. That will truly help enterprises bring the power of tokens to consumers. In essence, Jensen calls for databases to natural! It’s now early in the second quarter of the 21st century. Time for databases to be reinvented. Again.
Abstract
Enterprise computing has long been built around transactions – boolean algebra over ACID. In other words, deterministic units of work executed over structured data. Recent advances in AI, however, introduce a complementary unit of computation: tokens. Tokens represent language, context, and reasoning, enabling systems to interpret unstructured data and perform multi-step inference.
This article examines the emerging duality of tokens and transactions, based on Jensen Huang’s 2026 NVIDIA GTC keynote. He argues that databases, data systems, and data platforms are central to — from systems of record to systems of cognition that actively participate in reasoning, retrieval, and agent execution loops. All to enable intelligent consumer and enterprise applications. That’s really the job to be done.
We explore architectural implications, evolving database responsibilities, and key design challenges for building enterprise systems that integrate deterministic transactions with probabilistic token-based computation.
1. Introduction: A Shift in the Unit of Compute
Enterprise systems have historically been defined by transactions and analytics (via data warehousing). A transaction encapsulates a unit of business logic—atomic, consistent, isolated, and durable. Analytical databases (data warehousing engines) are optimized for analyzing large volumes of data to discover business insights. Both operational and analytical databases relational; both support transactions; both support SQL. Operational databases are optimized for executing and managing such transactions efficiently and reliably. And yet, modern AI systems operate on a fundamentally different unit: tokens.
Tokens are the atomic units of language models. They represent fragments of meaning—words, subwords, or symbols—that collectively encode information, context, and reasoning. Every AI interaction—prompt ingestion, retrieval, reasoning, and response generation—is expressed in tokens.
Recent developments suggest a structural shift:
AI systems increasingly perform multi-step reasoning, not single-step queries
Workloads are iterative and probabilistic, not deterministic.
Data consumption extends beyond structured tables to multimodal data. E.g. data in PDFs, spreadsheets, images, etc.
How do enterprise systems reconcile transaction-based computation with token-based reasoning? How do you reconcile deterministic computing we’ve been used to with the probabilistic computing of the generative AI era?
2. The Duality: Transactions and Tokens
Transactions remain the backbone of enterprise systems, enforcing data integrity and consistency, supporting auditability and compliance, and representing the authoritative record of business truth. They underpin core operations such as updating account balances, recording orders, and maintaining inventory—activities that demand precision and reliability.
Tokens introduce a fundamentally different paradigm in computing. Rather than representing state, they capture context and meaning, enabling systems to perform approximate reasoning instead of exact computation. They make the applications human. Tokens operate over unstructured and multimodal data, allowing AI systems to interpret natural language, summarize documents, generate insights, and plan and execute actions. As recent industry developments suggest, AI workloads have reached an “inference inflection,” where real-time usage increasingly dominates over training.
A critical misconception is that tokens will replace transactions. In reality, transactions define ground truth, while tokens interpret and act on that truth. Together, they form a complementary foundation for a new class of systems—one that integrates deterministic execution with probabilistic reasoning, enabling enterprises to both preserve correctness and unlock intelligence.
3. The Changing Role of Databases
The shift to token-based computation is fundamentally redefining the role of databases—from systems that answer queries to systems that construct context. Traditionally, databases were optimized to retrieve precise results, such as returning revenue for a given region. In AI-driven systems, however, the objective is no longer just row retrieval but context assembly: gathering relevant documents, retrieving semantically similar records, and combining structured and unstructured data into a coherent input for reasoning. This additional requirements requires databases to evolve by adding vector indexing, hybrid query execution that blends structured predicates with semantic search, and pipelines that dynamically assemble context for downstream AI processing.
At the same time, databases are evolving from query-answer systems into participants in a broader question-answer system. The answer doesn’t not necessarily come from databases systems and its boolean logic. AI systems introduce multiple layers of memory, including durable storage (traditional databases), semantic memory (embeddings and vector indexes), and short-term memory (model context such as KV cache). Agentic workloads place intense pressure on these layers due to frequent context reuse and expansion—a phenomenon often described as “memory pounding.” As a result, databases must support low-latency access across memory tiers, integrate with context caching mechanisms, and manage stateful sessions and reasoning histories that persist across interactions.
Perhaps most significantly, databases are transitioning from passive data stores to active participants in execution workflows. In traditional architectures, applications query databases and receive results. In agentic systems, AI agents orchestrate workflows, invoke databases as tools, and embed data access directly within multi-step reasoning loops. Increasingly, database interactions are expected to be performed by agents rather than humans, which requires databases to expose programmatic, agent-friendly interfaces, support semantic abstractions beyond SQL, and enforce policy-aware access controls. In this new paradigm, the database is no longer just a backend component—it becomes an integral part of the reasoning and execution fabric of enterprise systems.
4. Architectural Implications
The integration of tokens and transactions leads to a layered architecture that separates truth, retrieval, and reasoning into distinct but tightly coupled components. At the foundation lies the ground truth layer, consisting of OLTP databases, data warehouses, and dataframes, whose primary responsibility is to maintain correctness, ensure consistency, and provide authoritative data. This layer continues to anchor enterprise systems, serving as the definitive source of record against which all higher-level reasoning must be validated.
Above this foundation sits the semantic retrieval layer, which enables systems to bridge structured and unstructured data. This layer includes vector databases, hybrid indexing systems, and multimodal data pipelines that support semantic search and retrieval-augmented generation. Its role is to unify disparate data forms and assemble relevant context for downstream processing. As the scale and complexity of data grow, technologies such as GPU-accelerated data processing—exemplified by libraries like cuDF and cuVS—are emerging to address the performance and freshness challenges inherent in this layer.
At the top of the stack is the reasoning and agent layer, where intelligence is applied and actions are executed. This layer comprises language models, orchestration frameworks, and tool-using agents that interpret user intent, perform multi-step reasoning, and drive decisions and workflows. Correspondingly, the data flow within enterprise systems evolves from a simple query-response model into a continuous loop—context is assembled, retrieved, reasoned over, acted upon, and fed back into the system. This shift introduces iterative execution patterns, dynamic query generation, and cross-system orchestration, fundamentally transforming how applications interact with data and how value is created.
5. Data Evolution: From Structured to Multimodal
A significant shift in modern data systems is the elevation of unstructured data from passive archives to active participants in computation. Historically, structured data was queryable and operationally central, while unstructured data—documents, images, videos, and logs—was largely stored and seldom analyzed at scale. Today, AI systems can parse, embed, and index this unstructured data, transforming it into searchable and usable information. As a result, the vast majority of enterprise data—often estimated at 80–90%—can now be incorporated into query and reasoning workflows, fundamentally expanding the scope of what databases must manage.
This transformation requires databases to evolve beyond traditional tabular models and support multimodal data processing. Systems must now integrate document storage, embedding pipelines, multimodal indexing to unify structured and unstructured data, and importantly, right SQL language extensions and optimizer improvements to execute queries on this data efficiently . The database is no longer just a repository of records; it becomes a platform for representing meaning across diverse data types, enabling richer and more comprehensive insights.
6. Querying in the Agentic Era
Query interfaces are undergoing a fundamental transition from declarative queries to intent-driven interactions while still being declarative. While SQL remains essential, users increasingly interact with systems through natural language, expecting not just answers but outcomes. Natural language interfaces can translate requests into SQL or workflows, but simple NL-to-SQL translation is insufficient for complex enterprise scenarios that require reasoning, iteration, and orchestration.
In this new paradigm, query execution is increasingly mediated by agents rather than direct user input. Agents decompose tasks, select appropriate tools, execute multiple queries, and synthesize results into actionable outputs. For example, instead of executing a single query to retrieve revenue, an agent might detect anomalies, gather relevant data, correlate with external signals, and generate recommendations. This shifts the role of querying from a single-step operation to a multi-step reasoning process embedded within workflows.
Supporting this evolution requires a new approach to query planning and optimization. Future systems must operate across structured predicates, vector similarity search, ranking models, and the cost of token generation. This introduces challenges such as cost-based optimization across heterogeneous modalities, dynamic query planning that adapts to context, and execution strategies that balance performance, accuracy, and cost in real time.
7. Performance and Cost Considerations
“There are three things important in the database world: performance, performance, and performance” – Bruce Lindsay
Token-driven systems introduce new dimensions of performance that extend beyond traditional database metrics. One of the central trade-offs is between latency and context: larger context windows improve reasoning quality but increase both latency and computational cost. As a result, system design (includes embedding generation, vector index options, hybrid index, hybrid search, quantization methods, similarity search) must carefully balance the depth of context with effectiveness, efficiency, and responsiveness.
At the same time, new throughput metrics are emerging that redefine how performance is measured. Metrics such as tokens per second, time to first token, and tokens per watt reflect the operational realities of AI-driven systems. The notion of the data center as a “token factory” captures this shift, emphasizing the production and delivery of intelligence rather than just query execution.
These workloads also place unprecedented pressure on storage and indexing systems. Frequent context retrieval, large working memory requirements, and repeated access patterns stress storage architectures in ways that differ from traditional workloads. Additionally, vector indexes must be continuously maintained—rebuilt, incrementally updated, and aligned with evolving datasets—making freshness and maintenance efficiency as critical as query performance.
8. Security and Governance
Agent-driven systems introduce new categories of risk that require a rethinking of security and governance. Autonomous data access, cross-system interactions, and external communication expand the attack surface and increase the complexity of enforcing policies. As a result, security can no longer rely solely on perimeter-based approaches.
Instead, security must move closer to the data itself, becoming context-aware and policy-driven with guardrails. This includes row-level and field-level access control, as well as policies that govern what data can be accessed, under what conditions, and by which agents. Systems must enforce these constraints dynamically, ensuring that data access remains compliant even as workflows become more autonomous and distributed.
In parallel, hardware-level protections are emerging to strengthen data security. Confidential computing technologies enable data to remain encrypted even during processing, ensuring that sensitive information is protected not only from external threats but also from infrastructure operators. Together, these advances redefine the database as a critical enforcement point for both security and governance in AI-driven systems.
9. Open Challenges and Research Directions
The convergence of tokens and transactions introduces a range of open challenges that span systems design, optimization, and governance. One of the most pressing issues is the development of unified cost models that can optimize across SQL queries, vector search, and model inference. Traditional query optimizers are not equipped to handle the interplay between data retrieval and token generation, necessitating new approaches that consider both data and compute costs holistically.
Another critical challenge lies in defining coherent memory abstractions. Systems must unify session memory, persistent storage, and reasoning context without compromising performance or isolation. This requires new models for managing state across interactions, ensuring that context can be reused efficiently while maintaining consistency and security.
Additional challenges include indexing for multimodal data and governing agent behavior. Maintaining freshness, efficiency, and hybrid queryability across rapidly evolving datasets is non-trivial, particularly when combining structured and unstructured data. At the same time, ensuring safe execution, traceability, and compliance for agent-driven workflows demands new frameworks for governance and observability.
10. Conclusion: Toward Agentic Data Systems
The emergence of tokens as a unit of computation does not diminish the importance of transactions; rather, it creates a new synthesis. Transactions anchor truth, providing the deterministic foundation upon which enterprises operate, while tokens enable reasoning, allowing systems to interpret, infer, and act. Databases must evolve to support both, bridging the gap between correctness and intelligence.
The future enterprise data platform will integrate multiple capabilities into a unified system. It will combine structured and unstructured data, support hybrid retrieval and reasoning, participate actively in agent execution loops, and enforce governance and policy at every level. This represents a significant evolution from traditional architectures, requiring databases to operate as both data stores and intelligent substrates.
In this context, databases are transitioning from systems of record to systems of cognition. The challenge—and opportunity—for database systems is to become the foundation on which intelligent systems reason, act, and learn. Those that successfully make this transition will define the next generation of enterprise computing.
Closing Reflection
The key architectural question is no longer how to query data efficiently, but how to make data continuously usable for reasoning, action, and autonomous systems. This shift reflects a broader transformation in enterprise computing, where the value of data is no longer defined solely by retrieval speed or storage efficiency, but by how effectively it can be interpreted, contextualized, and acted upon in real time.
The answer lies in embracing both tokens and transactions—not as competing paradigms, but as complementary foundations. Transactions provide the deterministic backbone that ensures correctness and trust, while tokens enable the flexible, probabilistic reasoning required for intelligence and adaptation. Together, they form the basis of systems that can both preserve truth and generate insight.
The future of enterprise systems will be defined by how well this integration is achieved. Organizations that successfully unify transactional integrity with token-driven reasoning will be able to build platforms that are not only data-driven, but intelligence-driven—capable of supporting autonomous systems that continuously learn, adapt, and act.
References:
Comments
Post a Comment