Databricks launches autonomous data AI agent

Databricks has launched Genie Code, an autonomous AI agent able to carry out tasks like building pipelines, debugging failures, shipping dashboards and maintaining production systems.

Genie Code is a new addition to its business intelligence product Genie, which lets any knowledge worker chat with their data and get answers instantly using the context and semantics captured by Unity Catalog; Genie Code extends this approach to data professionals.

Genie Code handles full ML workflows end-to-end, reasoning through complex problems to plan, write, and deploy models, while logging experiments to MLflow and fine-tuning serving endpoints for peak performance.

Genie Code accounts for the differences between staging versus production environments, building workflows for change data capture and applying data quality expectations. It also monitors Lakeflow pipelines and AI models in the background to triage failures and investigate anomalies, autonomously analysing agent traces to fix hallucinations and tuning resource allocation before a human intervenes.

Integrated with Unity Catalog, Genie Code enforces existing governance policies and access controls, understanding business semantics and audit requirements and federates enterprise data, including data from external platforms.

Through persistent memory, it also automatically updates internal instructions based on past interactions and coding preferences, and on real-world data science tasks, Databricks claimed that Genie Code more than doubled the success rate of leading coding agents (from 32.1% to 77.1%)..

"Software development has shifted from code-assistance to full agentic engineering in the past six months,” said Ali Ghodsi, co-founder and CEO of Databricks.

“Genie Code brings this revolution to data teams. We're moving from a world where data professionals are assisted by AI to one where AI agents do the work, guided by humans. We are calling this Agentic Data Work. It will fundamentally change how enterprises make decisions.”