To serve over 1 billion weekly users, OpenAI leverages a custom-built online storage platform called Habitat and a new productivity tool known as the Data agent in ChatGPT Work. Habitat is the globally distributed storage layer that handles more than 70 million requests per second, while the Data agent allows enterprise users to connect directly to corporate data sources like Snowflake and Databricks to generate insights using natural language.

The Backbone of OpenAI: Scaling Habitat to 1 Billion Users

Every action within an OpenAI product—from logging in to starting a new conversation—requires rapid, reliable access to data. Habitat was developed to decouple storage logic from product engineering, allowing developers to focus on features rather than database management.

Initially launched at DevDay 2023 as a simple Python library for GPTs, Habitat has evolved into a massive distributed system managing over 500 petabytes of data across 40 geographic regions.

Moving from a Library to a Centralized Service

By mid-2025, the original client-side Python implementation reached its limits. Managing updates across dozens of microservices became a bottleneck. To solve this, OpenAI transitioned Habitat into a standalone service. This centralization provided:

• Single point of control: Easier deployments and observability.
• Data Security: Centralized enforcement of access control and audit logging.
• Stability: Isolation of storage logic from application bugs.

The Strategic Pivot: From Python to Rust

OpenAI initially accepted the performance overhead of Python to prioritize development speed. However, as the service scaled to 20 million requests per second, managing tail latencies and CPU bottlenecks became critical. In Q2 2026, using Codex and GPT-5.5, OpenAI engineers rewrote the entire Habitat service in Rust.

Putting Data to Work: The New ChatGPT Data Agent

While Habitat powers the backend, the newly released Data agent in ChatGPT Work empowers users to interact with their own enterprise data. Instead of waiting for data science teams to build reports, employees can now ask plain-language questions to generate interactive dashboards and actionable insights.

Seamless Enterprise Integration

The Data agent acts as a bridge between the user and trusted corporate data sources. It supports a wide array of platforms, including:

• Cloud Warehouses: Amazon Redshift, Google BigQuery, Snowflake.
• Operational Databases: MongoDB, ClickHouse, Redis.
• Collaboration Tools: Google Drive, SharePoint.
• BI Ecosystems: Tableau, Microsoft Power BI, Sigma.

Fusing Intelligence with Efficiency

The Data agent does not just retrieve data; it interprets it using the organization's specific business terms and metric definitions. By integrating with semantic layers like dbt or Databricks Genie, the AI ensures that its answers are grounded in the company's unique context.

Engineering for Reliability at Scale

Scaling to 70M+ requests per second required solving complex distributed systems problems, such as metastable failures. OpenAI engineers discovered that default Last-In-First-Out (LIFO) connection reuse in Python’s aiohttp could cause traffic to concentrate on slow or overloaded pods. By patching the connection pool to use First-In-First-Out (FIFO) reuse, they broke the feedback loop and stabilized the service.

Furthermore, OpenAI uses Envoy for connection fan-in, upgrading Python’s HTTP/1 connections to HTTP/2 multiplexing. This prevents "thundering herd" problems where downstream resources are flooded during deployments.

Frequently Asked Questions (FAQ)

What is OpenAI Habitat?

Habitat is OpenAI's internal online storage platform. It provides a NoSQL API (inspired by TAO) that abstracts away database management for product engineers, handling authorization, encryption, and routing for over 500PB of data.

How does the Data agent protect sensitive company data?

Queries made by the Data agent enforce the existing permissions of the connected account. This includes table, row, and column-level restrictions, ensuring users only see the data they are authorized to access.

Why did OpenAI choose Rust over Python for Habitat?

While Python allowed for rapid prototyping, Rust offered significant performance gains. The Rust-based version of Habitat is 6x more CPU efficient and 15x more memory efficient, which was necessary to handle the growth to 1 billion users.

Can I use the Data agent to build dashboards?

Yes. The Data agent can create interactive dashboards within ChatGPT or publish views directly to third-party tools like Tableau and Power BI using natural language prompts.