Introducing PromptQL: A Programmable API for LLMs Built on Hasura

Overview

After months of development focused on the challenges surrounding AI and data architecture, the team behind Hasura is excited to announce PromptQL, a new tool and runtime designed specifically for Large Language Models (LLMs). PromptQL builds a programmable API on top of the graph structures already created with Hasura, enabling AI application developers to build robust applications quickly.

The Problem: Data Architecture and AI Performance

The development of PromptQL was driven by two critical realizations regarding current AI limitations:

1. The Limitation of Direct Reasoning: Assistants perform significantly better when tasked with writing programs to solve problems rather than reasoning through problems directly.
2. The Risk of Hallucination: Existing AI models often perform poorly on real-world problems, leading to a lack of user trust. When an AI provides a poor answer or no answer, it requires extensive human intervention. While architectures like RAG (Retrieval-Augmented Generation) and purpose-built tools have emerged to address this, they often lack standardization.

The Solution: PromptQL and Hasura DDN

Hasura addresses these issues by leveraging its metadata-driven approach to building APIs. Originally known as the GraphQL on Postgres tool, Hasura has significantly generalized its capabilities since its early days, including the launch of Hasura DDN (Data Discovery Network).

This metadata-driven approach is uniquely suited for LLMs because it allows developers to:* Build Quickly: Create APIs that are structured and clear enough for LLMs to understand.* Standardize Data Access: Provide data that can be queried in a standardized way, similar to how humans use GraphQL or REST, but now accessible to LLMs.* Create External Memory: Move data out of the LLM's context window to reduce the risk of hallucination. This is achieved by giving the LLM an external memory in the form of stored artifacts.

Key Capabilities

PromptQL allows users to build data architectures without boundaries. The system supports:* Diverse Data Sources: Users can store data in various locations without restriction.* Custom Vector Databases: Integration with custom vector databases featuring specialized search mechanisms.* Fine-Tuned Models: Support for incorporating fine-tuned LLMs into the architecture.

Community Feedback and Adoption

The announcement has received positive feedback from the developer community, particularly those familiar with Hasura's backend capabilities:* Production Viability: Users with experience managing production codebases on Hasura view it as a superior option to competitors like Supabase and Payload CMS for building backends. The structured nature of Hasura has been credited with significantly increasing developer productivity, allowing teams to onboard new members much faster.* Integration with PostgreSQL: Community members are interested in how PromptQL interacts with native PostgreSQL AI extensions like pgai. While some users plan to combine both technologies to maintain a stable, familiar stack, others are exploring whether Hasura's new features might render additional extensions obsolete.* Future-Proofing: There is a strong desire for a technology stack that remains stable and does not require frequent changes every time a new vector database is announced.

Conclusion

PromptQL represents a shift in AI architecture, prioritizing good data architecture as the foundation for good AI architecture. By providing a programmable API that leverages Hasura's graph capabilities, it aims to solve the trust and performance issues currently plaguing AI assistants, allowing developers to build reliable AI applications with less human intervention.