Skip to content
MiraiTech
AI Lab

Experiment · Agents

Voice-to-SQL agent

Whisper turns speech into text, GPT-4o writes the SQL through function calling, the query runs against the database, and the answer comes back in plain language. Queries are checked against the schema before they run.

Type
Agents
Status
Live
Models
GPT-4o, Whisper Large-v3

Results

  • 94% query accuracy
  • 1.2 s from speech to result
  • Queries checked against the schema

Query accuracy was 94% across 150 benchmark questions. Voice to result took 1.2s on average (300ms speech-to-text, 600ms LLM, 200ms execution, 100ms rendering). All 8 test users preferred voice to writing raw SQL for exploratory analysis. No injection attempt succeeded in adversarial testing.

What we tested

This experiment tests whether a voice interface lets non-technical people explore a database on their own. You ask a question such as "What were our top 5 customers by revenue last quarter?" and get the SQL, the results and a plain-language explanation, in under 2 seconds from the end of speech.

Method

The pipeline uses Whisper Large-v3 for speech-to-text, GPT-4o with function calling to generate SQL and a React frontend to show the results. We tested it on 150 benchmark questions covering single-table lookups, multi-join aggregations, date-range filters and ambiguous phrasing. Each question was asked by 3 different speakers to see how it handled accents and speech patterns. The model gets the schema from CREATE TABLE statements and sample rows in the system prompt.

Findings

  1. 01

    94% query accuracy on structured questions

    With the schema in the prompt, GPT-4o wrote correct SQL for 94% of the benchmark questions. Most failures involved vague time references ("recently", "a while ago") or implicit joins.

  2. 02

    Schema in the prompt beat fine-tuning

    CREATE TABLE statements plus 3 sample rows per table gave higher accuracy than a model fine-tuned on 500 query pairs, and the approach adapts to a new schema immediately.

  3. 03

    People asked 3x more questions by voice

    Whisper adds about 300ms of latency. In user testing, non-technical users asked 3x more exploratory questions by voice than through a text SQL interface.

  4. 04

    Function calling blocks SQL injection

    The model can only call predefined query functions with validated parameters and never writes raw SQL strings. That removes SQL injection as an attack path.

How it is built

The browser captures audio with the Web Audio API and streams it to the FastAPI backend. Whisper transcribes it, and the transcript goes to GPT-4o together with the schema and function-calling tools defined for SELECT queries. The generated SQL is checked against a read-only allowlist of operations and run against the database. The results come back with a plain-language summary from a second LLM call.

Whisper Large-v3 handles speech recognition with word-level timestamps. GPT-4o generates SQL through function calling, with Pydantic-validated schemas. FastAPI runs the backend with connection pooling to SQLite, which can be extended to Postgres. The React frontend records audio with the Web Audio API and charts results with Recharts.

  • Whisper
  • FastAPI
  • SQLite
  • React
  • LangChain

Problems we hit

Ambiguous time references

"Last quarter" depends on the fiscal calendar. We added a configurable fiscal calendar to the context, and the agent asks a clarifying question when a time reference is ambiguous.

Follow-up questions

A follow-up such as "Now break that down by region" depends on the previous query. The agent keeps the last 3 query contexts so it can resolve references like "that".