Text-to-SQL lets a person ask a question in plain language, in Greek too, and get an answer computed from the company database. On public benchmarks the best systems reached about 82% correct answers in September 2026, close to the 93% of human experts. On real company data warehouses, the best agent frameworks reached 10.8%. The danger is not an error message but a fluent wrong number that nobody can tell from a right one. For most businesses the answer that works is a curated layer: a small set of tested database views, or 15 to 40 ready reports where the AI only picks the report and fills in the dates and names.
What is text-to-SQL?
Text-to-SQL is a language model turning a question such as "how many orders did we ship to Thessaloniki last month" into a database query, running it and returning the result. It promises that managers can ask the ERP or the e-shop database anything without waiting for a report. The model has to know which tables hold the answer, how they join, which rows count, and what the question actually means, and each of those is a place to go wrong. If your data still lives in spreadsheets, the first step is getting it into a database, as argued in your spreadsheet is an ERP that is failing quietly.
How accurate is text-to-SQL?
Text-to-SQL is accurate on clean benchmark databases and much less accurate on real ones. On BIRD, the standard benchmark, the top systems reach about 82% execution accuracy against a human expert level of 92.96% (BIRD, September 2026). On BEAVER, built from real private data warehouse query logs, state-of-the-art agent frameworks using a current frontier model reached 10.8%, and 30.1% even with hints for each step (BEAVER). A production deployment reported by one large cloud provider started at 36.77% on real user questions and reached 53.61% after substantial engineering. Plan against something between 10% and 55% on your own raw schema, not against the benchmark figure.
Why do real databases break it?
Real databases break text-to-SQL because they carry knowledge that is not written anywhere the model can see:
- Cryptic column names: heavy abbreviation cut accuracy by up to 17% in a controlled study.
- Missing foreign keys: common in MySQL databases that grew from application code, such as many e-shops, where relationships live in the program, not in the database.
- Soft deletes: a column like
deleted_atthat every correct query must filter, which the question never mentions, so counts come out inflated. - Business conventions: status 7 means cancelled, amounts in cents in one table and euros in another, a fiscal year that does not start in January.
- Two sources for one number: a reporting table and the source tables, both correct, giving different totals.
The typical failure is a query that runs, returns rows and answers a slightly different question than the one asked. Why models produce confident wrong answers at all is covered in why AI hallucinates.
What makes text-to-SQL reliable?
What makes it reliable is giving the model a smaller, documented target than the raw database. In one 2026 study, a governed semantic layer, where the model picks from documented, tested building blocks and code assembles the query, answered 37 of 38 benchmark questions correctly on every run (97.4%) against 21 of 38 (55.3%) for free SQL generation, with no wrong-but-executed answers (semantic path compilation study). Another found that improving the description of the schema and its examples gained 12 to 25% accuracy, where tuning the retrieval machinery gained 3 to 12% (context artifacts study). On a PHP and MySQL stack the practical version takes about a week: 10 to 30 database views, each with a one-line business description, each already carrying the deleted-row filter, the right joins and any customer filter, each tested with a query whose answer is known, and a model allowed to query only those views.
Is it safe to let AI query your database?
It is safe only if the database enforces the limits, not the prompt. The user's question is untrusted input, and so are the rows the query returns, because text written by users can carry instructions to the model, as covered in prompt injection. The controls belong in the database: a dedicated read-only account that can see only the exposed views; customer and tenant filters built into those views rather than left for the model to remember; a time limit on every statement, which MySQL sets in milliseconds for SELECT statements only and MariaDB in seconds for almost any query (MariaDB, aborting statements); a read replica instead of the database that serves checkout; and a log of every generated query with the question and the user. One MariaDB detail causes silent wrong numbers: it does not enforce strict grouping by default, so turn on ONLY_FULL_GROUP_BY for the model's session. How agent access should be scoped in general is covered in AI agent permissions.
When are ready-made reports the better answer?
Ready-made reports are the better answer when most questions repeat, which in most businesses they do. Collect 100 real questions from the people who will ask them. If 80% or more fall into fewer than 40 query shapes, build those as tested reports with parameters, such as date range, customer and status, and let the model only choose the report and fill the parameters. That turns a hard generation task into an easy selection task, accuracy on the covered questions approaches 100%, and a question outside the set is refused instead of answered wrongly. The numbers an owner actually needs every morning are usually few, as argued in the three numbers an owner needs at 8am, and choosing instead of writing is the idea behind structured outputs.
What changes when the questions are in Greek?
When the questions are in Greek, the model copes with the language; the data is the harder part. Current models turn a Greek question into SQL about as well as an English one. The trouble is Greek values in the database: whether "Αθήνα", "ΑΘΗΝΑ" and "Αθηνα" match depends on the table's character set and collation, which decide whether comparisons ignore accents and case. Use utf8mb4, choose the collation deliberately, and normalise names on the way in. The same accent problem in search, with a measured example, is in an AI knowledge base without a vector database. Answers computed from one set of numbers everyone trusts are the core of AI in operations and supply chain.
Frequently asked questions
Can managers ask the ERP questions directly?
Yes, if every answer comes with the filters the system understood, such as the date range and the status, so a wrong interpretation is visible. Without that, a wrong number looks exactly like a right one.
Do I need a data warehouse first?
Not for a small or medium business. A set of well-described views on a read replica of your main database is enough to start, and it is where most of the accuracy comes from anyway.
Should the AI be allowed to change data?
No. Text-to-SQL for questions should be read-only by design. Changes to data belong to the application, with its own validation and approvals.
If your team waits for someone to pull numbers from the database, send me twenty questions they asked last month. How many fit a handful of reports, and how many need real query generation, is usually clear from that list.