AI can sort incoming email and support tickets into the right queue in seconds, but a frontier model such as GPT or Claude is often the most expensive way to do it. When the categories are stable and you have a few hundred labelled examples, a small fine-tuned model matches or beats a prompted frontier model at 20 to 200 times lower cost per message. When there are no labels yet, or the categories change every week, a prompted large model is the right place to start. Either way, what decides accuracy is the category list itself: built from real messages, written as decision rules with hard examples, and never invented in a meeting.
What does AI email and ticket triage do?
AI triage reads each incoming message and makes three quick decisions: which category it belongs to, which details to pull out, such as an order number or a customer code, and whether it is sure enough to act or should pass the message to a person. The category decides the queue, the priority and often the reply template. Classification picks from a closed list you define; extraction fills fields that may or may not be present. Both should return a fixed structure rather than prose, as explained in structured outputs, and pulling fields from attached invoices and forms is covered in AI document processing.
How do you design the categories?
You design the categories from a sample of real messages, not from memory. Take a few hundred recent emails, have the person who handles them write a short note on each, and group the notes into labels. A category list made in a meeting captures the common cases people remember and misses the long tail, where the overlaps live. Then write each label as a rule:
- Apply when: an observable property of the text, for example "the customer says a specific charge is wrong".
- Do not apply when: the neighbouring label it gets confused with, and what separates them, for example "asks what a charge is for, without saying it is wrong".
- Two to four hard examples from real messages, each with its correct label and a short reason.
Add a priority order for messages that fit two labels, and an "other" category with a target rate. If "other" grows beyond roughly 10 to 15% of traffic, the list is missing a category, not the model failing.
Is a large language model the best classifier?
A large language model is rarely the best classifier once labelled examples exist. A 2026 comparison on four standard benchmarks found fine-tuned small encoder models within 1.6 points of prompted GPT-4o and Claude Sonnet 4.5 where the large models won, and up to 3.3 points ahead where they lost, at roughly 0.005 to 0.03 US dollars per thousand items against 0.19 to 2.70 dollars, and faster (Valdes Gonzalez, 2026). Another study found a decades-old method, Naive Bayes, reaching 89.1% on a news classification benchmark, level with a zero-shot 27-billion-parameter model at 89.0% and ahead of a far larger one at 84.8%, running on an ordinary processor (Sada et al., 2026). The large model's real advantage is when you have no labels at all.
When should you use a large model, and when a small one?
Use the choice that fits your volume, your labels and how often the categories change:
- No labelled examples, categories still moving, under about 100,000 messages a month: a prompted large model.
- Stable categories, 500 or more labelled examples, high volume: a fine-tuned small model, or a simple classifier on top of text embeddings.
- Labels that need real reading comprehension or an external rule: a large model, and accept the cost.
The practical hybrid starts with the large model labelling messages, people correcting its labels, and a small model trained on the corrected set, which then handles the confident majority while the large model takes what it is unsure about. My own assistant works this way at its front door: a 5-million-parameter model I trained from scratch recognises 35 request types in Greek and English and runs on a phone (BSLM). More on small models in small language models.
How do you know when the AI is unsure?
You know it from a threshold set on your own held-out messages, never from the model's word alone. A model's stated confidence ranks messages usefully, but it is not a calibrated probability, and it shifts when the prompt wording changes. Ask for a coarse confidence level such as high, medium or low, sort a held-out set by it, see how accuracy falls as coverage grows, and choose the review rate your team can staff. Everything below the threshold goes to a person, and every corrected message becomes a new labelled example. How to build that held-out set is in LLM evals.
What does sorting a thousand emails cost?
Sorting a thousand emails costs from a fraction of a cent to about a dollar, depending on the system. By my arithmetic at September 2026 prices, for a 600-token email and a one-word label: about 1.30 US dollars per thousand on Claude Sonnet 5 called one by one, about 0.65 on the smaller Claude Haiku 4.5, about 0.33 through the batch interface, which all three major providers discount by 50% (Anthropic, batch processing), and about 0.10 when the shared instructions are also cached. A fine-tuned small model costs 0.005 to 0.03 dollars. For email that does not need an instant answer, batching is the first saving, not the last.
What is different about Greek email?
Greek email mixes writing styles more than English: many customers write in Greeklish, drop accents, switch to English for product names or write in capitals. Put all of these into the labelled examples and the test set, not only clean Greek, and choose a multilingual model for the small classifier. The same normalisation rules that help Greek search, such as removing accents for matching while keeping the original text, help classification too, as covered in RAG in Greek. Sorting is the first step of an AI agent for customer support, which then answers the routine messages and drafts the rest.
Frequently asked questions
How many labelled examples do I need?
A few hundred good examples are enough for a solid small classifier on a stable category list. Start with a large model to draft the labels and have a person correct them, which is faster than labelling from scratch.
Can the same AI also answer the emails?
It can draft answers, which is a separate step with its own checks. Sorting decides where a message goes; answering needs the right data and usually a person's approval, as described in AI customer support for an e-shop.
What happens when the categories change?
A prompted large model adapts the moment you change the list. A small trained model needs new examples and retraining, which takes minutes for a simple classifier. Watch the "other" rate: when it rises, the list needs a new category.
If your team sorts email or tickets by hand, send me your current categories and a month's volume. Whether a small model, a large one or both fits usually shows from those two facts.