What a Real Applied AI Case Looks Like (Not a Demo)
By Anis Hammouche·July 6, 2026
A real applied AI case runs on your real files, every week, and moves a number you were already tracking. A demo works once, on a clean example. Three signs tell them apart: a precise task at the start, real data with its exceptions, and a result you measure in your own business.
Take Sofia, a freelance accountant (an invented case, like every example in this article). She sees a reel where AI reads an invoice and fills in a spreadsheet in a few seconds. She tries it on a clean invoice: perfect. Then she runs the 40 supplier invoices from the month. One scan is crooked, one invoice runs over two pages, one line has two VAT rates, one invoice is in English. She rereads everything and the evening is gone. The demo did not lie. It just picked its example.
Sign 1: it starts from a precise task
A demo starts with the tool: "look what it can do". A real case starts with a problem on the ground: "this task costs me time every month". That changes everything, because you know what the tool has to improve, and you will be able to say whether it did.
The test is simple. Can you name the task in one sentence, with its trigger and its result? "Every month I enter supplier invoices in my tracking spreadsheet" is a task. "I would like to be more efficient with AI" is a wish, and nobody can check that it has been granted. If the answer stays vague, you are holding a demo dressed up as a project.
Sign 2: it holds up on your real files
This is the sign that eliminates the most demos. A demo uses chosen data: clean, complete, no surprises. Your day-to-day contains something else. Crooked scanned PDFs, empty fields, clients who write in two languages, odd cases you know by heart and the tool does not.
Anthropic's documentation on evaluations gives the same advice to people who build tools: design your tests to mirror the real task, and do not forget edge cases. You can do the same without writing a line of code. Take ten of your real files, three of which already give you trouble by hand, and run them all. AI can also produce a wrong but plausible result, as the CNIL (the French data protection authority) points out. That is why you reread the output, not only the input. I go through that habit in AI Invents Figures, Sources and Terms.
Sign 3: you measure the result, and it runs every week
A demo is judged by the effect it has in the moment. A real case is judged by what changes in your calendar or your books: time saved, fewer errors, more files handled. Without a measure you do not know whether you gained anything, and the tool ends up forgotten in a browser tab.
There is a second, quieter test: does it actually run? A tool you relaunch when you think of it, that you have tried twice, is not an applied case. The day it is part of your week, with its own slot in your calendar, it is. The full method for counting what it earns you is in AI ROI: How to Measure What a Tool Really Earns You.
Real case or demo: how to decide
| Criterion | Real applied case | Demo |
|---|---|---|
| Starting point | A named task, with its trigger | The tool, "look what it does" |
| Data | Your files, exceptions included | A chosen, clean example |
| Justification | Time, errors or files, before and after | An impression, "it feels modern" |
| Frequency | It runs every week | It ran once, in front of you |
| Proof | You can rerun it on a new file | A video or a screenshot |
Here is how that looks by trade, for three invented businesses:
| Business | What the demo shows | What decides |
|---|---|---|
| Freelance accountant | One clean invoice read in a few seconds | The 40 invoices of the month, crooked scans included, without corrections |
| Three-person web agency | The minutes of a calm ten-minute meeting | Minutes of real client meetings, with interruptions, with no rewrite |
| Self-employed electrician | A quote generated in one click | Quotes sent without him fixing a price or a condition |
The left column impresses and shows well on video. The right one takes longer to check, but it is the only one that pays. A demo is not useless: it can open a lead. The mistake is to take it for a result.
A half-hour test before you commit
Copy this sheet and fill it in before you invest time or money in a tool you saw in a demo:
Precise task: [what the tool must do, in one sentence]
Trigger: [what starts the task, for example an invoice arriving]
Files tested: [10 real files, 3 of which already cause trouble]
Passed without correction: [number out of 10]
Corrections needed: [type of error and time to fix]
Time before, without the tool: [minutes per file]
Time after, review included: [minutes per file]
Verdict: [I continue / I narrow it / I stop]
Frequently asked questions
Is a demo useless? No. It shows that something is possible, and that is often where an idea starts. The problem begins when you take it as proof that it will work for you. It says nothing about your data, your exceptions or how often you will use it.
Is a test that works on ten files enough? It is a good start, not a guarantee. It tells you whether the idea holds on a representative sample. Then look at a whole month: that is where rare cases show up. If you cannot reread the outputs, do not scale it.
How do I measure when the result is hard to put a number on? Narrow the task until a number appears: minutes per file, errors found, files handled in the week. A task too vague to be measured is often too vague to be automated.
Do you need a big company for a case to be "real"? No. Size does not change the logic. A freelancer who handles ten files a week has a real case as soon as the tool runs on those real files and the gain is measured.
Sources
- Anthropic, Define success criteria and build evaluations (documentation, read on November 1, 2026)
- CNIL, Les questions-réponses de la CNIL sur l'utilisation d'un système d'IA générative (in French)
The examples of Sofia, the agency and the electrician are fictional.
The next step is Automating Unpaid Invoice Reminders, a use you can run through this sheet. I share this kind of test in my newsletter, you can sign up right below.
