AI summaries in client reports, without made-up numbers

An AI summary in a client report is a few sentences, written by a language model from that period's numbers, that tell the client what happened and what comes next. It saves the account manager the slowest part of reporting, but only if every number in it is checked against the data before it is sent, because language models sometimes state figures that are not in their input.

This guide covers what the summary should say, why models get numbers wrong, a method that catches it, and how to keep a human in the loop without losing the time saved.

What the summary should say#

The summary replaces the first page of the four-part client report. Four short parts, in this order:

Example numbers, made up for illustration.

Two rules make it trustworthy. "What we did" must come from a log of real changes, never from the model's idea of what an agency usually does. "What's next" must come from your plan, not from generic advice.

Why models invent numbers#

A language model writes the most likely next words. Asked to summarize a month, it may write "conversions rose 18 percent" because that sentence is plausible, even when the data says 12. NIST calls this confabulation: confidently stated but erroneous or false content (NIST). OWASP lists the same failure, under misinformation, among the main risks of AI applications, and recommends automatic validation of key outputs and human oversight (OWASP).

Common forms in reports:

• A percentage computed wrongly from two correct numbers.
• A number from the wrong period, last month's figure presented as this month's.
• A rounded number that no longer matches the table under it.
• A cause that is not in the data: "thanks to the holiday season" when nothing says so.
• A change the agency did not make, invented to fill "What we did".

One wrong number in front of a client costs more trust than the summary saves time.

A checking method that works#

Treat the model as a writer, never as the source of facts.

1. Compute every number first, in code. Totals, changes, percentages, the best campaign. The model receives finished facts, not raw rows to do arithmetic on.
2. Write a plain version from rules. A template summary filled in with the facts is always correct, if dull. It is the fallback.
3. Ask the model to rewrite the plain version in better words, with the same facts.
4. Check every number in its answer against the facts and the plain version. One number that is not there, and the whole answer is dropped for the plain version.
5. Check claims of action. If the answer says something was done and the change log is empty, drop it.
6. Keep site text out of the instructions. Campaign and page names come from visitors and ad platforms and can carry hostile text. Mark them as data, and reject an answer that contains a link or address that was not in the facts.

This catches made-up numbers. It cannot catch a correct number with a misleading explanation, which is why a person still reads the summary.

Keep a human in the loop#

Three modes fit most agencies:

Whatever the mode, the account manager should be able to edit the words. The best summaries add one sentence only the person on the account knows: "The owner was away for two weeks, so calls went to voicemail." (Example.)

What data goes to the model#

A summary needs aggregate numbers and campaign names, nothing personal. Do not send customer names, emails or phone numbers to a model to write a report. Check where your reporting tool's AI provider processes data, and whether your client contracts say anything about it.

Should you tell clients the summary is AI-written?#

If an account manager reads and stands behind every summary, it is the agency's report. If summaries go out automatically, say so in the report or in the contract. Clients forgive a tool. They do not forgive finding out.

Where Sporetag fits#

From Sporetag's report summary code and help, on the Studio and Agency plans: every client report opens with What happened, Why, What we did and What's next, written from that period's numbers. What we did comes from changes logged in Sporetag; What's next from the site's Autopilot plan. With the Assistant on, a model rewrites a rule-written draft, using one AI action per report period, and every number in its answer must be in the data or the plain version goes instead. Automatic sends it, Check first emails you and holds the report until you approve, and you can edit the words in either mode. The Highlights block is separate and uses no AI.

Common questions#

Can AI write my client reports?

It can write the summary at the top from numbers you give it. Compute the numbers in code, have the model write the words, and check every number in its answer before the report goes out.

Why does AI get numbers wrong in reports?

Language models predict plausible words. They can compute a percentage wrongly, mix up periods or state a figure that sounds right but is not in the data. NIST calls this confabulation.

How do I stop an AI summary from making up numbers?

Give the model finished numbers instead of raw data, compare every number in its answer with the facts, and fall back to a plain rule-written summary when one does not match.

Should a person review AI-written client reports?

Yes for new clients, large accounts and bad months. For routine months with automatic number checks, a quick read before sending is enough.

Sources#

• NIST: AI 600-1, Artificial Intelligence Risk Management Framework: Generative AI Profile (July 2024). Checked October 2026.
• OWASP Gen AI Security Project: LLM09:2025 Misinformation. Checked October 2026.
• The checking method is how Sporetag's report summary works; figures marked as examples are made up for illustration.