The Number Looks Right. That's the Problem.
Paste a column of ten invoice amounts into ChatGPT and ask for the total. You'll get back a number, formatted neatly, possibly with a pound or dollar sign, delivered with complete confidence. It will probably be wrong.
Not wildly wrong — not £47 when the real answer is £4,700. More likely it's off by £12.50, or it's skipped a row, or it's added a figure twice. The kind of wrong that passes a quick glance but fails an audit.
That specific failure — plausible-looking arithmetic presented beautifully — is the one this article is about. It's different from AI inventing facts it doesn't know (that's a separate problem), and different from AI giving you inconsistent answers on consecutive days. This one is structural. It comes from what a language model actually is, and once you understand it, the rule for handling it becomes obvious.
What AI Is Actually Doing When You Ask It to Add Up
A language model — the technology behind ChatGPT, Claude, Gemini, Copilot, and most of the AI tools you're likely using — was built to predict text. Feed it a prompt, and it generates the most plausible continuation, word by word (or more precisely, token by token). That's it. That is the entire mechanism.
It is extraordinarily good at this because it has processed an almost incomprehensible amount of human writing, and human writing has patterns. Sentences have grammar. Arguments have structure. Business emails follow conventions. The model has learned all of those patterns deeply enough to produce output that reads like an intelligent human wrote it.
Arithmetic is not a pattern. It is a procedure. The correct answer to 847 + 293 is not the most plausible answer given how similar-looking calculations tend to turn out — it is 1,140, full stop, arrived at by carrying a digit. A language model has no mechanism to carry digits. It has seen a great many arithmetic problems and their answers in its training data, so it has learned that certain kinds of problems tend to have certain kinds of answers. When the numbers are simple enough and common enough, it gets it right. When they're not, it generates a plausible-looking figure and presents it with the same confidence it would present anything else.
This is why the error is so dangerous: the formatting is impeccable. The answer sits in a table, or in bold text, or in a bullet list. It does not look like a guess. It looks like the output of a calculator. It is not.
The Situations You'll Actually Run Into
Here are the specific scenarios where this bites business owners.
Pasting a column of figures and asking for the total. This is probably the most common one. You copy revenue figures, invoice amounts, or expense line items into a chat window and ask for a sum. For a short, clean list of round numbers, the model will often be right. Add more rows, add decimals, add anything that requires carrying or borrowing — and the odds of a quiet error climb.
Asking for a margin or a per-unit cost. "If I'm paying £3.40 per unit and selling at £8.99, what's my gross margin?" is a multi-step calculation. The model has to get the subtraction right, then get the division right, then get the percentage right. Each step is an opportunity for a plausible-but-wrong generation. Compound calculations compound the risk.
Comparing two quotes. "Supplier A charges £0.85 per unit with a £150 delivery fee on a minimum order of 500. Supplier B charges £0.91 with free delivery and no minimum. Which is cheaper for an order of 300?" This involves conditional logic, multiplication, and comparison — exactly the kind of thing that looks simple but requires actual arithmetic at every step. A human can work it out with a calculator in 90 seconds. An AI will produce a confident answer that may or may not survive the same 90-second check.
Working out how many working days fall between two dates. Date arithmetic sounds trivial until you factor in weekends, and it immediately stops being a pattern-matching problem. Errors here mean missed deadlines built into contracts.
Totalling a table it has just extracted from a PDF or document. This one is particularly sneaky. You ask the AI to pull a table out of a long document, which it does impressively well. Then — often in the same breath — it totals the columns. The extraction might be accurate; the arithmetic on top of it might not be. You trust the whole output because the extraction was good.
Why the Same Question Can Be Right in One Tool and Wrong in Another
You may have noticed that some AI tools handle arithmetic reliably while others don't. This is not random variation — it reflects a design choice.
The better tools don't try to generate a numerical answer from the language model. Instead, when they detect a calculation, they stop and hand it off: either to a proper calculator function, or — more commonly — to a piece of code the model writes and then actually runs. Python, for instance, will carry digits correctly every time. So the model writes sum([847, 293]), executes that code, gets 1,140 back, and reports 1,140. The arithmetic is done by software that is designed to do arithmetic, not by a text predictor doing its best.
This is why you'll see some tools show something like "running code…" or "analysing with Python…" before giving you a number. That's a good sign. It means the answer came from a calculator, not a guess.
It's also why an answer that shows its working is worth considerably more than one that just states a figure. If the model writes out the individual steps — lists the rows it included, shows the intermediate totals — you can check each step. A bare number gives you nothing to verify against.
The article on the AI already sitting in your spreadsheet is worth reading alongside this one: the AI features built into Excel and Google Sheets tend to handle arithmetic more reliably precisely because they're operating inside software that calculates, rather than a chat window that predicts.
The Practical Rules
None of this means AI is useless for financial or numerical work. It means you use it for the right parts of that work.
Never accept a number you haven't spot-checked. This is the non-negotiable. It doesn't matter how confident the AI sounds, how neatly the figure is formatted, or how reasonable it looks at a glance. Pick two or three rows at random and verify them yourself with a calculator or a spreadsheet formula. If they're right, you can have higher (not absolute) confidence. If one is wrong, you know to check everything.
Ask it to show the steps and the source rows, not just the answer. Instead of "what's the total?", ask "list each line item and its amount, then show me the running total." This gives you something to audit. A confident wrong total is dangerous; a wrong subtotal you can spot is fixable.
Give it raw data rather than a screenshot. A language model reading numbers from an image is adding an extra layer of possible transcription error on top of the arithmetic error. Copy and paste actual numbers as text whenever you can.
Use it to explain and structure a calculation, while doing the arithmetic somewhere that actually calculates. This is the split that works well. Ask AI to tell you what formula you need — "what's the right way to calculate weighted average cost of goods sold?" — and then build that formula in a spreadsheet. You get the conceptual clarity from the model and the numerical accuracy from software designed for it.
Treat any figure destined for a customer, a tax return, or a bank as requiring human verification without exception. Full stop. Quote totals, invoice line items, VAT calculations, cash flow projections: if it comes from or passes through an AI, a human needs to check it before it goes anywhere that matters. This isn't about distrust of AI broadly — it's about knowing the specific failure mode of this specific technology.
The same principle applies to the broader question of where AI deserves your trust. The hallucinations piece — AI will confidently get things wrong — covers the factual equivalent of this problem: confident output that hasn't been verified against reality. Arithmetic errors and factual errors come from the same underlying mechanism; the difference is that arithmetic errors tend to be quieter and more trusted because the output looks numerical.
What AI Is Actually Good For in a Numerical Context
To round this out: here's where AI genuinely helps with numbers, even given everything above.
- Interpreting figures, not generating them. "Here's my gross margin percentage for the last three quarters. What might explain the trend?" is a great question. The AI is reasoning about numbers you've already calculated correctly, not doing the arithmetic itself.
- Structuring analysis. "What should I be looking at when comparing these two supplier quotes?" gives you a framework. You then do the maths.
- Explaining financial concepts in plain English. "What's the difference between gross margin and net margin, and which one should I be focused on?" is exactly the kind of question language models handle well.
- Catching missing variables. Describing a calculation you're planning to do and asking "what have I forgotten to account for?" often surfaces things like VAT, delivery costs, or setup fees that you'd have missed.
- Drafting the narrative around numbers. Writing the commentary that goes alongside a report, summarising what a set of figures means — that's text work, and language models are very good at text work.
The pattern is consistent: use AI for the words around the numbers, and use a calculator, a spreadsheet, or code-executing AI tools for the numbers themselves.
The arithmetic weakness is one of the most predictable things about current AI. That makes it, in a way, the easiest to manage. You don't have to distrust AI broadly — you just have to know this specific thing and build a habit around it.
If you want help thinking through where AI fits in your business and where the real risks sit, let's talk. A single conversation is usually enough to separate what's worth trying from what will waste your time and money.