Last updated: September 2026
Here is an uncomfortable claim, stated plainly: you should stop using a chat AI to produce your engineering numbers. Not stop using it entirely, and not because it is useless, but because the one job you are most tempted to hand it, the actual calculation, is the one job it is architecturally worst at. A language model predicts text; it does not compute. It will hand you a clean, confident, wrong number in the same voice it uses for a right one, and on engineering work a silently wrong number is the expensive kind. Use AI to set up and explain the problem, then let a real computational tool do the arithmetic, and verify.
This is the pointed version of the argument. For the balanced capability answer see whether AI can do engineering calculations, and for the wider situational guide see when not to use AI in engineering. It sits under our roundup of the best AI tools for engineers.
Short answer: Stop letting the language model be the thing that produces the number. It represents numbers digit by digit and predicts the next token rather than calculating, so it fails at exact arithmetic in a way you cannot prompt away, and multi-step reasoning is a known weak point. In practice the failure is the worst kind: an answer that is almost right, delivered with full confidence. The fix is a role split. Use AI to parse the problem, pick the approach, and explain the result; hand the numeric solve to a code interpreter or a computational engine, which beats the model’s own reasoning on math benchmarks; and verify every load-bearing number against an independent check. AI belongs around the calculation, never inside it.
The claim, stated plainly
You can use AI around the math. You should not let the raw language model be the thing that produces the number. That distinction is the whole argument: the setup, the explanation, and the sanity-checking are language tasks a model is genuinely good at, while the numeric solve is a computation task it is built badly for. Treat the model’s arithmetic as a draft to be checked, never as the result.
Why the arithmetic step fails, and why you cannot prompt it away
This is not a bug that a cleverer prompt fixes; it is how the model is built. A language model predicts the next token from patterns, so it produces what a plausible answer looks like rather than computing what the answer is. Recent interpretability work found that large models represent numbers with a separate per-digit encoding in base ten, and their errors are distributed across the digits of the answer rather than clustered near the correct value (Levy and Geva, 2024). In other words, the model encodes what numbers look like, not what they are worth. On top of that representation problem, multi-step reasoning is a documented bottleneck: the GSM8K benchmark of 8,500 grade-school word problems was built precisely because even the largest models “still struggle to robustly perform multi-step mathematical reasoning” (Cobbe et al., 2021). And the failure gets worse with units: work decomposing math word problems found that models “still encounter difficulties in handling numeral and measurement conversions,” and that minor changes in numbers or units can drastically alter their performance (NUMCoT, 2024), which is the mechanism behind why ChatGPT gets units wrong.
What actually breaks in practice
Engineering-grade problems break fastest. A 2025 benchmark of engineering tasks found that model accuracy declines as complexity rises, degrades under minor perturbations, and remains substantially below human performance, leading the authors to conclude current models lack the high-level reasoning needed for real-world engineering (EngiBench, 2025). The dangerous part is the shape of the failure. In the 2025 Stack Overflow Developer Survey, the single most common frustration, cited by 66 percent of developers, was AI output that is “almost right, but not quite” (Stack Overflow, 2025). An almost-right number does not announce itself; it passes review and lands in the design. What one silent unit error can cost is not hypothetical: NASA lost the Mars Climate Orbiter in 1999 because one system produced impulse values in pound-force-seconds while the navigation software expected newton-seconds, a mismatch that ended a mission that cost about 327 million dollars (NASA Mishap Investigation Board). That was a human software error, not AI, but it is exactly the class of quiet numeric mistake an unverified AI calculation invites.
The reframe: use AI to set up and explain, not to compute
The fix is not to abandon AI but to give it the right job. Let the model do the language work: restate the problem, choose the governing equations, lay out the method, and explain the result in context. Then hand the actual arithmetic to a tool that computes deterministically, a code interpreter running Python or a computational engine. This is not a stylistic preference; it measurably works. Program-aided approaches that offload the solve step to a Python interpreter beat a large model’s own chain-of-thought reasoning on the GSM8K math benchmark by about 15 percentage points (Gao et al., PAL, 2022), because a real interpreter does the computing the model cannot.
A reliable workflow: setup, compute, verify
Put the three roles in order and the risk drops sharply. Use AI to translate the problem and structure the method. Move the arithmetic to a computational engine or a code interpreter that actually calculates. Then verify every load-bearing number against an independent tool or a hand check before it drives a decision. That last step is not optional busywork; it is the reason the workflow is trustworthy. The practitioner data backs the caution: in the same 2025 survey, only about 3 percent of developers said they highly trust the accuracy of AI output, while roughly 46 percent actively distrust it (Stack Overflow, 2025). Those are people who use these tools daily, and they still do not trust the numbers. Neither should you, until a deterministic tool has produced them and a second check has confirmed them. The habit of treating confident output as unverified until proven is the same one we argue for in AI hallucination in engineering.
Frequently asked questions
Should I stop using ChatGPT for engineering math?
For the actual numeric solve, yes. A language model predicts text rather than computing, so it produces confident, sometimes wrong numbers. Use it to set up the problem, choose the method, and explain the result, but run the arithmetic on a code interpreter or a computational engine and verify the answer. Treat any number the model itself produces as a draft to be checked, never as the result.
What should I use instead for the calculation?
A code interpreter running Python, or a computational engine such as a solver or computer algebra system. Program-aided approaches that offload the solve to a Python interpreter outperform a model’s own chain-of-thought reasoning on math benchmarks by a wide margin, because a real interpreter computes deterministically. The model can still write and explain the calculation; it just should not be the thing executing it.
Is it ever OK to let the model compute directly?
Only for throwaway estimates you will independently verify, and never for load-bearing or safety-relevant numbers. Even then, be aware that small changes in the numbers or units can flip the result, and that an almost-right answer is the most common and most dangerous failure mode. If a number matters, it belongs on a deterministic tool with a verification step.
Why does it get simple arithmetic wrong so confidently?
Because it represents numbers digit by digit and predicts the next token rather than calculating, so it generates a fluent answer with the same confidence whether that answer is right or wrong. Its errors are spread across the digits rather than landing near the true value, which is why the mistakes can look plausible. Confidence in a language model is a property of its fluency, not evidence that the number is correct.
Sources
- Levy and Geva, per-digit number representations in LLMs, arXiv 2410.11781 (2024)
- Cobbe et al., GSM8K, arXiv 2110.14168 (2021)
- NUMCoT, numeral and measurement conversion, arXiv 2406.02864 (2024)
- EngiBench, LLM performance on engineering tasks, arXiv 2509.17677 (2025)
- Gao et al., PAL: Program-Aided Language Models, arXiv 2211.10435 (2022)
- Stack Overflow 2025 Developer Survey (AI trust and frustration)
- Mars Climate Orbiter, NASA Mishap Investigation Board (unit-mismatch analogy)
Written by the CognitiveFuture editorial team. We build our guidance from peer-reviewed and preprint research and a large practitioner survey, each linked above; the Mars Climate Orbiter is cited only as an analogy for the cost of a silent numeric error, not as an AI incident. We do not independently benchmark tools, and we treat every AI-produced number as unverified until a deterministic tool computes it and a qualified engineer confirms it.
Richard Johnson
Richard Johnson is an AI specialist at one of the world's largest technology companies, where he has spent the past three years helping organizations adopt AI. CognitiveFuture extends that work publicly: gathering the available evidence on each tool, from vendor documentation to independent reviews and user feedback, and cutting a crowded market down to the right choice for the job in front of you.
