How AI is changing fraud detection in audit

From manual balance sheet review to graph analytics: what we automate today and what still requires expert judgement.

Artificial intelligence does not detect fraud: it detects deviations from a learned pattern. The gap between the two is precisely the space in which the auditor works. This article does not repeat that “AI makes it possible to analyse 100% of the population”. It focuses on what that analysis requires in order to produce evidence: how models behave when fraud is rare, why published metrics tend to be inflated, what a graph sees that a balance sheet does not, and where the standards require us to keep thinking.

The base rate problem: an excellent model that is almost always wrong

When fraud is rare, the vast majority of alerts from any model are false positives, even if the model is very good. This is arithmetic, not technology. The probability that an alert corresponds to actual fraud (the positive predictive value, PPV) depends on three quantities: sensitivity s, specificity e and prevalence π.

\mathrm{PPV} = \frac{s\,\pi}{s\,\pi + (1-e)(1-\pi)}

Consider a general ledger of 1,000,000 entries, 100 of which are fraudulent (π = 0.01%), and a model with 95% sensitivity.

Specificity	Frauds detected	False positives	Alerts that are fraud (PPV)
99%	95	9,999	0.9%
99.9%	95	1,000	8.7%
99.99%	95	100	48.7%

Three consequences follow from this that are rarely explained.

• The ROC curve AUC is misleading. It is insensitive to class imbalance: a model with an AUC of 0.95 can be useless on a real ledger. The relevant metrics are the area under the precision–recall curve and, above all, precision in the top k alerts.
• The right question is one of capacity. The audit team can investigate a finite number of items. A model is not judged by “how often it is right”, but by how many frauds there are among the 50 or 200 items that will actually be examined. AI is a ranking problem, not a classification problem.
• Anomaly is not fraud. Well-designed fraud seeks to look normal. Unsupervised models mostly find errors, process exceptions and unusual but legitimate transactions. That has value, but it is not the same thing, and the audit report cannot confuse the two.

Journal entries: from rules to local anomalies

The journal entry testing required by NIA-ES 240 remains, on many engagements, a set of filters: weekend postings, round amounts, unusual users, post-closing entries. It has two weaknesses. Filters combine poorly (each criterion added either multiplies the number of items or reduces it to zero) and they are predictable: management knows what is being filtered.

Benford, properly applied. Benford’s law can only be expected to hold for quantities that span several orders of magnitude and arise from multiplicative processes. A complete ledger mixes processes that by design do not follow it: payroll, calculated depreciation, fixed instalments. That is why a global analysis usually returns a “non-conformity” with no meaning whatsoever. It should be applied by stratum (process, account, user) using Nigrini’s mean absolute deviation (MAD) threshold: for the first digit, anything above 0.015 is non-conforming. And what is most useful to the auditor is usually not the first digit, but the duplicate-amount and last-digit tests, which expose transactions split just below authorisation limits.

Global and local anomalies. Unsupervised learning changes the approach: instead of defining what is suspicious, it learns what is normal. Schreyer et al. (2017) trained deep autoencoders on real ledgers extracted from SAP (307,457 and 172,990 lines) and introduced a distinction that should be part of every auditor’s vocabulary:

• Global anomaly: a value that is rare in itself, such as an account that is hardly ever used or an atypical posting time. An isolation forest or a rule finds it without difficulty.
• Local anomaly: a rare combination of values that are common individually. A regular user, a regular account, a regular period, but that user never posts to that account against that offsetting account. This is the typical signature of management override of controls, and no univariate filter can see it.

The results call for humility: F1 scores of 32.9 and 16.9 on injected synthetic anomalies, not on real frauds. Better than the alternatives, but far from the “100% of the population analysed” that is so often promised.

Three technical pitfalls.

1. The unit of analysis is the journal entry, not the line. The information lies in double entry: which accounts move together. A debit to receivables with a credit to revenue is trivial; the same pair, posted manually by a treasury user on the last day of the financial year, is not. A model that scores isolated lines loses precisely that structure.
2. Recurring fraud is learned as normal. If the model is trained on the year under audit and the manipulation is repeated every month, the fraudulent pattern becomes part of “normal”. Models must be trained on prior periods or on comparable entities, and the content of the baseline must be monitored.
3. Concept drift. An ERP change, a reorganisation or a new business line will trigger alerts with no fraud involved. The model needs maintenance, and that maintenance must be documented.

Financial statements: lessons from the Bao case

The classic benchmark is the Beneish M-score (1999): eight indices built on year-on-year changes (days sales in receivables, gross margin, asset quality, sales growth, depreciation, SG&A expenses, leverage and total accruals), with an indicative threshold of −1.78. It was followed by the F-score of Dechow et al. (2011). Both share one premise: the analyst decides which ratios matter.

Bao, Ke, Li, Yu and Zhang (2020), in the Journal of Accounting Research, challenged that premise. They trained an ensemble model (RUSBoost, with random undersampling to compensate for the rarity of fraud) on 28 raw financial statement items, with no ratios. The labels were the SEC enforcement releases (AAERs) between 1991 and 2008. And they evaluated performance using NDCG@k on the top-scoring 1% of firms, consistent with the capacity logic described above.

Model (test period 2003–2008)	Variables	AUC	NDCG@k
Logistic regression	14 ratios	0.672	0.028
SVM with financial kernel	28 raw items	0.626	0.020
RUSBoost	28 raw items	0.725	0.049

The usual takeaway was that machine learning on raw data outperforms expert-designed ratios. The most instructive part came later. Walker (2021) replicated the published code and found that the treatment of serial fraud failed in 17 of the 413 enforcement cases. The same fraudulent firm appeared in both training and test sets under different identifiers, so the model “recognised” cases it had already seen. Ten of the 16 published hits came from those 17 cases. Once the error was corrected, the positive predictive value fell from 4.5% to 2.5%, below the 2.63% of the benchmark logistic regression. The authors published an erratum.

For an auditor, the case offers four lessons that apply to any commercial tool:

• Leakage between training and test data is the most common and most costly error. With panel data, validation must be temporal, with a gap between periods (Bao left two years), and by entity, not by observation.
• Even without errors, the ceiling is low. With a prevalence of around 0.7% of firm-years, getting 1 in every 25 flagged firms right was already a publishable result.
• The labels are detected frauds, not committed frauds. The model learns what the SEC pursues. Fraud that was never detected is excluded from the sample by construction.
• Spain has no equivalent to the AAERs. For unlisted companies there are not enough labels to train proprietary supervised models. The realistic approach is to combine unsupervised detection, comparison with similar entities and sector knowledge.

Graphs: what a balance sheet cannot show

A balance sheet aggregates; fraud rarely sits in a balance and almost always sits in a relationship. A graph represents entities as nodes and relationships as edges, and enables questions a table cannot ask: who is connected to whom, in how many steps and through which intermediaries.

The ledger as a network. Boersma (KPMG, 2019) proposed modelling the general ledger as a network with two types of node: accounts (what moves) and entry patterns or business processes (why it moves). Every company has a stable topology: revenue reaches receivables through a small number of paths. A new path into revenue, or a process that only appears at closing, is a structural anomaly that no amount-based test will detect.

The third-party network and related parties. Suppliers, customers, employees and directors are connected through shared attributes: IBAN, address, telephone number, email domain, common directors. Add the appointments published in the BORME (Spain’s Official Companies Register Gazette) and links two or three steps removed emerge that management has not disclosed. This is the technical answer to a structural problem in NIA-ES 550: the auditor depends on a list of related parties provided by the very party that has an interest in leaving some out. Metrics such as betweenness centrality point to bridge nodes, and community detection algorithms (Louvain, Leiden) isolate groups that trade almost exclusively among themselves.

Circularity. Fictitious revenue generated through circular transactions (A sells to B, B to C and C back to A, with the money travelling the same cycle) is, formally, a directed cycle in the transaction graph. The difficulty lies in searching for cycles that respect time, i.e. with increasing dates and amounts preserved except for a small commission, rather than simple topological cycles, which abound in any commercial network. Tax authorities already apply graph representation learning to dismantle circular invoicing schemes, as in the study on GST in the Indian state of Telangana. It is the same pattern as intra-EU VAT carousel fraud.

The bottleneck: entity resolution. Before analysing a graph, one has to decide that “Construcciones García, S.L.” and “CONSTRUCCIONES GARCIA SL” are the same node. This decision is not neutral. A wrong merge creates a path that does not exist and, therefore, a false link; a missed merge hides a real one. The similarity threshold is a judgement with consequences for the evidence and must be documented as such. On top of this there is the GDPR: matching employees’ addresses against suppliers’ addresses requires a legal basis and a proportionality assessment.

Text and language models: where they add value and where they mislead

Linguistic signals are real, but weak and fragile. Larcker and Zakolyukina (2012) analysed analyst conference calls. CEOs of companies that later restated their accounts used more words of extreme positive emotion and fewer words of anxiety; CFOs used more negations. Their models outperformed chance by 6–16%, comparable to accounting-based models. The problem is that these signals degrade as soon as they become known: executives receive communication training, and many corporate texts are now drafted, in part, by a language model. An indicator that the observed subject can manipulate stops measuring what it used to measure.

Where an LLM really adds value: contracts versus accounting records. The clearest value lies not in “detecting lies”, but in reading at scale what used to be sampled: customer contracts, minutes, amendments. The question is specific: are there clauses that alter revenue recognition (rights of return, consignment arrangements, bill-and-hold, variable consideration) that the accounting records do not reflect? Revenue fraud often lives in that gap between what the document says and what the journal entry says.

Why its output is not, on its own, evidence.

• The false negative is the dangerous part. A model hallucination is usually caught when reviewing the output. A clause the model “fails to find” is invisible. Before relying on an extraction, its completeness must be measured against a manually reviewed sample.
• Non-determinism and reproducibility. The same query can produce different results, and models are updated. NIA-ES 230 requires that an experienced auditor be able to understand what was done and with what result. This means documenting the model version, instructions, parameters and input documents.
• The auditee controls the input data. This is a new risk: a document provided by the entity may contain hidden text with instructions aimed at the model (prompt injection). In a fraud context, the prudent assumption is that someone will try.

The PCAOB, in its July 2024 spotlight, described firms’ use of generative AI as still mostly administrative. It stressed that responsibility for the work remains with the engagement team and identified the auditability of source data and output as a central concern.

The regulatory fit: the standards already ask for the hard part

The NIA-ES neither prohibit nor require AI, but several of their requirements clash with how a model works. These are best resolved when designing the work, not when documenting it.

Unpredictability versus determinism. NIA-ES 240 requires an element of unpredictability in the selection of procedures. A model that is identical every year, with the same variables and the same threshold, is exactly the opposite: management can learn what it does not flag. The elegant solution is statistical: treat the model score as a stratification criterion.

1. The highest-risk stratum (the top-scoring k items) is examined in full.
2. The rest of the population, which the model considers normal, is sampled randomly.
3. Errors found in that second stratum make it possible to estimate the model’s own failure rate and project it in accordance with NIA-ES 530.

The model thus stops being a black box that one trusts and becomes a sampling design with valid inference. The random component also provides the unpredictability the standard requires.

Integrity of source data. Before any model, NIA-ES 500 requires the auditor to evaluate the reliability of information produced by the entity. In practice: reconcile the extracted ledger with the trial balance, check the continuity of journal entry numbering and verify that the extraction includes manual entries, closing entries and entries from sub-ledger systems. An excellent model on an incomplete ledger produces a false conclusion with the appearance of rigour.

Risk, IT environment and response. NIA-ES 315 (revised) already addresses automated tools and techniques in risk assessment. And NIA-ES 240 retains two presumptions that no model can switch off: the risk of management override of controls and the risk of fraud in revenue recognition.

Document how each alert is closed. The real work is not generating alerts, but closing them. NIA-ES 230 requires the auditor to be able to reconstruct why each flagged item was considered explained and on what evidence. Closing 400 alerts with “reviewed, no exceptions” is worse than not generating them at all: it documents that the auditor had the information and did not use it.

What lies ahead. In July 2025 the IAASB published the revised ISA 240, effective for audits of periods beginning on or after 15 December 2026. It strengthens the “fraud lens” in risk assessment and responses, expands transparency in the auditor’s report for listed entities and links it to ISA 570 on going concern. Its application in Spain will depend on the adaptation published by the ICAC.

What we automate today and what still requires judgement

The dividing line does not separate simple tasks from complex ones. It separates what can be defined in terms of the data from what requires understanding incentives, context and what is not recorded.

Area	What we already automate	What still requires expert judgement
Journal entries	Scoring 100% of the ledger, detecting global and local anomalies, digit tests by stratum	Deciding what is normal for this entity this year and closing each alert with evidence
Financial statements	Manipulation indicators, comparison with similar entities	Interpreting a high indicator in a company that is growing, restructuring or under financial strain
Third parties	Resolving entities, building the graph, detecting cycles and communities	Judging whether an indirect link is an undisclosed related party or a coincidence
Documents	Extracting clauses from thousands of contracts and comparing them with the accounting records	Assessing verbal agreements, side letters and anything not in writing
Estimates	Retrospective review of prior estimates against actual outcomes	Detecting management bias accumulated across individually reasonable estimates
Planning	Stratifying the population by risk and measuring the model’s failure rate	Formulating specific fraud hypotheses based on incentives, pressures and opportunities

Three limits are not technological and will not go away:

• Collusion makes fraud look normal. When several people coordinate the scheme, the data is internally consistent and there is no anomaly to detect. Only confrontation with external, independent evidence can break it.
• Bias in estimates is not an outlier. Ten reasonable estimates, all at the favourable end of the range, amount to manipulation that no anomaly model will flag, because each one on its own is normal.
• Scepticism cannot be delegated. A model suggests where to look; it does not decide which of management’s explanations is insufficient. That decision, and the responsibility that comes with it, remains with the auditor.

AI does not replace the experienced auditor: it requires them to be even more so. Anyone who does not understand the base rate, data leakage or entity resolution will not know when a tool is giving them evidence and when it is merely giving them the appearance of rigour.