Five techniques share the label "AI-powered". On ten hard names they contradict each other

"AI-powered" is not a technique. In watchlist screening it is a label sitting on top of at least five: edit distance (Jaro-Winkler, Levenshtein), phonetic coding (Double Metaphone, Soundex), transliteration tables (ICAO Doc 9303, BGN/PCGN), embedding similarity, and LLM adjudication. They are not stages of one method and they do not agree. On the ten name pairs computed below, one pair that is the same person scores 0.802 and is rejected at a 0.85 threshold, while a pair that is two different people scores 0.943 and is accepted. Only three of the five can be given a reproducible number at all; the other two cannot, and a vendor page that prints one for them is decorating.

Edit distance counts the single-character operations between two strings. Levenshtein counts insertions, deletions and substitutions. Jaro counts characters matching inside a sliding window, plus transpositions. Jaro-Winkler adds a prefix bonus, and the origin of that bonus matters: Winkler defined it in 1990 for the US Census Post Enumeration Survey, raising the score by 0.1 × (1 − score) for each of the first four characters that agree. The assumption is that in an American surname file, the first four characters are the part a keypunch operator gets right. In a sanctions file, the first character is precisely what a transliteration changes.

Phonetic coding reduces a name to a sound-shaped code. Soundex, per the US National Archives, produces a letter and three digits — the worked example on that page is W-252 — and was built so that SMITH and SMYTH file together in a census index. It is an English-surname instrument. Double Metaphone is far better travelled and returns two codes, a primary and an alternate, so one name can carry two pronunciations. Both are ASCII machines: feed them a national character and the behaviour is undefined, which the Łukasz row below shows in a way no amount of prose can.

Transliteration tables are not fuzzy at all. They are published, one-to-one mappings from a non-Latin script into Latin characters. ICAO Doc 9303 Part 3, Section 6, carries the tables states use for passport machine readable zones. What the standard is honest about, and what most screening copy is not, is that the mapping is often one-to-many: it recommends "AE or A" for Ä, "AA or A" for Å, "N or NXX" for Ñ. Doc 9303 also separates transliteration from transcription, the looser phonetic rendering that depends on the target language — its own example is Omar Khayyam in English against Omar Chajjam in German. BGN/PCGN romanisation, maintained jointly by the US Board on Geographic Names and the Permanent Committee on Geographical Names, is another table again. So the same Cyrillic or Arabic name legitimately reaches your file in several Latin spellings, none of them wrong.

Embedding similarity compares two names by the distance between vectors a model assigns them. It is the only one of the five that can learn that Bill is William, or that a company trades under a shortened style. It is also the only one whose number means nothing outside the model that produced it.

LLM adjudication does not compare strings at all. It reads a candidate in context — the listing text, the dates of birth, the nationality, the addresses — and writes down a judgement. It is a reviewer, not a scorer.

We computed every number below with rapidfuzz 3.14.6 (Jaro, Jaro-Winkler, Levenshtein), jellyfish (Soundex) and the metaphone package's doublemetaphone (Lawrence Philips' algorithm). Strings were lowercased; no other normalisation was applied, because the point is what the raw algorithms do. The Nordic and Polish surnames are given in their ASCII forms — the one exception is the Łukasz row, which keeps the stroke deliberately, because that row is about what happens when it survives to the algorithm. Levenshtein similarity is 1 − distance ÷ length of the longer string. The threshold column uses 0.85, a common default and not a recommendation.

The same pairs, phonetically. Codes are printed exactly as the libraries returned them, one code per whitespace-separated token; Double Metaphone returns a (primary, alternate) pair and the alternate is shown only where it is not empty.

Embeddings and LLM adjudication get no numbers in these tables, deliberately. An embedding score is a property of a specific model at a specific version; printing "0.91" for it here would imply a reproducibility that does not exist. Qualitatively: an embedding is the technique most likely to link Bill Whelan to William Whelan and Nagy Péter to Péter Nagy, because those relationships live in usage rather than in characters — and, for the same reason, the technique least able to tell you why it did. LLM adjudication, given the surrounding identifiers, is the only one of the five that can separate Sigurðsson from Sigurðardóttir on the correct ground — that the shared element is a patronymic and not a family name — and the only one that can flag Zhang Wei against Chang Wei as a romanisation question rather than a score.

The threshold is inverted, not badly set. At 0.85 the engine accepts Mohamed / Mohamud at 0.943 and Zhang Wei / Zhang Wen at 0.956 — two pairs a reviewer has to keep apart — while rejecting Muhammad / Mohamed at 0.802, which is one name in two transcriptions, and Nagy Péter / Péter Nagy at 0.300, which is one person written family-name-first as Hungarian writes it. The pairs it gets wrong score higher than the pairs it gets right. Moving the threshold does not fix that: lowering it to 0.80 admits Muhammad / Mohamed and also Sigurðsson / Sigurðardóttir, and it never reaches 0.300.

Some scores are not wrong; they are undecidable. Zhang Wei against Chang Wei returns 0.926, and no number can settle it. Chang is the Wade-Giles romanisation of the same Chinese surname Pinyin writes as Zhang, and the Pinyin romanisation of two different surnames. Whether 0.926 is a true match depends entirely on which romanisation system each source used — a fact that lives in the provenance of the record, not in the string. This is the case where a higher score would make the file worse, because it would hide a question that has to be asked.

The prefix bonus lands on the wrong pairs. Compare the Jaro and Jaro-Winkler columns. Where the first characters differ — Aleksandr / Oleksandr, Zhang Wei / Chang Wei, Łukasz / Lukasz once the stroke is counted as a different character — the bonus is exactly zero, because the technique has nothing to reward. Where it is largest, +0.107, is Sigurdsson / Sigurdardottir: the shared prefix is Sigurd, the father's given name, so the algorithm's strongest vote of confidence is cast on a pair that is by construction a brother and a sister. The 1990 census assumption has not travelled.

Two edit-distance metrics can disagree by 0.46. Kowalska-Wiśniewska against Kowalska returns 0.884 on Jaro-Winkler and 0.421 on normalised Levenshtein. Length sensitivity is the whole difference. A team that says "we use fuzzy matching at 0.85" has said nothing until it says which metric.

Diacritics fail silently, not loudly. Łukasz against Lukasz passes on Jaro-Winkler at 0.889 and looks fine. Underneath, Soundex returned Ł220 against L220 and Double Metaphone returned KS against LKS — the Ł was dropped entirely, because the algorithm is ASCII and had nothing to do with it. Fold the character first and both codes agree exactly. ICAO's table says Ł transliterates to L; if that fold does not happen before the phonetic pass, the phonetic pass is doing harm rather than nothing.

Company names are a different problem wearing the same clothes. Orion Shipping Ltd against Orion Shipping Limited scores 0.964, and Orion Shipping Ltd against Orien Shipping Ltd — a one-letter typo in the distinctive part — scores 0.974. The higher score is the less interesting match. Legal-form tokens are noise carrying most of the string length; strip them into a normalised form before scoring, and score the distinctive element separately, or the metric spends its resolution on "Ltd".

There is no universal weighting. There is a defensible one for a given customer base, and the honest version is a policy you can show a supervisor.

Two mechanical rules do more work than any weighting. First, expand before you score: a query name and a listed name should each become the set of spellings the published tables permit, and the score should be the best over that set — which is what alias and transliteration expansion means when it is real. Second, compare tokens as a set where the culture warrants it. Nagy Péter against Péter Nagy scores 0.300 whole-string and 100 on a token-sorted ratio. That is not a smarter algorithm; it is the same algorithm asked a sensible question.

An LLM helps in exactly two places.

Explaining a candidate. Given the listing text and the counterparty record, a model can state which identifiers agree and which conflict, and say so in sentences a reviewer and an auditor can both read. That is a real gain over a queue of bare numbers, and it is where most of the review time actually goes.

Reading context the string never carried. Date of birth ranges, nationality against place of birth, a vessel's IMO number against a renamed hull, whether the listed "Mohamed" is a father's name in a compound. None of that is in the string comparison, and all of it is what decides the disposition.

It launders in one place, and the tell is easy to spot: an LLM must never produce or adjust the similarity score. If the pipeline computes 0.802 and a model raises it to 0.91, the number stops meaning anything and the audit trail stops being reproducible. A model that dismisses a candidate is making a decision; a model that annotates one is supporting a human decision. Article 76(5) of Regulation (EU) 2024/1624 draws the line in law from 10 July 2027: a decision to enter, refuse or maintain a business relationship — or to change the extent of due diligence — that results from an automated process or an AI system must be subject to meaningful human intervention, and the customer must be able to obtain an explanation of it and to challenge it.

The second tell: a model's confidence is not evidence. If the reasoning cites no field, it is prose, and prose that cannot be checked is worse than a bare score, because it reads as though it can.

The word is used for three different things, and only the third is worth paying for. A score is shown: that is not an explanation. A generated paragraph is shown: an explanation only where it names the fields it relied on. Or, per candidate, the record states which name form matched which listed form, by which technique, at what value, which identifiers agreed and which conflicted, and what the reviewer concluded — and that record can be reproduced later. Only the third is an artefact an auditor can test.

The reproducibility clause is the load-bearing one. Lists change; models change; thresholds get retuned. A screening result that is recalculated when any of those move is not evidence of what you knew on the day. That is why our own Full Search expands aliases and transliterations before anything is scored, and why its AI review stage reasons in writing about each candidate match instead of moving a number. Each result is stored as a point-in-time snapshot, never recomputed.

Five questions to put to any vendor, including us:

1. Which metric produces the score, and what is the prefix or length behaviour?
2. Is the query expanded into transliteration and alias variants before scoring, or is one spelling compared with one spelling?
3. Are diacritics folded before the phonetic pass, and which table is used?
4. Does anything in the AI layer modify the numeric score? (The answer should be no.)
5. Can you reproduce a match decision from six months ago, byte for byte, including which list version it ran against?

If the answers arrive as "AI-powered", you have learned which of the five techniques the vendor is using: none in particular.