AntifrogEnquiries
Theory Lesson 06
RISK / 7 MIN

What the queue cannot tell you.

Precision, recall and the customer cost of being wrong.

01

Define the population first

Choose an event, a time window and an outcome definition. A payment attempt is not a customer; a flagged event is not confirmed fraud. Labelled samples should include events outside the alert queue, or missed fraud will remain invisible. In real evaluations, delayed and uncertain labels need explicit treatment.

02

Precision asks about the alerts

Precision = true positives / (true positives + false positives). If 30 of 100 flagged events are confirmed fraud, precision is 30%. It tells you how concentrated the queue is, not how much of all fraud the system found.

03

Recall asks about all actual positives

Recall = true positives / (true positives + false negatives). If there are 50 confirmed fraud events and the rule catches 30, recall is 60%. You need outcomes beyond the queue to calculate this. A tiny, highly selective queue can miss most fraud.

04

Put the mistakes into context

For this synthetic example, 30 caught fraud, 70 legitimate alerts, 20 missed fraud and 880 correctly unflagged legitimate events produce a false-positive rate of 70/950 = 7.4%. That is different from the 70% of alerts that are legitimate. Compare costs and capacity as well as rates; a ratio alone cannot choose a policy.

Check your understanding

30 of 100 alerts are fraud. There are 50 fraud events overall. What is recall?