5

One of the origins of ROC curves seems to be to compare radar systems in WWII (source). How did they actually compute the False Positive Rate when they didn't have an estimate for True Negatives?

If I understand correctly, the FPR is FPR = FP/(FP+TN). But what would TN be in this case? Every time the radar has correctly identified that there is no bomber over them? But that would require them to arbitrarily quantize the time span: E.g., "The system correctly predicts that today there was no bomber" or "The system correctly predicts that this hour there was no bomber".

In comparison, I think it would have been much more intuitive for them to use precision instead of FPR. So why did they go with ROC instead of PR curves?

brnl
  • 53
  • 3
  • 1
    Radar engineering is an arcane science but the long and short of it is that lots of things that are obviously not aircraft can be detected/appear on a radar screen, such as clouds, storms, or flocks of birds. Just think about the visuals of a TV weather forecasts: the images of clouds overlayed on a map are created by radar. – Sycorax Feb 01 '22 at 18:50

2 Answers2

1

The ROC curve was originally developed as tool to quantify how good a radar engineer was at discerning radar noise from true signal, and where to draw thresholds to optimize particular aspects of performance. If there is no blip on the radar at all, there is no event to classify as either positive or negative. I can't find evidence to support it, but I imagine statistics were generated only from potential hits that needed to be evaluated by the engineer - they see a blip on the radar, and have to classify it as either a plane or radar noise. A true negative would be a blip on the radar classified as not a plane, which is indeed just radar noise. Complete silence on the radar isn't something that would need to be classified by the engineer, as they have no data to classify on. All engineers would decide that an empty radar screen indicates no plane, so this isn't a useful metric by which to compare engineers.

Basically, only radar blips are the events that need to be classified, and those are inherently events that are already quantized in time. An empty radar screen means "no plane" to everyone, but we want the ROC to determine how good someone is at detecting possible planes. The ROC here is a measure of how often a radar blip is correctly classified as a plane or not.

Nuclear Hoagie
  • 5,553
  • 16
  • 24
0

Isn't there also a 'time window' issue with FP? Radar 'detects' bomber(s) now: No bombs in next 5 min? 20 min? hour? day?

Where do you draw the line? Maybe just use the same lag for both?

I'm betting those under threat of attack 'knew' the answer and used it consistently.

BruceET
  • 47,896
  • 2
  • 28
  • 76
  • Don't radar systems typically have a *sweep*, a periodic rotation of the instruments that provides information about some arc (possibly a circle)? – Sycorax Feb 01 '22 at 18:59