Kaarana

Technology

It is the Math. that is the problem – Kaarana – Medium

Anand Venkatanarayanan

Why do biometric devices that work in one setting do not work well in other settings? The usual answers to this question are:

We have seen almost all these answers to explain significant biometric authentication failures on the field, but so far no one has considered the question

Has the UIDAI made fantastical assumptions about the success of biometric authentication by ignoring basic probability theory taught in 10th Standard Mathematics books?

During this post, we will explore this question slowly.

How do you measure failures?

We will only concern ourselves with False Rejection Rate (FRR) which causes exclusion during biometric authentication. Every device used in the field is certified by STQC (Standardization Testing and Quality Certification) to have a FRR of < 2%. So why do field reports always report > 10% on average? To understand this, we must understand a bit of probability.

It is intuitive to understand probability using a coin. We don’t really need to conduct an experiment to understand that P(Heads) = ½ and P(Tails) = ½. However, for biometric authentication, we need to conduct an experiment to determine the Probability (FRR) because it is not obvious like the coin toss example.

The FRR is determined thus (through counting):

Sub Populations

The next question to ask, once we get this number (< 2%) is:

Is this a constant number across the entire population?

STQC itself knows that this is not so, since the biometric testing depends on National Institutes of Standards and Technology (NIST) standards. The variance of across finger print quality and gender is given below:

So even given the < 2% FRR, we can already conclude the following without looking at any authentication data:

1) There is a direct correlation with increasing age and increasing authentication failures.

2) If < 2% is the average failure rate, given the rapid fall of finger print quality as one ages, the failure rates will be much higher for aged people.

3) There could be a potential 25% increase in failures for women (2.0 vs 2.5 Image quality, assuming linear relationship between failures and image quality) and for older women it could get worse.

There is still the question of how do we convert these FRR probability numbers to understand exclusion? For that we must play a little game.

Probability and Outcome are different

There are two players (A, B) in the game and a single coin. If the coin turns up as Head, A wins ₹1 from B and if the coin turns up as Tail, B wins ₹1 from A. Now the key question:

How much would A and B have won, if they start from zero and play the game long enough?

The answer is obvious. It Is Zero and can be written down as (½ * 1) — (½ * 1) for both A and B.

Now let us change the rules and ask a different question:

How many B’s will return empty handed, if this game is played long enough?

You don’t have to answer this question through computation, but intuitively we can understand it that a lot of people will return empty handed.

In other words, the conditions of the game are more important to understand the people who would return empty handed than the probability of getting a tail. It is impossible to understand the impact (people returning empty handed) by looking at the raw probability of getting a tail.

Mathematically this is taught as (X) and F(X) are not one and the same thing, where (X) is a random variable (Fooled by Randomness is a good start).

Biometric Game of Authentication

We can now understand why exclusion caused by biometric authentication is wide spread on every program that it touches.

All the above changes, worsens the expectations function and thus causes exclusion. Mathematically the expectations function is concave, while the probability function is linear and one cannot surmise the former from the later. Yet UIDAI and STQC test and report only the probability function and refuse to publish data about the expectation function.

Conclusion

A class 10 student can understand the distinction between probability and expectation functions. UIDAI seems to behave like a student, who wrote the wrong answer to a standard probability vs. expectation question and when called out, alleges malefic intent on the examiner.

It is the mathematical understanding of the student that is the problem here, not the malefic intent of the examiner. May be RD Sharma will help?

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top