Stealth Quotas

You probably haven’t given much thought recently to the wisdom of racial and gender quotas that allocate jobs and other benefits to racial and gender groups based on their proportion of the population. That debate is pretty much over. Google tells us that discussion of racial quotas peaked in 1980 and has been declining ever since. While still popular with some on the left, they have been largely rejected by the country as a whole. Most recently, in 2019 and 2020, deep blue California voted to keep in place a ban on race and gender preferences. So did equally left-leaning Washington state.

So you might be surprised to hear that quotas are likely to show up everywhere in the next ten years, thanks to a growing enthusiasm for regulating technology – and a large contingent of Republican legislators. That, at least, is the conclusion I’ve drawn from watching the movement to find and eradicate what’s variously described as algorithmic discrimination or AI bias.

Claims that machine learning algorithms disadvantage women and minorities are commonplace today. So much so that even centrist policymakers agree on the need to remedy that bias. It turns out, though, that the debate over algorithmic bias has been framed so that the only possible remedy is widespread imposition of quotas on algorithms and the job and benefit decisions they make.

To see this phenomenon in action, look no further than two very recent efforts to address AI bias. The first is contained in a privacy bill, the American Data Privacy and Protection Act (ADPPA). The ADPPA was embraced almost unanimously by Republicans as well as Democrats on the House energy and commerce committee; it has stalled a bit, but still stands the best chance of enactment of any privacy bill in a decade (its supporters hope to push it through in a lame-duck session). The second is part of the AI Bill of Rights released last week by the Biden White House.

{snip}

The problems begin when the supporters of these measures explain what they mean by discrimination. In the end, it always boils down to “differential” treatment of women and minorities. The White House defines discrimination as “unjustified different treatment or impacts disfavoring people based on their “race, color, ethnicity, [and] sex” among other characteristics. While the White House phrasing suggests that differential impacts on protected groups might sometimes be justified, no such justification is in fact allowed in its framework.  Any disparities that could cause meaningful harm to a protected group, the document insists, “should be mitigated.”

The ADPPA is even more blunt. It requires that, among the harms to be mitigated is any “disparate impact” an algorithm may have on a protected class – meaning any outcome where benefits don’t flow to a protected class in proportion to its numbers in society. Put another way, first you calculate the number of jobs or benefits you think is fair to each group, and any algorithm that doesn’t produce that number has a “disparate impact.”

Neither the White House nor the ADPPA distinguish between correcting disparities caused directly by intentional and recent discrimination and disparities resulting from a mix of history and individual choices. Neither asks whether eliminating a particular disparity will work an injustice on individuals who did nothing to cause the disparity. The harm is simply the disparity, more or less by definition.

Defined that way, the harm can only be cured in one way. The disparity must be eliminated. For reasons I’ll discuss in more detail shortly, it turns out that the disparity can only be eliminated by imposing quotas on the algorithm’s outputs.

The sweep of this new quota mandate is breathtaking. The White House bill of rights would force the elimination of disparities “whenever automated systems can meaningfully impact the public’s rights, opportunities, or access to critical needs” – i.e., everywhere it matters. The ADPPA in turn expressly mandates the elimination of disparate impacts in “housing, education, employment, healthcare, insurance, or credit opportunities.”

And quotas will be imposed on behalf of a host of interest groups. The bill demands an end to disparities based on “race, color, religion, national origin, sex, or disability.” The White House list is far longer; it would lead to quotas based on “race, color, ethnicity, sex (including pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual orientation), religion, age, national origin, disability, veteran status, genetic information, or any other classification protected by law.”

{snip}

All this reeducating has a cost. The quotafied output is less accurate, perhaps much less accurate, than that of the original “biased” algorithm, though it will likely be the most accurate results that can be produced consistent with the racial and gender constraints.  To take one example, an Ivy League school that wanted to select a class for academic success could feed ten years’ worth of college applications into the machine along with the grade point averages the applicants eventually achieved after they were admitted. The resulting algorithm would be very accurate at picking the students most likely to succeed academically. Real life also suggests that it would pick a disproportionately large number of Asian students and a disproportionately small number of other minorities.

The White House and the authors of the ADPPA would then demand that the designer reeducate the machine until it recommended fewer Asian students and more minority students.  That change would have costs. The new student body would not be as academically successful as the earlier group, but thanks to the magic of machine learning, it would still accurately identify the highest achieving students within each demographic group. It would be the most scientific of quota systems.

{snip}

But it can hide the unfairness. When algorithms are developed, all the machine learning, including the imposition of quotas, happens “upstream” from the institution that will eventually rely on it. The algorithm is educated and reeducated well before it is sold or deployed. So the scale and impact of the quotas it’s been taught to impose will often be hidden from the user, who sees only the welcome “bias-free” outcomes and can’t tell whether (or how much) the algorithm is sacrificing accuracy or individual fairness to achieve demographic parity.

In fact, for many corporate and government users, that’s a feature, not a bug. Most large institutions support group over individual fairness; they are less interested in having the very best work force—or freshman class, or vaccine allocation system—than they are in avoiding discrimination charges. For these institutions, the fact that machine learning algorithms cannot explain themselves is a godsend. They get outcomes that avoid controversy, and they don’t have to answer hard questions about how much individual fairness has been sacrificed. Even better, the individuals who are disadvantaged won’t know either; all they will only know is that “the computer” found them wanting.

If it were otherwise, of course, those who got the short end of the stick might sue, arguing that it’s illegal to deprive them of benefits based on their race or gender. To head off that prospect, the ADPPA bluntly denies them any right to complain. The bill expressly states that, while algorithmic discrimination is unlawful in most cases, it’s perfectly legal if it’s done “to prevent or mitigate unlawful discrimination” or for the purpose of “diversifying an applicant, participant, or customer pool.” There is of course no preference that can’t be justified using those two tools. They effectively immunize algorithmic quotas, and the big institutions that deploy them, from charges of discrimination.

{snip}