“Technology is the great equalizer”. We have all heard some variant of that phrase. Just for kicks, I googled it. Websites trumpeting technology’s use in the classroom, to increase diversity, to bring justice all use the exact phrase. Most of us are familiar with such tech-utopian language. Propelled by impartial math and science, technology only has the capability for good. In more recent times, the ‘truism’ has sustained some damage. Technology, and the math which powers it, also has the potential to exacerbate inequality, to facilitate censorship, and to perpetuate systems which segregate people and obscure truth.
Here’s the part where I make an imperfect analogy (and don’t bother to hammer out all the nuances). Every year the CDC, and health officials, make informed decisions about which influenza vaccine to order. Viruses mutate, scientists and health officials must contend with that every year when developing vaccines. Some have termed this work an evolutionary arms race. In other words, scientists and health officials must remain vigilant of the changes in the viruses and respond accordingly. In other words, they must monitor them.
Just now are we realizing that technology and math are both a medicine and a virus. The tools math has created evolved so quickly, beyond any individual’s comprehension and certainly beyond the public awareness. Cathy O’Neil book Weapons of Math Destruction focuses on one of the more pernicious viruses in technology- opaque algorithims and mathematical models. She names them weapons of mass destruction and their effects are manifold.
One such model played a large part in the housing crash of 2008 and the subsequent collapse of financial institutions in the United States. Another stratifies education and dictates who benefits most from good education (hint: it’s not the lower class). A third incarcerates minority populations and increases recidivism. In all cases, the models start from unexamined assumptions and data until their scale and their damage become widespread. These are two of the three requirements for Weapons of Math Destruction to exist.
The crux of the problem is not an uncaring force of nature, as in the case of viruses. The problem is perpetuated by lack of willpower which is correlated to the desire of the system’s beneficiaries to ignore the problem at best and actively suppress it at worst. This is the third requirement: opacity. We are dissuaded from understanding how these models function and, in some cases, we aren’t even given the information. Effectively, this means the Red Queen is winning right now. O’Neil reassures the reader that medicine for the problem does exist. We can modify these algorithims to better serve the utopian dreams. Where no modification exists, we can find one through rigorous testing. Where new technology does not do what it purports, we can even revert to older methods.
Weapons of Math Destruction is by no means comprehensive. It offers a brief glimpse into different algorithims affecting education, housing, employment, credit, and the justice system. The book demonstrates more the width of the problem than the breadth/depth. Even so, O’Neil’s book furnishes readers with a good foundational understanding of a timely issue.