Commentary: "Expect the machines to take control"
Ask a classroom of university students to list the researcher advantages of digital archives, whether digitized or born digital, and you get predictable answers: available without respect to location, time of day, age, physical mobility, or political system--in other words, democratic access. Some savvier students may say it helps preserve papers and photographs because they aren’t handled. A few may complain gently about the thousands of hits they get for a simple query, but most are happy to search without having to develop a search strategy. They fail to note that the actual researcher is a machine.
The broad uses of artificial intelligence (AI) and the increasingly sophisticated algorithms employed raise significant issues for researchers, for human rights and for archives.
Consider this: According to the Daily Mail, the European Union plans to introduce “swarms of AI driven robots that can patrol borders by surveillance of the sea and land.” The robots’ data will be fed to a control room where it will be linked to data from “static sensors.” An algorithm will then identify “threats” to be sent to “border authorities” and “operational personnel.” A leader of the International Committee for Robot Arms Control told the Italian magazine Il Manifesto the group wants “to prevent two functions of the machines: target selection and targeting.” AI systems for facial recognition are known to incorporate bias: according to researchers at two U.S. universities “three commercially released facial-analysis programs from major technology companies demonstrate both skin-type and gender biases.” In tests, “the three programs’ error rates in determining the gender of light-skinned men were never worse than 0.8 percent. For darker-skinned women, however, the error rates ballooned — to more than 20 percent in one case and more than 34 percent in the other two.” So how accurate will the EU robots be in identifying border crossers, smugglers, and polluters, the targets of the EU programs? https://www.dailymail.co.uk/sciencetech/article-7022589/Swarms-automated-drones-controlled-AI-patrol-Europes-borders-using-powerful-sensors.html; http://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212
Or think about this records issue: A non-profit organization in California developed an algorithm which “can, at the touch of a button, delete the criminal records of thousands of people,” Digital Journal reported. Called “Clear My Record,” the algorithm examines “thousands of lines of conviction data and determines eligibility [for destruction] within minutes.” The developer says it could “clear 250,000 convictions throughout California by the end of 2019.” Let’s hope the algorithm was taught records retention rules. http://www.digitaljournal.com/tech-and-science/technology/justice-algorithm-wipes-the-criminal-past-of-thousands/article/548754
Or this structural archives issue: A records management official in a government agency that is installing an artificial intelligence system to manage the agency’s records from retention to retrieval to destruction was asked if the AI system eventually would be transferred to the national archives along with the records. He said he thought that the national archives would never hold AI-powered records, that the records would stay permanently in the agency and be managed by it with the national archives having solely a monitoring role.
Or this freedom of information issue: Is an algorithm employed by a government reachable as a record under a freedom of information act? Italy has a case on point. After a national public exam to hire teachers, the Italian Ministry of Education used an algorithm to assign teachers to schools. A legal battle followed, as teachers unhappy with assignments demanded to know how the algorithm worked. Italy’s highest administrative court decided that the algorithm was an administrative document, created by humans who gave it specific instructions, and therefore was covered by the right of access law. (Thanks to Giulia Barrera for the information.) https://parer.ibc.regione.emilia-romagna.it/notizie/algoritmi-delle-pa-una-sentenza; for the court decision see https://www.giustizia-amministrativa.it/portale/pages/istituzionale/visualizza/?nodeRef=&schema=cds&nrg=201704477&nomeFile=201902270_11.html&subDir=Provvedimenti
Finally, in addition to concerns about bias in targeting as a human rights issue and destruction and retrieval as archives issues, AI adopters are realizing that “training” a major AI system is a massive task. Moreover: As reported in n a new paper, researchers at a U.S. university “performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 pounds of carbon dioxide equivalent—nearly five times the lifetime emission of the average American car (and that includes manufacture of the car itself).” So in addition to the complex intellectual task of training, there is an environmental impact. https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/
The Council of Europe (CoE) has taken a step to address the human rights concerns that arise from the machine as researcher. CoE’s Commissioner for Human Rights issued “Unboxing Artificial Intelligence: 10 Steps to Protect Human Rights.” Step 7, on data protection and privacy, says member states should have legislative safeguards when AI systems are used to process “genetic data; personal data relating to offences, criminal proceedings and convictions, and related security measures; biometric data; personal data relating to ‘racial’ or ethnic origin, political opinions, trade-union membership, religious or other beliefs, health or sexual life.” https://rm.coe.int/unboxing-artificial-intelligence-10-steps-to-protect-human-rights-reco/1680946e64 Now we need to think how those principles will apply to archives.
As long ago as 1951Alan Turing, the brilliant World War II era British mathematician and computer visionary, wrote in the essay Intelligent Machinery, A Heretical Theory, “It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers… They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control.” http://uberty.org/wp-content/uploads/2015/02/intelligent-machinery-a-heretical-theory.pdf
It’s time to take notice.