August has featured the word ‘algorithm’ far more than most months in living memory, especially for those of us working in the world of education.
Working with and around technology for the last few years has opened my eyes to how easily humans can be dazzled by its ‘magic’ and the efficient solutions it appears to provide. Did that happen with the exams fiasco making headlines in the UK this summer? Quite possibly. A solution that seeks to replicate human decision-making in a fraction of the time is a naturally compelling one but it doesn’t take much scratching at the surface to discover that it’s a sword with two edges.
It takes very little effort or time to unearth instances of algorithms gone bad, assuming they were ever good in the first place.
This article from the British Medical Journal shares how staff at St George’s Hospital Medical School sought to automate its admissions process. It wasn’t long before they discovered that the programme displayed bias against females and people with non-European-looking names.
This article highlights how an algorithm created to determine the likelihood of someone convicted of a crime re-offending resulted in significant racial bias.
This article reveals the bias that existed in a recruitment algorithm created for Amazon that selected male candidates for positions more frequently than female.
In this TED talk from 2016, Joy Buolamwini, founder of the Algorithmic Justice League, spoke of the ‘coded gaze’ that drives facial recognition software and how she’s fighting it. This article in New Scientist describes the inaccuracy of gender facial recognition across algorithms from IBM Microsoft and Chinese company Megvii.
These examples can easily be accompanied by numerous others. What’s perhaps even more concerning is the number of algorithms still in operation around the world – making decisions on behalf of humans at this very moment – whose data hasn’t been interrogated and their bias not yet revealed.
As the men ‘in charge’ attempted to gain face after the algorithmic mess of exam results in recent days and weeks, it was apparent no responsibility was being taken for the source of its creation. Boris Johnson shared with pupils in one school earlier this week that he was afraid their ‘grades were almost derailed by a mutant algorithm‘ as if the programme had unexpectedly taken on a life of its own and there’s nothing the humans could have done to stop it.
In the film, I, Robot, we see Will Smith say to a machine that has indeed taken on a life of its own, ‘you are just a machine. An imitation of life’. Yet algorithms are machines that do not just imitate life; they learn from it, identify patterns and therefore amplify it. Whatever can be located in society is to be found at the very heart of an algorithm. The examples of algorithmic bias listed above were fed by historical data and decision making; they were fed by their creators and greedily consumed their biases.
Algorithms are what they eat. If their diet consists of racial, class, gender bias and a general disregard for equity, guess what they will become? They recognise patterns in the data they’re given. Our society is filled with a multitude of patterns of discrimination and all an algorithm can do is learn from this and magnify it. This can be countered in a number of ways, least of all by asking the critical questions at the point of inception and assembling ‘full spectrum teams who can check each other’s blind spots’ (Buolamwini 2016) as the programme is created, implemented and acted upon.
Instead of seeking to use algorithms to make efficient our decision-making so that the discrimination baked into our education system and society as a whole is acted on at scale, we could actually use algorithms to correct and address our biases by revealing them.
On a daily basis, our biases drive our decision-making at every turn. It’s difficult to notice these smaller acts as they happen but over time, we may begin to identify the patterns of our behaviour. An algorithm’s job is to make things more efficient, speed things up. Perhaps there’s a world in which they can be used to expose our discriminatory biases so we can act on them sooner? I’d like to hope that an organisation somewhere is already working on algorithms for equity.
For a while now, there’s been a niggling feeling that my presence on social media had trapped me in an unhealthy bubble. Going between Twitter accounts, I noticed that topics that were trending differed from what was trending when I logged into the BAMEed Network account as part of my steering group role, and different again when I looked at what my partner saw. What I interacted with the most was what I was presented with, it generated ‘more of the same’ suggestions for me to interact with and I consumed what I was given. The use of the platform had made me lazy. I was asking my network for recommendations of things to read and I went to them for help with decision-making and getting work done. There were inherent dangers in this. I thought I was accessing a full spectrum of perspectives as I worked hard to make my network diverse but could the technology really be trusted and were my efforts to overcome its bias working? Was it amplifying assumptions about who I was and what I wanted based upon data about society at large? I couldn’t be certain and therefore I was.
Just over a week ago now, I temporarily deactivated my Twitter account and wrested control of my life. There were many drivers for this but becoming suspicious of the diet I was being fed was one major one. Life beyond social media is of course not free from the internet nor discrimination in society but the time away has opened my eyes. I no longer feel powered by a machine. There are times when I miss the connection and community social media provides but if I choose to make tentative steps towards a return, I will do it consciously, fully aware of the ways my timeline is dictated by an algorithm. Now that we live in a world where technology is largely inescapable, we all have a responsibility to reflect on the ways in which it is impacting our lives and almost certainly manipulating us too.
I’m reminded of a speech I heard Reni Eddo-Lodge give a few years ago now in response to the question, what can I do about racism? Clearly she wasn’t about to answer this for every single audience member. That was for each of us to research and decide for ourselves but she did say that we should seek to make whatever difference we could in our immediate context – our homes, our workplaces, our communities.
This moment feels like one of those junctions that appear if we’re willing to notice them. It’s a junction where I choose to pause, stop the momentum of my present and consider how I’ll choose to respond.
‘Until they become conscious, they will never rebel’ (Orwell 1949).
‘One believes things because one has been conditioned to believe them’ (Huxley 1932).
Buolamwini J (2016) How I’m fighting bias in algorithms. Available at: https://www.ted.com/talks/joy_buolamwini_how_i_m_fighting_bias_in_algorithms?language=en#t-94426
Orwell G (1949) 1984.
Huxley A (1932) Brave New World.
It’s nice to read a post of someone that is going through similar things, feeling the same feelings and drawing similar conclusions although perhaps encountering circumstances from a slightly different perspective.
I believe the bubble you refer to, regarding your online accounts, is what the T3 Commission report on Tackling the information crisis would call a Public Truth Domain. An ecosphere of opinion and experience (including fact-checking) that is formed from our online behaviour. For me, reading this report in 2018 jumping down the rabbit hole of not so subtle algorithms, Russian/Eastern Bloc social media hacking and vast ‘behavioural surplus’ data mining.
A strange set of feelings ensued, in that everything I was encountering seemed to be a great conspiracy, other than that I could track the information and thankfully the academics and journalists reporting on these phenomena were engaging in their responsibility of citing their references and research.
Earlier this year, on a whim (and because I thought the title was suitably dramatic) I picked up a copy of Zuboff’s “The age of surveillance capitalism” which has been an earth-shattering revelation to me. Zuboff’s contention is that a new form of capitalism, birthed at the dawn of the 21st century, has found that the last great resource available to technological markets is that of human behaviour as mined by our internet/social media and technology usage. It’s a stark and frightening read, but it links back to your remarks in that it agrees with what you have seen, that algorithms are not subtle they are only as strong as the data mined to source them. A Harvard professor in (social) psychology, Zuboff’s text is 500 pages but is supplemented with her notes and citations that push the book to 700 pages.
On the strength of that book alone I have stopped using Google at home and am removing as much of my personal data as I can from my google accounts. A stage state of affairs as I am now being ‘instructed’ that I must use Google to teach my learners…….. I fear for their privacy and online safety.
The BBC recently ran an article about how algorithms are being used by managers, companies and political organisation in order to avoid accountability and confrontation when decisions are made through machine learning. Very much in line with what we have seen in the qualifications fiasco of the last month. (I’ll link at the bottom).
There is so much to say on this subject, so much that is imminently threatening how we educate our young people but I’ll end with this…
Algorithms can only return a result based on previous data. Education is something different. In fact, it is the antithesis of an algorithm. Education is the process whereby the outcome does not resemble the data input; space where opportunities to change and grow are encouraged, where lives are changed, where perspectives and understanding encounter unrecognisable transformation.
Sorry, this comment has gone on far too long…… It was lovely to read your post.
Hello Ken! So lovely to hear from you. I do miss our long conversations on topics such as this and I hope the start of term is going well for you.
I am going to investigate the rabbit hole of the ‘public truth domain’ and the links you suggest including Zuboff sound fascinating… an information crisis indeed. The internet and the advance of technology holds much to be hopeful about but I do wonder if we’re edging towards a place where there’s far more to counter it. Our behaviour and thoughts are of course, manipulated in many ways without technology, but I am concerned about the level to which this is amplified and made easier at scale when tech is added. It’s led by the intentions behind the developments, which will inevitably be economic…
What you say about education is wonderful and so very true. Education is the obvious counter to your fears for young people – to educate them as best you can about being mindful and raising their awareness of what they interact with and how it might be (almost certainly is) manipulating them.
Hi again Hannah, I thought you might find this interesting.
This new documentary has been released today. It paints an incredibly worrying picture and it interviews the significant players that engineered these software features on the inside and now are throwing a light on how these technologies have developed.
It’s called “The Social Dilemma”
Oh sorry, I forgot. There’s a couple of really good youtube videos that work really well as an introduction to Zuboff’s findings.
VPRO Backlight (Dutch public broadcasting documentary)
Channel 4 News