You are being watched - the rise of educational surveillance

"Big Brother is Watching" (CC BY 2.0) by abracapocus_pocuscadabra

The use of artificial intelligence in education seems increasingly to be more about surveillance and control than fostering learning. During the past year there has been considerable concern about the ethics of remote proctoring where students were subject to constant surveillance by webcam, microphone and analysis of keystrokes and mouse clicks during online exams, see for example an article in Inside E-learning, Learning or surveillance? What students say about edtech and covid-19. Any unexpected activity in the student's behaviour (eg "abnormal eye movement") or movement in the room is detected and may lead to disqualification. Not only are such methods highly intrusive it is unclear what happens to all the data collected by the responsible companies. There must be better ways of assessing a student's ability than this.

If remote proctoring raises concerns then the next example sends shivers down the spine. An article on the site Rest of the world, China is home to a growing market for dubious “emotion recognition” technology, describes how classrooms are equipped with face recognition technology and each student's facial reactions are constantly monitored. 

Every second, the surveillance cameras installed in each classroom at Niulanshan First Secondary School in Beijing snap a photo. The images are then fed into the Classroom Care System, an “emotion recognition” program developed by Hanwang Technology. It identifies each student’s face and analyzes their behavior: a student rifling through their desk might be labeled “distracted,” while another looking at the board would be labeled “focused.” Other behavioral categories include answering questions, interacting with other students, writing, and sleeping. Teachers and parents receive a weekly report through a mobile app, which can be unsparing: In one, a student who had answered just a single question in his English class was called out for low participation — despite the app recording him as “focused” 94% of the time.

Imagine then coupling this with analysis of every mouse click and keystroke and we have total control, or rather the illusion of control since the conclusions made by AI may be based on in-built biases (like the student called for low participation in the example above). The idea that algorithms can accurately assess a student's emotions by analysing facial expression is ridiculous. We all know how hard it is to read another person's face. But once you get AI making decisions it becomes almost impossible to question them since we tend to see computers as impartial and infallible. No matter how questionable such technologies may be there's big money to be made. The article claims that the emotional recognition market may be worth more than $33 billion by 2023. Money talks. For further serious concerns about facial recognition, see an article in Mashable, 9 scary revelations from 40 years of facial recognition research.

There are, of course, positive applications of AI in education but the key question in all cases is who owns the data, on what terms and do the students have the right to be forgotten? We all need to be very cautious about letting these technologies into the classroom.