New research on the use of artificial intelligence to monitor workers has led to calls for more protections for employees.
The Data on Our Minds report, for the Institute for the Future of Work, sounds the alarm over “algorithmic affect management”, which could lead to the use of AI biometric surveillance in the workplace.
“Algorithmic affect management” (AAM) is the strategic tracking, evaluating, and managing of workers through algorithms, taking over tasks that used to be performed by human managers. This innovation in management is especially common in the gig economy.
The UK needs to protect employees from biometric tracking and emotion recognition technologies at work, particularly as it is no longer entirely subject to EU law, argue the report’s authors led by Dr Phoebe V Moore from the University of Essex.
Use of AAM technology has begun among certain companies in the UK, Moore said, making for an interesting situation, she added in her blog, “because the AI Act [in the EU] requires regulatory sandboxes and simulated environments to test high-risk category products, given emotion recognition is now in the forbidden category for AI systems.”
The technology was starting to allow companies to monitor workers’ moods and feelings, as well as biometric data like heart rate, eyeball tracking or temperature, the report found.
Moore wrote that the rise of remote working since the Covid pandemic saw management-led data-gathering become quite widespread. She cited a study that found the use of “bossware” had increased three-fold in the past five years. Even in 2019, a Gartner survey of 239 large corporations found that more than 50% were using some form of non-traditional monitoring, including analysis of the text of workers’ emails and social media use, scrutiny of meetings, and gathering biometric data.
The Institute said that when linked with powerful management software that uses algorithmic decision-making, AAM can cross-reference mood and physical performance with sickness and other data, “opening a significant new frontier in workplace surveillance”. It argued that AAM had the potential for mental and physical harm – and that new regulation and governance was urgently needed.
In particular, said the Institute, there was an urgent need for regulations that addressed the potential for direct or indirect discrimination based on protected characteristics and the risks associated with neurosurveillance.
The Employment Rights Bill and the Data Bill, both at committee stage in the UK parliament, offered the opportunity to establish protections against exploitative practices arising from AAM technologies, said the Institute, adding: “The current patchwork of protections is insufficient to safeguard workers’ privacy, physiological, and mental integrity when AAM technologies are deployed.”
Moore found that the rationales offered by companies for using AAM technologies have ranged from the protection of occupational safety and health to the monitoring of wellness and fitness, the protection of trade secrets and cyber security and productivity optimisation.
The UK is becoming a hotbed for facial and emotion recognition trialling” – Dr Phoebe V Moore
She said that any rationale for their use should be balanced with proportional protections for workers from psychosocial harm.
Two cases in the UK mentioned by the authors concerned Serco Leisure, which was earlier this year ordered by the Information Commissioner’s Office to stop using facial recognition technology to monitor attendance of leisure centre employees. It was also found in June this year that eight railway stations, including London’s Euston and Waterloo, were trialling emotional recognition of passengers using AI cameras integrated with Amazon’s advanced machine learning algorithms. The system could identify agitated or distressed passengers as well as detect age and gender. Campaigner Big Brother Watch submitted a complaint to the ICO about the trial.
Moore said the UK was “becoming a hotbed for facial and emotion recognition trialling”.
Jeni Tennison, founder and executive director of tech campaign group Connected by Data, said the Institute’s report underscored “the need for inclusive, fair data and AI governance, and provides actionable recommendations on how organisations can engage workers in meaningful ways.”
Frank Pasquale, professor of law at Cornell Tech University, said: “Workers, unions, and concerned members of the public will find important insights here. And the report’s wise policy recommendations should guide action by regulators.”
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
