Good Intentions, Risky Inventions: A Method for Assessing the Risks and Benefits of AI in Mobile and Wearable Uses

One of the mobile AI uses featured in the Atlas of AI Risks for mobile and wearable uses. Pokémon Go has done wonders for public health, getting millions off their sofas and into parks, museums, and even small businesses. However, the game's AI has not always been so well-behaved. Sensitive areas, such as cemeteries, were treated the same as busy city squares, and locations were added without property owners' consent, forcing them to opt out later. Worse still, the game's in-game hubs were more likely to pop up in predominantly white neighbourhoods, exposing biases within the system.

We analyzed 54 real-world cases of AI in mobile devices and developed an interactive tool to bring these findings to light. Under the EU AI Act, AI systems are classified by risk levels—ranging from low to unacceptable risk—to ensure their use is safe and fair, particularly in sensitive applications. The researchers found that even AI apps deemed “low-risk” under the EU AI Act, like Pokémon Go, can still lead to significant problems, such as safety risks or unfair tracking of certain communities.

To dig deeper into the risks and benefits of mobile AI, the researchers employed a three-step process, using Large Language Models (LLMs) to generate realistic AI use cases in over 40 domains, including healthcare. Each case was broken down into five components: domain, purpose, capability, AI user, and AI subject—essential for complying with the EU AI Act. They rated the risks of each AI application as low, high, or unacceptable, and evaluated the benefits of these use cases by assessing how they aligned with the United Nations' Sustainable Development Goals. The process, achieving over 85% accuracy, effectively ruled out hallucinations by the LLMs.

LLM-based method for identifying risks and benefits for mobile and wearable computing
Overview of our method. A three LLM-prompt pipeline for generating mobile and wearable uses of AI (prompt #1), classifying each use's risks according to the EU AI Act (prompt #2), and determining whether each generated use is beneficial according to the UN's Sustainability Development Goals (prompt #3). Out of 138 generated uses, as many as 80 were considered high risk according to the EU AI Act, primarily aligning with Sustainable Development Goals 3 (good health and well-being), 10 (reduced inequalities), and 16 (peace and justice). Our method was validated by two experts in mobile and wearable technologies, a legal and compliance expert, and a cohort of nine individuals with legal backgrounds who were recruited from Prolific, confirming its accuracy to be over 85%.

The study revealed a sobering truth: while many AI applications promise benefits—such as improved health monitoring, increased productivity, and greater social equality—58% of them were classified as high risk under the EU AI Act. These risks frequently involve sensitive data, automated decision-making, or the treatment of vulnerable groups. Even more troubling, AI uses that promote goals like peace, justice, and reduced inequalities—areas where AI is supposed to be most beneficial—were among the highest risk, often due to technologies like facial recognition and their application in sensitive sectors such as migration, defense, and healthcare.

As our findings show, the promise of AI is immense, but so are the dangers. Left unchecked, AI systems can deepen inequality, erode privacy, and harm those they are intended to help. Policymakers and developers alike must prioritize governance and transparency to ensure that AI serves all members of society, not just a select few.

Publications

  • Good Intentions, Risky Inventions: A Method for Assessing the Risks and Benefits of AI in Mobile and Wearable Uses. ACM Mobile HCI 2024 PDF
  • The Atlas of AI Incidents in Mobile Computing: Visualizing the Risks and Benefits of AI Gone Mobile. ACM Mobile HCI (demo) 2024 PDF

Code and data


We'll never share your email with anyone else.

N.B.: If you do not receive the instruction message within a few hours, please check your junk/spam e-mail folder just in case the email was moved there.