College students used Meta's smart glasses to doxx people in real time

Two Harvard students have created an eerie demo of how smart glasses can use facial recognition technology to instantly capture people's identities, phone numbers and addresses. The most disturbing thing is that the demo uses current, widely available technologies such as Ray-Ban Meta smart glasses and public databases.

AnhPhu Nguyen, one of the two students, posted a video showing the technology in action, which was then picked up by 404 media. The technology, called I-XRAY, leverages the Meta smart glasses' ability to livestream videos to Instagram. A computer program then monitors this stream and uses AI to identify faces. These photos are then fed into public databases to find names, addresses, phone numbers and even relatives. This information is then reported back via a phone app.

In the demo, you can see Nguyen and Caine Ardayfio, the other student behind the project, using the glasses to identify multiple classmates, their addresses and names of relatives in real time. Perhaps even scarier, Nguyen and Ardayfio are also seen talking to complete strangers on public transportation, acting as if they know them based on the information the technician has gathered.

Facial recognition technology has been shockingly accurate for some time, and I-XRAY is largely just a concatenation of existing technologies. It is partly based on PimEyes, the The New York Times described in 2022 as an “alarmingly accurate” facial search engine that “everyone can use.” Concern about this technology has increased since it was revealed that Clearview AI was using facial recognition to aid law enforcement. What's new about Nguyen and Ardayfio's demo is how it combines the technology with a consumer device that is discreet and easily accessible.

“The purpose of developing this tool is not for misuse, and we are not releasing it,” Nguyen and Ardafiyo wrote in a document explaining the project. Instead, the students say their goal is to raise awareness that this isn't some dystopian future – it's all possible now with existing technology. In particular, they note that I-XRAY is unique because large language models (LLMs) allow it to work automatically and draw relationships between names and photos from huge data sources.

Data protection has always been a major concern when it comes to smart glasses. Google Glass originally failed, among other things, due to public criticism of its inclusion in public spaces without consent. However, it is also true that over the last decade, people have become more and more accustomed to being filmed due to the rise of smartphones, vloggers and TikTok. The worrying thing about modern smart glasses, however, is that they don't stand out quite as much as Google Glass.

The Ray-Ban Meta glasses used in this demo look just like any other Ray-Bans glasses. While this is crucial to the adoption of smart glasses, it also makes it harder for people to see when someone is wearing a camera on their face. The Meta glasses have a privacy light that automatically turns on when you record a video. However, in our testing, we found that the light is hard to notice when you're outside in bright light, and that people often don't notice it when filming – especially in crowded, public spaces.

Meta, for its part, warns users against committing glass holes in its Ray-Bans privacy policy. It encourages users to “respect people’s preferences” and to gesture clearly or use voice controls when recording videos, livestreaming or taking photos. However, the reality is that people may also choose not to follow wearable etiquette, regardless of what Meta says. The edge I asked for a comment. Meta spokesperson Dave Arnold responded to our email by citing the Terms of Service, which reiterates the same guidelines.

This is a sobering reminder of how smart glasses can be misused, but there are some steps people can take to protect themselves. In their document, Nguyen and Ardafiyo list reverse face search and people search databases that you can opt out of. However, keep in mind that it's nearly impossible to completely delete your online presence – all you can do is make your information less available.

Update, October 2nd: Added answer from Meta.