Windows AI feature that catches anything labeled a security “disaster”

Microsoft is set to launch a new AI-powered reminder feature that captures everything you do on your PC. Recall is part of the new Copilot Plus PCs that will debut on June 18, but experts who have tested the feature are already warning that Recall could be a “disaster” for cybersecurity.

Recall is designed to use local AI models to capture everything you see or do on your computer, then give you the ability to search and recover anything in seconds. There’s even an explorable timeline you can browse through. Everything in Recall is designed to stay local and private on the device, so no data is used to train Microsoft’s AI models.

Despite Microsoft’s promises of a secure and encrypted reminder experience, cybersecurity expert Kevin Beaumont found that the AI-based feature has potential security vulnerabilities. Beaumont, who briefly worked at Microsoft in 2020, tested Recall last week and found that the feature stores data in a plain text database. This could make it trivial for an attacker to use malware to extract the database and its contents.

“Every few seconds, screenshots are taken. These are automatically OCRed by Azure AI, run on your device and written to an SQLite database in the user’s folder,” explains Beaumont in a detailed blog post. “This database file contains a record of everything you have viewed on your PC in plain text.”

Beaumont shared an example of a plain text database on X, criticizing Microsoft for telling the media that a hacker could not exfiltrate Recall activity remotely. The database is stored locally on a PC, but it can be accessed from the AppData folder if you are an administrator on a PC. Two Microsoft engineers recently demonstrated this at Build, and Beaumont says the database is accessible even if you’re not an administrator.

The fear is that Recall makes it easier for malware and attackers to steal information. InfoStealer Trojans already exist to steal credentials and information from PCs, and hackers are currently distributing this type of malware to steal and sell information. “Recall allows threat actors to automate the deletion of everything you’ve viewed in seconds,” says Beaumont.

Beaumont has exfiltrated its own Recall database and created a website where you can download a database and search it instantly. “I deliberately hold back technical details until Microsoft ships the feature, because I want to give them time to do something,” he says.

Microsoft currently plans to enable reminder by default on Copilot Plus PCs. In my own testing on a pre-release version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you check an option which then opens the Settings panel. . Microsoft is, however, reportedly discussing whether to change this configuration process.

Reaction to Microsoft’s recall news was swift, with privacy advocates calling it a “potential privacy nightmare” and the UK’s Information Commissioner’s Office stepping in to inquire to Microsoft over its use of the AI-based feature.

Microsoft maintains that Recall is an optional experience and that it has built privacy controls into the feature. You can disable certain URLs and applications, and Recall will not store any material protected by digital rights management tools. “Recall also does not take snapshots of certain types of content, including InPrivate web browsing sessions in Microsoft Edge, Firefox, Opera, Google Chrome, or other Chromium-based browsers,” Microsoft says on its FAQ page explanatory.

However, Recall does not perform content moderation, so it does not hide information such as passwords or financial account numbers in its screenshots. “This data may be in snapshots stored on your device, especially when sites do not follow standard Internet protocols, such as hiding password input,” Microsoft warns.

However, Microsoft’s FAQ page does not address the possibility of malware attempting to steal the Recall database. “Recall snapshots are kept on the Copilot Plus PCs themselves, on the local hard drive, and are protected by data encryption on your device and (if you have Windows 11 Pro or a Windows 11 SKU d ‘enterprise) BitLocker,’ explains Microsoft.

As Beaumont points out, disk encryption is only effective in certain scenarios. “When you’re connected to a PC and running software, things are decrypted for you,” Beaumont says. “Encryption at rest is only useful if someone comes to your house and physically steals your laptop – that’s not what hackers do.”

Microsoft may well need to rework Recall, or recall it, if you like. There are clearly obvious gaps in the way data is stored here that need to be filled, and making this an opt-out experiment worries privacy campaigners. Recall’s launch comes just weeks after Microsoft CEO Satya Nadella called on employees to make security Microsoft’s “top priority,” even if that means prioritizing it over new features.

“If you are faced with a trade-off between security and another priority, your answer is clear: Make security” Nadella said (emphasis mine) in an internal memo obtained by The edge. “In some cases, this will mean prioritizing security before other things we do, like releasing new features or providing ongoing support for existing systems.”

The edge contacted Microsoft for comment on security and privacy concerns related to Recall, but the company did not respond in time for publication.