The fact our TVs are gathering our information isn’t new, but what and how they’re tracking — and if we can stop it — is hard to figure out. Researchers at Northeastern University monitored 12 TVs from 10 manufacturers to test whether opting out of automatic content recognition (ACR) actually works. While opting out was effective for most TVs — including LG, Amazon Fire TV, a Sony Bravia running Google TV, and those with Vidaa OS (which can be found on TVs from Hisense, Toshiba, and smaller brands like the Supersonic used in the study) — Samsung and Roku kept sending some data. And Walmart’s Vizio TV continued on as if the opt-out hadn’t even happened.

ACR systems sample the audio or video playing on a TV, convert them into digital fingerprints, and send them to a database to identify what’s being watched. TV companies use that information to make recommendations of what to watch next, provide viewing metrics, and allow for targeted ads.

The researchers wanted to understand when ACR was running on each TV, whether opting out did anything, and what type of data ACR systems were capturing. Because there are many different TV operating systems, and manufacturers keep smart TVs locked down, ACR activity needs to be inferred based on where the TV sends data.

To answer those questions, the team developed a suite of automated experiments across all the TV operating systems. They used Northeastern’s Mon(IoT)r software and SPHERE Research Infrastructure testbeds to capture and analyze all network traffic from each TV, which let them determine when ACR tracking was active and figure out where the TVs were likely sending that data.

They played content on each TV from first-party streaming services such as LG Channels and The Roku Channel; third-party streaming like Netflix and YouTube; over an HDMI cable; using the over-the-air antenna; and via screen mirroring, all to see if there was any difference in ACR traffic on different inputs.

They found that several of the TVs tested didn’t seem to have any ACR activity whatsoever, including a Westinghouse Xumo TV and three TVs from Sony, Hisense, and TCL running Google TV OS. (Another Sony TV with Google TV did have ACR enabled.)

Of the TVs with ACR-associated network traffic, all of them scan over-the-air transmissions, while everyone but Google and Amazon uses it on first-party streaming services (such as The Roku Channel and LG Channels).

Third-party apps like Netflix and YouTube? No ACR except a small amount on webOS, which disappears when opted out. Screen mirroring also looks to be safe from monitoring on all TVs but the one running Vidaa OS.

To figure out whether each OS used audio or video fingerprinting, the experimenters played content that had both video and audio, just audio with a black screen, and just video with no audio, and watched for changes in network traffic to the ACR endpoints. They found that Tizen likely uses video fingerprinting, webOS and Vidaa use audio, and Fire TV OS, Google TV OS, Roku OS, and Vizio OS use a combination of both.

Ideally, opting out of ACR activity should stop all data transmission to suspected ACR endpoints, and for webOS, Fire OS, Vidaa OS, and the Sony Google TV, that’s what happened. But it wasn’t always the case. Samsung’s Tizen OS activity decreased, but not completely, with ACR-correlated traffic still showing while using the antenna, first-party streaming, and HDMI. And on the two Roku OS TVs (one from TCL and one from Roku), ACR still applied to the Roku Channel app. TCL discloses this limitation in menu settings, while the Roku TV does not. Opting out on the Vizio TV didn’t change ACR-correlated traffic at all. (Vizio settled a lawsuit with the FTC in 2017 related to consumer data gathering.) I reached out to Walmart for comment about the Vizio TV’s performance in the study but did not hear back before press time.

As the paper notes, because almost all network traffic sent from the TVs to the suspected ACR endpoints was encrypted, the researchers couldn’t be entirely sure that the traffic that remained after opting out was ACR traffic.

“We interpret the continued presence of traffic consistent with ACR tracking after opt-out as suspicious, though we cannot confirm the contents of the network traffic (due to encryption), nor whether data is processed or retained on the server side.”

It doesn’t seem to matter if you opt out during the setup process or later down the line. “For the large-scale automated tests, we applied the opt-out through the TVs’ settings menus after setup, not during the setup process,” project lead professor David Choffnes told The Verge via email. But the researchers “also manually tested certain TVs (the Vizio TV and the Fire TV) after a factory reset, and the results were unchanged,” he told me. The testing order was also randomized to prevent earlier experiments from influencing later ones.

Disturbingly, during the study, the researchers discovered a vulnerability with Vidaa OS exposing unencrypted information. Because of the vulnerability, the researchers were able to access an MQTT broker — an intermediary between Vidaa OS and Unruly, a video ad platform — and receive ACR results from other Vidaa TV users.

Over a period of three hours, they collected data from over 1 million unique TVs. They informed Vidaa of the vulnerability, which the company confirmed and has since addressed by requiring a login for the server (although messages are still sent unencrypted).

The paper recommends ways to protect ourselves from TVs harvesting our data. The easiest is to disconnect your TV from the internet, both wired and wireless, and instead use a streaming device that doesn’t run ACR.

If you want to continue using the built-in apps, they suggest using webOS, Fire TV OS, Google TV on Sony, and Vidaa OS — just make sure to opt out of everything you can. They also provide a blocklist of endpoints you can restrict using blockers like Pi-hole that don’t break the TVs or affect view quality.

The researchers call on TV makers to make changes, including listing every input that ACR watches and putting that information in the privacy policy (some companies do this more clearly than others), and to authenticate and encrypt all ACR traffic. And they say regulators need to hold manufacturers to task and make sure opt-out agreements work fully to protect us.

I agree that there needs to be more transparency from TV manufacturers about ACR and, more broadly, what data of ours they collect and when. The Northeastern study helps to pinpoint when data is gathered, but there’s still a lot that’s seemingly intentionally hidden. And it shouldn’t be up to independent researchers to discover and disclose any of this. It should be the responsibility of the companies doing the collecting. It’s time for them to stop hiding what they’re doing.