SF faces ‘alarming’ lack of enforcement options for AI misinformation

As the November election approaches, fake videos and audio snippets generated by artificial intelligence continue to appear online, stoking concern that quickly advancing “deepfake” technology could be used to sway races and sow political misinformation.

Concerned that state and federal lawmakers will be too slow in their response, San Francisco Supervisor Dean Preston consulted with local election officials for the past several months to figure out what The City might be able to do on its own to fend off AI-generated falsehoods .

But with the election less than six months away, The City’s response remains uncertain. A long-awaited board hearing reviewing the matter this week turned up as many questions as it did answers, highlighting the complex challenges San Francisco would face if it attempts to fill in the enforcement gap left by state and federal authorities.

As things stand, officials with The City’s Election Department and Ethics Commission testified that neither of their agencies has the legal authority to regulate false political content.

In response, Preston said he finds the lack of enforcement options “alarming.”

“I think most San Franciscans would be surprised to learn that if they received something in the mail or watched a video or got a robocall that was blatantly AI-generated, false endorsement, blatantly illegal, that essentially there’s no one to report that to that’s really going to do anything about it,” he said.

Preston has been considering drafting a local requirement that election material that includes AI-generated content include a disclaimer.

It seems such a measure will be put on hold for the time being as Preston waits to see whether a package of state bills introducing new AI safeguards manages to advance in the coming months.

Instead, Preston said, he is focusing his efforts on administrative responses, encouraging local election officials to consult with deepfake experts to draw up an action plan for the November election.

In particular, Preston has asked them to review a report from the Brennan Center for Justice, which drew up a list of recommendations for how election officials can best prepare for the looming challenges AI poses .

The report — which draws on the experience gleaned during a “tabletop” exercise held last year involving election officials in Arizona — lays out a number of scenarios for how AI attacks could take place, including the release of AI-generated photos that falsely portray election officials involved in criminal activities.

The report also provides a playbook for how to respond. Among the recommendations, it encourages election officials to familiarize themselves with the capabilities of AI, develop plans to respond quickly in case of crisis and figure out ahead of time which authorities to contact if an AI-related attack occurs.

In line with these recommendations, the Elections Department already has plans in place to issue alerts about widely circulating election misinformation, though agency officials are not aware of any AI-specific regulations.

“The department’s primary role is to be a trusted source of information regarding election-related processes such as voter registration, voting and results reporting,” said John Arntz, the department’s director.

California already has a law that bans deceptive deepfakes targeting political candidates, but as written the only way to enforce it is for political campaigns to bring lawsuits.

As one way to broaden enforcement, one of the measures currently up for consideration in California’s legislature would provide a route for election officials, law-enforcement agencies and others to push social-media companies to scrub misleading deepfakes off their platforms, according to the California Initiative for Technology and Democracy, which co-sponsored the legislation.

But even if the measure passes, it's currently on course to take effect at the start of 2025, meaning it would come too late to affect the November election.

Preston said that he plans to bring a resolution to the full board in support of the package.

“It doesn’t make a lot of sense for each city across California to have to be creating its own extensive regulation of something that really should be addressed federally or at the state level,” he said.

But, he said, “if they’re not going to address it, then we’ll need to.”

If San Francisco does decide to pass a local enforcement measure, it will need to navigate a complex legal landscape, including constitutional free-speech protections that have stymied a previous attempt by The City to crack down on false political claims, said Ethics Commission Executive Director Patrick Ford.

“That’s going to be a very fraught area,” Ford said. “There’s going to be a lot of First Amendment concerns when you’re getting into what’s being said.”

And if his agency were to be tasked with enforcement, Ford said, it would need additional resources, including help procuring software tools to identify deepfakes.

So far it appears that there have not been many examples of deepfakes targeting San Francisco’s election system, said Preston, who has raised the question with election officials during his review.

One recent borderline example: A video circulated last month on X appeared to feature an AI-generated voice formulated to sound like former President Barack Obama, endorsing tech entrepreneur and online firebrand Garry Tan for San Francisco mayor, despite the fact that Tan has given no indication that he intends to run.

While the poster’s account includes the label "parody," some X users seemed to be taken in.

Referring to deepfake misinformation, Preston said, “fortunately, it’s been limited to date in San Francisco — and we want to keep it that way.”