L.A. Schools Probe Charges its Hyped, Now-Defunct AI Chatbot Misused Student Data

Independent Los Angeles school district investigators have opened an inquiry into claims that its $6 million AI chatbot — an animated sun named “Ed” celebrated as an unprecedented learning acceleration tool until the company that built it collapsed and the district was forced to pull the plug — put students’ personal information in peril.

Investigators with the Los Angeles Unified School District’s inspector general’s office conducted a video interview with Chris Whiteley, the former senior director of software engineering at AllHere, after he told The 74 his former employer’s student data security practices violated both industry standards and the district’s own policies.

Whiteley told The 74 he had alerted the school district, the IG’s office and state education officials earlier to the data privacy problems with Ed but got no response. His meeting with investigators occurred July 2, one day after The 74 published its story outlining Whiteley’s allegations, including that the chatbot put students’ personally identifiable information at risk of getting hacked by including it in all chatbot prompts, even in those where the data weren’t relevant; sharing it with other third-party companies unnecessarily and processing prompts on offshore servers in violation of district student privacy rules.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

In an interview with The 74 this week, Whiteley said the officials from the district’s inspector general’s office “were definitely interested in what I had to say,” as speculation swirls about the future of Ed, its ed tech creator AllHere and broader education investments in artificial intelligence.

“It felt like they were after the truth,” Whiteley said, adding, “I’m certain that they were surprised about how bad [students’ personal information] was being handled.”

To generate responses to even mundane prompts, Whiteley said, the chatbot processed the personal information for all students in a household. If a mother with 10 children asked the chatbot a question about her youngest son’s class schedule, for example, the tool processed data about all of her children to generate a response.

“It’s just sad and crazy,” he said.

The inspector general’s office directed The 74’s request for comment to a district spokesperson, who declined to comment or respond to questions involving the inquiry.

While the conversation centered primarily on technical aspects related to the company’s data security protocols, Whiteley said investigators probed him on his personal experiences with AllHere, which he described as being abusive, and its finances.

Whiteley was laid off from AllHere in April. Two months later, a notice posted to the company’s website said a majority of its 50 or so employees had been furloughed due to its “current financial position” and the LAUSD spokesperson said company co-founder and CEO Joanna Smith-Griffin had left. The former Boston teacher and Harvard graduate was successful in raising $12 million in venture capital for AllHere and appeared with L.A. schools Superintendent Alberto Carvalho at ed tech conferences and other events throughout the spring touting the heavily publicized AI tool they partnered to create.

Just weeks ago, Carvalho spoke publicly about how the project had put L.A. out in front as school districts and ed tech companies nationally race to follow the lead of generative artificial intelligence pioneers like ChatGPT. But the school chief’s superlative language around what Ed could do on an individualized basis with 540,000 students had some industry observers and AI experts speculating it was destined to fail .

The chatbot was supposed to serve as a “friendly, concise customer support agent” that replied “using simple language a third grader could understand” to help students and parents supplement classroom instruction, find assistance with kids’ academic struggles and navigate attendance, grades, transportation and other key issues. What they were given, Whiteley charges, was a student privacy nightmare.

Smith-Griffin recently deactivated her LinkedIn page and has not surfaced since her company went into apparent free fall. Attempts to reach AllHere for comment were unsuccessful and parts of the company website have gone dark. LAUSD said earlier that AllHere is for sale and that several companies are interested in acquiring it.

The district has already paid AllHere $3 million to build the chatbot and “a fully-integrated portal” that gave students and parents access to information and resources in a single location, the district spokesperson said in a statement Tuesday, and “was surprised by the financial disruption to AllHere.”

AllHere’s collapse represents a stunning fall from grace for a company that was named among the world’s top education technology companies by Time Magazine just months earlier. Scrutiny of AllHere intensified when Whiteley became a whistleblower. He said he turned to the press because his concerns, which he shared first with AllHere executives and the school district, had been ignored.

Whitely shared source code with The 74 which showed that students’ information had been processed on offshore servers. Seven out of eight Ed chatbot requests, he said, were sent to places like Japan, Sweden, the United Kingdom, France, Switzerland, Australia and Canada.

‘How are smaller districts going to do this?’

What district leaders failed to do as they heralded their new tool, Whiteley said, is conduct sufficient audits. As L.A. — and school systems nationwide — contract with a laundry list of tech vendors, he said it’s imperative that they understand how third-party companies use students’ information.

“If the second-biggest district can’t audit their [personally identifiable information] on new or interesting products and can’t do security audits on external sources, how are smaller districts going to do this?” he asked.

Over the last several weeks, the district’s official position on Ed has appeared to shift. In late June when the district spokesperson said that several companies were “interested in acquiring Allhere,” they also said its predecessor would “continue to provide this first-of-its-kind resource to our students and families.” In its initial response to Whiteley’s allegations published July 1, the spokesperson said that education officials would “take any steps necessary to ensure that appropriate privacy and security protections are in place in the Ed platform.”

In a story two days later in the Los Angeles Times, a district spokesperson said the chatbot had been unplugged on June 14. The 74 asked the spokesperson to provide documentation showing the tool was disabled last month but didn’t get a response.

Even after June 14, Carvalho continued to boast publicly about LAUSD’s foray into generative AI and what he described as its stringent data privacy requirements with third-party vendors.

On Tuesday, the district spokesperson told The 74 that the online portal — even without a chatty, animated sun — “will continue regardless of the outcome with AllHere.” In fact, the project could become a source of district revenue. Under the contract between AllHere and LAUSD, which was obtained by The 74, the chatbot is the property of the school district, which was set to receive 2% in royalty payments from AllHere “should other school districts seek to use the tool to benefit their families and students.”

In the statement Tuesday, the district spokesperson said that officials chose to “temporarily disable the chatbot” amid AllHere’s uncertainty and that it would “only be restored when the human-in-the-loop aspect is re-established.”

Whiteley agreed that the district could maintain the student information dashboard without the chatbot and, similarly, that another firm could buy what remains of AllHere. He was skeptical, however, that Ed the chatbot would live another day because “it’s broken”

“The name AllHere,” he said, “I think is dead.”