FTC Investigates OpenAI Over Data Leak and ChatGPT's Inaccuracy

The Federal Trade Commission has opened an expansive investigation into OpenAI, probing whether the maker of the popular ChatGPT bot has run afoul of consumer protection laws by putting personal reputations and data at risk.

The agency this week sent the San Francisco company a 20-page demand for records about how it addresses risks related to its AI models, according to a document reviewed by The Washington Post. The salvo represents the most potent regulatory threat to date to OpenAI's business in the United States, as the company goes on a global charm offensive to shape the future of artificial intelligence policy.

Analysts have called OpenAI's ChatGPT the fastest-growing consumer app in history, and its early success set off an arms race among Silicon Valley companies to roll out competing chatbots. The company's chief executive, Sam Altman, has emerged as an influential figure in the debate over AI regulation, testifying on Capitol Hill, dining with lawmakers and meeting with President Biden and Vice President Harris.

But now the company faces a new test in Washington, where the FTC has issued multiple warnings that existing consumer protection laws apply to AI, even as the administration and Congress struggle to outline new regulations. Senate Majority Leader Charles E. Schumer (D-N.Y.) has predicted that new AI legislation is months away.

The FTC's demands of OpenAI are the first indication of how it intends to enforce those warnings. If the FTC finds that a company violates consumer protection laws, it can levy fines or put a business under a consent decree, which can dictate how the company handles data. The FTC has emerged as the federal government's top Silicon Valley cop, bringing large fines against Meta, Amazon and Twitter for alleged violations of consumer protection laws.

The FTC called on OpenAI to provide detailed descriptions of all complaints it had received of its products making "false, misleading, disparaging or harmful" statements about people. The FTC is investigating whether the company engaged in unfair or deceptive practices that resulted in "reputational harm" to consumers, according to the document.

The FTC also asked the company to provide records related to a security incident that the company disclosed in March when a bug in its systems allowed some users to see payment-related information, as well as some data from other users' chat history. The FTC is probing whether the company's data security practices violate consumer protection laws. OpenAI said in a blog post that the number of users whose data was revealed to someone else was "extremely low."

The FTC declined to comment. OpenAI CEO Sam Altman said in a tweet Thursday evening that the company will "of course" work with the agency.

"it is very disappointing to see the FTC's request start with a leak and does not help build trust," he tweeted. "that said, it's super important to us that out [sic] technology is safe and pro-consumer, and we are confident we follow the law."

Altman also said that the company protects user privacy and designs its systems "to learn about the world, not private individuals."

News of the probe broke shortly before FTC Chair Lina Khan faced a combative hearing Thursday before the House Judiciary Committee, where Republican lawmakers analyzed her enforcement record and accused her of mismanaging the agency. Khan's ambitious plans to rein in Silicon Valley have suffered key losses in court. On Tuesday, a federal judge rejected the FTC's attempt to block Microsoft's $69 billion deal to buy the video game company Activision.

During the hearing, Rep. Dan Bishop (R-N.C.) asked Khan what legal authority empowered the FTC to make such demands of a company like OpenAI, as part of a broader line of inquiry into whether Khan's FTC is overstepping its powers. He noted that libel and defamation are typically prosecuted under state laws, a reference to the FTC's questions to OpenAI about disparagement of people.

Khan responded that libel and defamation aren't a focus of FTC enforcement but that misuse of people's private information in AI training could be a form of fraud or deception under the FTC Act. "We're focused on, 'Is there substantial injury to people?' Injury can look like all sorts of things," Khan said.

The agency has repeatedly warned that action is coming on AI, in speeches, blog posts, op-eds and news conferences. In a speech at Harvard Law School in April, Samuel Levine, the director of the agency's Bureau of Consumer Protection, said the agency was prepared to be "nimble" in getting ahead of emerging threats.

"The FTC welcomes innovation, but being innovative is not a license to be reckless," Levine said. "We are prepared to use all our tools, including enforcement, to challenge harmful practices in this area."

The FTC also has issued several colorful blog posts about its approach to regulating AI, at times invoking popular science fiction movies to warn the industry against running afoul of the law. The agency has warned against AI scams, using generative AI to manipulate potential customers and falsely exaggerating the capabilities of AI products. Khan also participated in a news conference with Biden administration officials in April about the risk of AI discrimination.

"There is no AI exemption to the laws on the books," Khan said at that event.

The FTC's push faced swift pushback from the tech industry. Adam Kovacevich, the founder and CEO of the industry coalition Chamber of Commerce, said it's clear that the FTC has oversight of data security and misrepresentation. But he said it's unclear if the agency has the authority to "police defamation or the contents of ChatGPT's results."

"AI is making headlines right now, and the FTC is continuing to put flashy cases over securing results," he said.

Among the information the FTC is seeking from OpenAI is any research, testing or surveys that assess how well consumers understand "the accuracy or reliability of outputs" generated by its AI tools. The agency made extensive demands about records related to ways OpenAI's products could generate disparaging statements, asking the company to provide records of the complaints people send about its chatbot making false statements.

The agency's focus on such fabrications comes after numerous high-profile reports of the chatbot producing incorrect information that could damage people's reputations. Mark Walters, a radio talk show host in Georgia, sued OpenAI for defamation, alleging the chabot made up legal claims against him. The lawsuit alleges that ChatGPT falsely claimed that Walters, the host of "Armed American Radio," was accused of defrauding and embezzling funds from the Second Amendment Foundation. The response was provided in response to a question about a lawsuit about the foundation that Walters is not a party to, according to the complaint.

ChatGPT also said that a lawyer had made sexually suggestive comments and attempted to touch a student on a class trip to Alaska, citing an article that it said had appeared in The Washington Post. But no such article existed, the class trip never happened and the lawyer said he was never accused of harassing a student, The Post reported previously.

The FTC in its request also asked the company to provide extensive details about its products and the way it advertises them. It also demanded details about the policies and procedures that OpenAI takes before it releases any new product to the public, including a list of times that OpenAI held back a large language model because of safety risks.

The agency also demanded a detailed description of the data that OpenAI uses to train its products, which mimic humanlike speech by ingesting text, mostly scraped from Wikipedia, Scribd and other sites across the open web. The agency also asked OpenAI to describe how it refines its models to address their tendency to "hallucinate," making up answers when the models don't know the answer to a question.

OpenAI also has to turn over details about how many people were affected by the March security incident and information about all the steps it took to respond.

The FTC's records request, which is called a Civil Investigative Demand, primarily focuses on potential consumer protection abuses, but it also asks OpenAI to provide some details about how it licenses its models to other companies.

The United States has trailed other governments in drafting AI legislation and regulating the privacy risks associated with the technology. Countries within the European Union have taken steps to limit U.S. companies' chatbots under the bloc's privacy law, the General Data Protection Regulation. Italy temporarily blocked ChatGPT from operating there due to data privacy concerns, and Google had to postpone the launch of its chatbot Bard after receiving requests for privacy assessments from the Irish Data Protection Commission. The European Union is also expected to pass AI legislation by the end of the year.

There is a flurry of activity in Washington to catch up. On Tuesday, Schumer hosted an all-senator briefing with officials from the Pentagon and intelligence community to discuss the national security risks of artificial intelligence, as he works with a bipartisan group of senators to craft new AI legislation. Schumer told reporters after the session that it's going to be "very hard" to regulate AI, as lawmakers try to balance the need for innovation with ensuring there are proper safeguards on the technology.

On Wednesday, Vice President Harris hosted a group of consumer protection advocates and civil liberties leaders at the White House for a discussion on the safety and security risks of AI.

"It is a false choice to suggest that we either can advance innovation or we protect consumers," Harris said. "We can do both."

— Cat Zakrzewski, The Washington Post