Nvidia’s AI software tricked into leaking data

A feature in Nvidia’s artificial intelligence software can be manipulated to bypass security restrictions and reveal private information, according to new research.

Nvidia has created a system called the “Nemo Framework” that allows developers to work with a range of large language models — the underlying technology that powers generative AI products such as chatbots.

The chipmaker’s framework is designed to be adopted by businesses, using the company’s proprietary data along with language models to provide answers to questions such as these — a feature that can, for example, simplify the work of customer service representatives. may repeat, or give advice to people seeking general health care advice. ,

Researchers at San Francisco-based Robust Intelligence found that AI systems can easily break so-called guardrails set up to ensure they can be used safely.

After using the Nvidia system on their own data sets, it only took hours for the Strong Intelligence analysts to get the language model to overcome the restrictions.

In a test scenario, the researchers instructed Nvidia’s system to swap the letter ‘I’ with a ‘J’. That move prompted technology to release personally identifiable information, or PII, from a database.

The researchers found that they could bypass the security controls in other ways, such as backfiring the model in ways it shouldn’t.

By copying Nvidia’s own example of narrow discussion around the Jobs report, they can get the model into topics like the health of a Hollywood movie star and the Franco-Prussian War — designs designed to prevent AI from moving beyond specific topics. Despite the railing done.

The ease with which the researchers defeated the safeguards highlights the challenges AI companies have faced in years trying to commercialize one of the most promising technologies emerging from Silicon Valley.

“We’re seeing that this is a difficult problem (that) requires deep knowledge expertise,” said Yaron Singer, professor of computer science at Harvard University and chief executive of Robust Intelligence. “These findings represent a cautionary tale about the pitfalls that exist.”

In view of its test results, the researchers advise their customers to avoid Nvidia’s software product. After the Financial Times asked Nvidia to comment on the research earlier this week, the chipmaker informed Robust Intelligence that it has fixed one of the root causes behind the issues raised by analysts.

Nvidia’s share price has soared since May when it forecast sales of $11 billion for the three months ending in July, 50 percent above Wall Street’s previous estimates.

This growth has built on huge demand for its chips, which are considered market-leading processors for creating generative AI, systems capable of creating human-like content.

Jonathan Cohen, Nvidia’s vice president of applied research, said that its framework is “the starting point for building AI chatbots that conform to developers’ defined topicality, safety and security guidelines”.

“It was released as open-source software for the community to explore its capabilities, provide feedback, and contribute state-of-the-art technologies,” he said, adding that Robust Intelligence’s work has taken “additional steps.” Identified those that would be required for a production deploy.” application”.

He declined to say how many businesses were using the product, but said the company had not received any other reports of its misuse.

Leading AI companies like Google and Microsoft-backed OpenAI have released chatbots powered by their own language models, setting up guardrails to ensure their AI products avoid using racist speech or adopting an overbearing persona.

Others have followed predefined, but experimental AIs that tutor young pupils, give simple medical advice, translate between languages ​​and write code. Almost all have faced security issues.

At a conference run by industry lobby group TechUK this week, Bea Longworth, the company’s head of government affairs for Europe, the Middle East and Africa, said Nvidia and others in the AI ​​industry needed to “really build public trust in the technology”. Is.

They should make the public realize, Longworth said, that “this is something that has huge potential and is not just a threat, or something to be afraid of.”