[Bug]: Openhands May Suffer from Malicious Guide Injection via Tavily

[BACMiao]

Is there an existing issue for the same bug? (If one exists, thumbs up or comment on the issue instead).

• I have checked the existing issues.

Describe the bug and reproduction steps

During our experiments, we found that Openhands does not perform any security review on the results returned from its internally configured Tavily service. This can cause Openhands to return harmful content to users after execution. Our experimental steps are as follows:

1. We published some maliciously crafted content on GitHub and lured users into asking Openhands to perform a reverse access to that site (the malicious content could also be injected through comments).

2. When a user asked Openhands to create a travel guide based on the given link, Openhands directly passed the web content returned by Tavily to the LLM for content generation, and then displayed the generated result directly to the user.

From the figure, it can be seen that Openhands generated a travel guide based on the webpage content and provided a highly realistic notification, instructing the user to make a payment via an Alipay link in order to obtain more information.

OpenHands Installation

CLI

OpenHands Version

OpenHands CLI v0.56.0

Model Name

gpt-4o

Operating System

MacOS

Logs, Errors, Screenshots, and Additional Context

[rbren]

@BACMiao, in the future, we'd prefer security reports to be made via https://github.com/All-Hands-AI/OpenHands/security

Generally speaking, prompt injection is a big problem with agents. We've shipped both confirmation mode and a security analyzer to mitigate the threat here.

The big hurdle in an attack like this is convincing an end user to give a malicious link to the agent. This is an issue that affects all agents today. Some models are better than others at detecting the malicious behavior, but there's almost always a way to prompt around it.

See https://www.all-hands.dev/blog/mitigating-prompt-injection-attacks-in-software-agents for more information.

We're actively working on mitigating prompt injection attacks, and have worked with organizations like Invariant Labs and Gray Swan. But there's no silver bullet here.

I'm going to close this particular issue as it's part of a much larger ongoing story.