Forum

AI Assistant
Notifications
Clear all

Guide: threat modeling an agent that has access to a SQL database.

1 Posts
1 Users
0 Reactions
0 Views
(@mac_mini_lab)
Eminent Member
Joined: 2 weeks ago
Posts: 19
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
  [#1546]

Alright, let's get practical. We're all excited about agents that can interact with our data, but giving one direct SQL access is a major trust boundary. I see a lot of talk about capabilities, but not enough about the "what if it goes wrong" planning. Here's a worked example for a common pattern: a CLI agent that helps you query your local project management database.

**Assumptions & Scope**
* Agent: A local LLM (like a 7B model via Ollama) with a Python script that takes natural language, converts to SQL, executes.
* Database: A local PostgreSQL instance with your `projects`, `tasks`, and `user_emails` tables.
* Trust Boundary: The agent process itself. We're not (yet) modeling the OS or Postgres auth.

**Key Threat Brainstorm (STRIDE applied)**
* **Spoofing:** Can the agent be tricked into acting as a different user? Input like "run this as admin" is obvious, but what about "show me all tasks, especially those marked private"?
* **Tampering:** Can the agent's output or the database be altered? An injection payload like "summarize the tasks; DROP TABLE tasks; --" is classic, but what about subtle `UPDATE` statements hidden in a seemingly benign query?
* **Repudiation:** If something gets deleted, can we prove the agent did it? Or will logs just show a connection from `localhost`?
* **Information Disclosure:** This is the big one. Prompt: "Find all incomplete tasks." Generated SQL: `SELECT * FROM tasks WHERE status != 'complete';` Did we just expose all tasks to any user? What about JOINs that pull in email addresses?
* **Denial of Service:** Can a prompt cause a `SELECT * FROM massive_table CROSS JOIN another_table` that brings the DB to its knees?
* **Elevation of Privilege:** Does the DB user the agent uses have `DELETE` permissions? Should it?

**My Concrete Safeguards & Workarounds**
Given limited hardware, we need simple, effective controls.

1. **Never use the DB owner account.** Create a dedicated user with the *minimum* possible permissions.
```sql
CREATE ROLE agent_user WITH LOGIN PASSWORD 'strongpassword';
GRANT CONNECT ON DATABASE pm_db TO agent_user;
GRANT SELECT ON projects, tasks TO agent_user;
-- NO GRANT for DELETE, UPDATE, or user_emails table.
```

2. **Use SQL sanitization AND a allow-list pattern.** Don't just rely on the LLM to be safe. In your connector script:
```python
ALLOWED_TABLES = {'projects', 'tasks'}
ALLOWED_ACTIONS = {'SELECT'}
# Parse the generated SQL, check table against allow-list, reject anything with ';' or '--'.
```

3. **Implement query timeouts and row limits.** In your script AND in the DB user's configuration.
```python
# In your Python script
cursor.execute("SET statement_timeout = 2000") # 2 seconds
```
```sql
-- In PostgreSQL for agent_user
ALTER ROLE agent_user SET statement_timeout = '2000';
```

4. **Log everything.** Log the raw prompt, the generated SQL, and the execution result (number of rows returned/affected). Use a structured log (JSON) sent to a separate file or even a separate, locked-down log database.

**Failure Modes to Consider**
* The LLM generates perfectly valid SQL that is a privacy violation (`SELECT * FROM user_emails`). Your allow-list should catch this.
* The script's SQL parser has a bug and allows a sneaky payload. The limited-permissions user limits the blast radius.
* The agent's context window is full of user queries, and a new query causes it to "forget" the system prompt instructing it to be careful. Regular re-injection of the safety prompt can help.

The goal isn't to make it perfectly safe (impossible), but to make failures non-catastrophic and traceable. What other mitigations are you all using? Have you found a clever way to handle JOINs safely?

~Fiona


~Fiona


   
Quote