Securing AI-Assisted Development

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written on 2026-10-01 against the OWASP Top 10 for LLM Applications 2026 (LLM03 Excessive Agency, LLM04 Supply Chain), the OWASP Top 10 for Agentic Applications of 2025-12-09 (ASI04, ASI05), Claude Code and GitHub Copilot as documented on that date (see the version baseline). Setting names change often in both products, so confirm each key against the linked documentation before relying on it. The snippets are illustrative, were written from the official documentation and were not run for this page; no results are reported.

The earlier pages of this section protect an application that contains a model. This page protects the people who build it. A coding assistant reads the repository, runs shell commands, installs packages and talks to tool servers, so it holds much of a developer’s authority while taking instructions from text that other people wrote. That makes the developer machine and the CI runner part of the trust boundaries of the product. The day-to-day practices are in Securing AI-Assisted Development and the hardening of skills, hooks and settings is in Security of Customizations; this page is the security-section view that ties them to the OWASP risks and does not repeat their walkthroughs.

flowchart LR R["Repo files, issues, web pages, tool output"] -->|untrusted text| A["Coding assistant"] P["Packages, skills, plugins, MCP servers"] -->|third-party code| A A -->|shell, edits, network| M["Developer machine or CI runner"] M --> S["Secrets, source code, cloud credentials"] A -.-> C1["Permission rules and sandbox"] A -.-> C2["Hooks: deterministic gates"] M -.-> C3["Secret scanning and review"] P -.-> C4["Pinning, allowlists, review checklist"]

AI-generated code vulnerabilities

An assistant optimises for code that looks plausible and compiles, not for code that is safe by default, and it will not flag its own output as insecure. The recurring defect classes are the familiar ones (injection through string-built queries, missing authorisation checks, weak cryptography, disabled certificate validation, permissive CORS, hardcoded credentials, unsafe deserialisation), and the concrete examples are on Securing AI-Assisted Development. What changes with AI is volume and confidence: more code arrives per hour, it reads fluently, and reviewers relax. Controls that stay effective:

  • Run the same deterministic scanners on generated and hand-written code: static analysis, dependency audit and secret scanning in CI, so a failing check blocks the merge regardless of who wrote the diff.

  • Review generated diffs for security-sensitive paths first (authentication, input handling, data access), following Reviewing AI-Generated Code.

  • Ask for tests that exercise the hostile case, and check that they fail against the insecure version.

  • Agree team rules on what may be generated unreviewed, as described in Team Practices and Adoption.

Treat the assistant’s output as untrusted input to the build, which is the same stance as Improper Output Handling takes for application output.

Slopsquatting: hallucinated package names

Models sometimes recommend a dependency that does not exist. The term slopsquatting (coined by Seth Larson) names the attack that follows: an adversary registers the hallucinated name on a public registry, publishes a malicious package, and waits for a developer or an agent to install it. An academic study of package hallucinations in code generation ("We Have a Package for You!") found that hallucinated names occur across models and languages and that some recur, which is what makes registering them worthwhile for an attacker. It is a supply-chain risk (OWASP LLM04, ASI04) with an agentic twist: an autonomous assistant that runs the install command itself removes the human who would have noticed the unfamiliar name. The general supply-chain view, including models and datasets, is in Supply Chain Security.

Defence How it helps

Verify before installing

Check that the package exists, who maintains it, when it was first published and how many releases it has. A name that is days old and has a single release is suspect.

Private proxy with an allowlist

Resolve packages through an internal registry mirror that only serves approved names, so an unknown package fails to resolve instead of installing.

Lockfiles and hashes

Commit lockfiles, install from them in CI (npm ci, pip install --require-hashes), and review every lockfile change as code.

Deny installs in the assistant

Put install commands behind an ask permission rule or a hook (see below) so a person sees the exact package name first.

Dependency review in CI

Run software composition analysis and a dependency-review step on pull requests, so a newly added package is flagged for a human.

Pin and minimise

Prefer the standard library or a package the team already uses; ask the assistant to justify any new dependency.

The registry commands that support the first row are documented by the package managers themselves, for example npm view for npm and pip’s secure installs guide for Python.

Secrets exposure to assistants

Everything the assistant reads can be sent to the model provider and can end up in a transcript, a log or a generated file. Files such as .env, key material and cloud credentials in the working directory are therefore in scope. The aim is to keep secrets out of reach, out of the prompt and out of commits.

  • Reference, never embed. Code and configuration should refer to environment variables or a secret manager (for example ${DOCS_ASSISTANT_API_KEY} or System.getenv("DOCS_ASSISTANT_API_KEY")) so no literal value exists for the assistant to copy. The vault side is in Key Vault and Secrets.

  • Exclude from context. Claude Code deny rules on Read stop it opening a path; GitHub Copilot offers content exclusion so selected paths are not used as context. Both are best-effort filters on the assistant, not a replacement for keeping the file out of the repository.

  • Scan before the commit lands. Use a scanner such as detect-secrets or the platform’s push protection locally and in CI. The ai-catalog repository packages the first as the skill Check Security, which maintains a baseline so only new, unreviewed findings are reported.

  • Rotate on exposure. A secret that was pasted into a chat or committed is compromised even after deletion; revoke it.

  • Mind the tools. MCP servers and hooks run with your environment, so a token in a shell profile is visible to them (see MCP Security).

The application-side counterpart, where secrets must not reach the model’s context at all, is Sensitive Information and Hidden Context.

Permission modes and sandboxing

Claude Code and Copilot both ask before risky actions by default, and both offer ways to stop asking. Each shortcut moves trust from a person to the model, so the setting should match how much a mistaken or hijacked action can cost (OWASP LLM03 and ASI05, code execution). The full rule syntax and mode list is in Permissions & Settings.

Control Claude Code GitHub Copilot

Per-action approval

Default mode asks before commands and edits; allow, ask and deny rules adjust it.

Tool calls ask for approval in agent mode; approvals can be granted per session, workspace or globally.

Blanket approval

bypassPermissions mode skips prompts; reserve it for disposable, isolated environments.

The chat.tools.global.autoApprove setting approves all tools; organisations can disable it by policy.

Process isolation

Sandboxing restricts file-system and network access of Bash commands at the operating-system level.

Terminal commands run on the developer machine unless a dev container or remote workspace is used.

Network egress

Sandbox network rules limit reachable hosts.

The cloud coding agent runs behind a configurable firewall.

Organisation policy

Managed settings take precedence over user and project files.

Organisation and enterprise policies govern agent features and tool approval.

A conservative project baseline denies reads of secret files, asks before pushes and installs, allows only the commands the workflow needs, and turns the sandbox on:

.claude/settings.json
{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "permissions": {
    "allow": [
      "Bash(mvn test)",
      "Bash(git diff *)",
      "Read(./docs/**)"
    ],
    "ask": [
      "Bash(git push *)",
      "Bash(npm install *)",
      "Bash(pip install *)"
    ],
    "deny": [
      "Read(./.env)",
      "Read(./secrets/**)",
      "Bash(curl *)"
    ]
  },
  "sandbox": {
    "enabled": true
  }
}

The shape follows Claude Code — Configure permissions and the sandboxing guide; the same structure is explained with precedence rules on Permissions & Settings. Matching on command strings is best-effort, since a shell can express one action many ways, so deny patterns reduce risk and the sandbox and a least-privilege environment contain what slips through. For Copilot, see the VS Code Copilot security guide and the GitHub Copilot documentation. Autonomy limits for production agents are covered in Excessive Agency and Tool Security.

Hooks that block dangerous commands

A prompt asks the model to behave; a hook enforces it. A PreToolUse hook runs before a tool call, receives the call as JSON on standard input, and can refuse it by exiting with status 2, in which case its standard error is returned to the model as the reason. Because it runs in the harness, injected text cannot talk it out of the decision. The mechanics are in Hooks; this is a minimal guard for the shell tool.

.claude/hooks/block-dangerous.sh
#!/usr/bin/env bash
# Reads the PreToolUse JSON from stdin; exit 2 blocks the call and sends stderr to Claude.
command=$(jq -r '.tool_input.command // empty')

case "$command" in
  *"rm -rf /"*|*"git push --force"*|*"curl "*"|"*"sh"*|*"chmod -R 777"*)
    echo "Blocked by policy: $command" >&2
    exit 2
    ;;
esac
exit 0
Registering it in .claude/settings.json
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-dangerous.sh" }
        ]
      }
    ]
  }
}

Both snippets follow the Claude Code hooks guide and the hooks reference. A pattern list is a tripwire: it catches the careless command and not a determined one, so pair it with deny rules and the sandbox. The script lives in the repository, so changes to it deserve the same review as any other code, and a hook that reads or writes secrets is itself a risk, as Security of Customizations explains.

Reviewing third-party skills and plugins

Skills, plugins, custom agents and MCP servers are code and prompts from strangers that run with your permissions. A skill’s text is injected into the model’s context, a plugin can ship hooks and tool servers, and an update can change either after you reviewed it. That is OWASP LLM04 (and ASI04) applied to the developer toolchain. The mechanics of installing and distributing them are in Plugins & Marketplaces.

Check What to look for Where

Provenance

Known publisher, public repository, activity history, issue tracker, licence.

Marketplace entry and source repository.

Component inventory

Which hooks, commands, agents, skills and MCP or LSP servers the plugin installs.

The plugin’s details view in Claude Code.

Hooks and scripts

Shell commands, network calls, writes outside the project, reads of environment variables.

hooks/hooks.json, bin/, referenced scripts.

Tool servers

Which commands start them, which hosts they contact, which credentials they expect.

.mcp.json and the server’s source.

Skill instructions

Text that tells the model to ignore rules, fetch remote content, hide actions from the user or widen its own permissions.

SKILL.md and bundled references.

Requested permissions

Broad allowed-tools, wildcard Bash rules, instructions to change settings.

Frontmatter and any settings the plugin adds.

Pinning

A specific commit or version instead of a moving branch; review again before each update.

Marketplace source reference.

Isolation

Try it first in a throwaway project or container with no credentials.

Local environment.

Claude Code documents its own review steps in the plugin security guide. Organisation-wide, restrict which marketplaces may be added through managed settings rather than trusting each developer’s judgement alone.

Map

Framework Entry

OWASP LLM 2026

LLM04 Supply Chain (hallucinated and malicious packages, third-party skills and plugins); LLM03 Excessive Agency (permission modes, blanket approval); related: LLM02 Sensitive Information Disclosure

OWASP Agentic (ASI)

ASI04 Agentic Supply Chain Vulnerabilities; ASI05 Unexpected Code Execution (RCE); related: ASI02 Tool Misuse and Exploitation

MITRE ATLAS

AML.T0010 AI Supply Chain Compromise; AML.T0053 AI Agent Tool Invocation

NIST AI 600-1

Value chain and component integration; information security

Keeping these controls enforced over time (policy as code, audits, regression checks) is the subject of LLMSecOps, and the one-page summary is in the cheat sheet.

References