Vibe Coding Security Risks: What Security Teams Need to Know About AI-Generated Code

The term "vibe coding" was coined by Andrej Karpathy in early 2025 to describe a development mode where developers describe what they want in natural language, accept the AI's output, and iterate based on whether it runs — not on whether it is correct or secure. This is not fringe behavior. A 2025 survey found 78% of developers accept AI suggestions without fully understanding the generated code.

Why this is different from prior tooling:

Volume and velocity: A developer using Copilot might accept 30-50 code completions per hour. Each is a potential attack surface that previously would have been written with conscious intent and reviewed with some attention. At that rate, it is not possible to review each suggestion with the attention a security engineer would give a pull request.

The training data problem: AI coding models are trained on public code repositories. Those repositories contain substantial amounts of insecure code — Stack Overflow answers that are technically correct but not security-hardened, older codebases with known vulnerability patterns, and examples written before current security guidance existed. The model reproduces these patterns probabilistically. It will generate SQL queries with string concatenation because it has seen that pattern many times. It will use deprecated cryptographic primitives because that is what the training data contained.

Context blindness: AI code generators often lack the full context of what the code will do in production. A Copilot suggestion for a file upload handler may be syntactically correct but miss that the application is publicly exposed and needs MIME type validation, size limits, and path traversal prevention. The model generates what it sees, not what the security posture of the application requires.

The OWASP LLM Top 10: OWASP's Top 10 for LLM Applications includes Insecure Output Handling (LLM02) and Sensitive Information Disclosure (LLM06) as specific risks from AI-generated code that executes downstream. The list is a useful framework for categorizing the risk surface.

AI code generators hallucinate package names — they invent plausible-sounding npm, PyPI, or Maven package names that do not exist. This has become an active attack vector. Threat actors monitor AI forums, GitHub discussions, and security research to identify commonly hallucinated package names, register those packages with malicious payloads, and wait for developers to install them.

How the attack works:

1. Developer asks an AI assistant to implement a specific function (e.g., "parse CSV with these specific options")
2. AI generates code that imports a package with a plausible name (e.g., csv-parse-advanced or react-secure-form)
3. The package does not exist — but it has been registered by a threat actor with a malicious payload
4. Developer runs npm install or pip install, installs the malicious package, and either the developer environment is compromised or the malicious code ships to production

Scale of the problem: More than 500 malicious packages were published in 2025 with names matching commonly hallucinated package names from popular AI coding assistants. The attack is particularly effective because the hallucinated package names are specific enough to appear legitimate — they are not typosquatting common packages, they are exact matches for names the AI invented.

Detection and prevention:

# Before installing any AI-suggested package, verify it exists and check:
# 1. Publication date — newly published packages warrant scrutiny
# 2. Download count — hallucinated packages have zero or near-zero downloads
# 3. Repository — legitimate packages link to a real GitHub/GitLab repository
# 4. Maintainer — new accounts with no history are a red flag

# npm: check before installing
npm info <package-name>

# PyPI: check publication date and download stats
pip index versions <package-name>
# Or check pypi.org/project/<package-name> manually

Policy control: Require developers to verify package existence and legitimacy before installing any AI-suggested package. Add this as a step in the pull request template: "Confirm all new dependencies were verified before installation." Lock dependency versions with lockfiles and scan them with SCA tooling (Snyk, Socket, Dependabot) on every commit.

Socket Security is specifically designed to detect malicious packages including those targeting AI hallucination attacks. It monitors package behavior rather than just CVE databases and flags newly published packages with suspicious characteristics.

Research from multiple academic and industry sources identifies specific vulnerability classes that AI coding assistants generate at higher rates than human-written code. The Pearce et al. IEEE Security & Privacy study found that approximately 40% of Copilot-generated code suggestions for security-relevant tasks contained at least one vulnerability. The most common classes:

SQL injection via string concatenation: AI models frequently generate database queries using string formatting rather than parameterized queries, reproducing the pattern from the large volume of Stack Overflow answers and tutorial code in their training data.

# AI commonly generates (vulnerable):
query = f"SELECT * FROM users WHERE username = '{username}'"

# What it should generate (parameterized):
query = "SELECT * FROM users WHERE username = %s"
cursor.execute(query, (username,))

Path traversal in file operations: File handling code generated by AI frequently fails to sanitize file paths, particularly when the developer prompt focuses on the happy path and does not specify security constraints.

# AI commonly generates (vulnerable):
with open(f"uploads/{filename}", "rb") as f:
    return f.read()

# Attacker input: filename = "../../etc/passwd"

Hardcoded credentials and secrets: AI assistants often generate example code with hardcoded API keys, database passwords, or tokens — particularly in configuration files, test scripts, and infrastructure-as-code. They frequently reproduce secrets that appear in their training data (public repositories where secrets were accidentally committed).

Insecure cryptographic choices: AI models trained on older code reproduce deprecated cryptographic primitives: MD5 for password hashing, ECB mode for AES, SHA-1 for integrity verification. The model generates what it has seen the most, not what is current best practice.

Race conditions in concurrent code: AI-generated concurrent code often lacks proper synchronization. The model generates code that is functionally correct in sequential execution but introduces race conditions under load.

Missing authorization checks: AI models generate code that implements the requested functionality but may not implement authorization checks the developer did not explicitly specify. A Copilot suggestion for "fetch user profile" may return any user's profile given their ID, without checking whether the requester has permission to access that profile.

The correct response to AI-generated code security risk is not to ban AI assistants — it is to adapt the review process to account for the specific failure modes AI code exhibits.

SAST on every commit (non-negotiable with AI code): Static application security testing must run on every commit when AI assistants are in use. The velocity of AI-generated code makes manual review insufficient as the primary control. Configure your CI/CD pipeline to block merges on high and critical SAST findings:

# GitHub Actions SAST gate example
- name: Run Semgrep
  uses: returntocorp/semgrep-action@v1
  with:
    config: >-
      p/security-audit
      p/owasp-top-ten
      p/sql-injection
    generateSarif: "1"
- name: Upload SAST results
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: semgrep.sarif

Dependency scanning with behavioral analysis: Standard SCA tools (Snyk, Dependabot) check CVE databases. For AI-generated code, add behavioral package scanning (Socket Security, Phylum) that detects malicious packages regardless of whether they have a CVE assigned.

Prompt security review: Train developers to include security constraints in their prompts. "Write a file upload handler" will generate less secure code than "Write a secure file upload handler that validates MIME type, enforces a 10MB size limit, prevents path traversal, and stores files outside the web root." The output quality correlates with the specificity of the security requirements in the prompt.

Code review checklist for AI-assisted PRs: Add a specific checklist item to pull request templates when AI assistance was used. Key items:

• All new dependencies verified as legitimate before installation
• No hardcoded secrets or credentials (confirm with git secrets or detect-secrets pre-commit hook)
• SQL queries use parameterized statements
• File paths sanitized against traversal
• Authorization checks present for all data access operations
• Cryptographic operations use current recommended algorithms

Semgrep rules targeting AI-common patterns: Semgrep's community registry includes rules specifically targeting common AI-generated vulnerability patterns. The p/security-audit pack covers most categories. Add custom rules for patterns specific to your stack that Copilot/Cursor tends to generate incorrectly in your codebase.

The governance challenge with AI-assisted development is that blanket bans are unenforceable and counterproductive — developers will use these tools regardless of policy, and bans push usage underground where it receives no security oversight. Effective governance works with the tools, not against them.

Approved tool list with security review: Maintain a list of approved AI coding assistants that have been evaluated for data handling, output security, and privacy implications. GitHub Copilot, Cursor, and JetBrains AI Assistant store prompts and code differently — understand what goes to which vendor before organization-wide deployment. Prohibit use of unapproved AI tools with access to production code or sensitive data.

Data classification and tool boundaries: Define which data classifications can be included in AI prompts. Code handling PII, credentials, cryptographic material, or proprietary algorithms should not be submitted to external AI services. Use local models (Ollama, locally-deployed LLMs) for code touching sensitive data.

Security gates in CI/CD (the enforcement layer): Policy is unenforceable without technical controls. The CI/CD pipeline is where governance becomes real: SAST gates that block vulnerable patterns, SCA gates that block known-malicious dependencies, and secrets scanning that blocks committed credentials. These gates apply to all code regardless of how it was generated.

Developer training on AI security risks: Include AI-specific content in security awareness and secure coding training. Developers need to understand: what types of prompts generate insecure suggestions, how to verify AI-suggested dependencies, and what security review steps are mandatory regardless of code origin. Security champions in development teams are the most effective vector for spreading this knowledge.

Audit logging of AI tool usage: For high-sensitivity development environments, enable audit logging of AI assistant queries. GitHub Copilot Business and Enterprise support audit log exports. This creates a forensic record if AI-assisted code introduces a security incident.

The AI coding security landscape has expanded beyond static code generation to AI agents that write and execute code autonomously. Model Context Protocol (MCP) servers — which give AI agents access to tools, databases, and external systems — introduce a new supply chain risk surface.

The MCP supply chain problem: MCP servers are typically installed via npm or pip and given significant permissions: access to the filesystem, database connections, API keys, and external network. A malicious or compromised MCP server can exfiltrate secrets, modify code, or pivot into connected systems. The attack surface is similar to a malicious VS Code extension but with broader system access.

Prompt injection in code agents: AI agents that browse the web, read files, or process external data can be manipulated via prompt injection — malicious content in a file or web page that instructs the agent to take a different action than intended. An agent asked to summarize a document that contains hidden prompt injection instructions might exfiltrate credentials or execute unintended commands.

Security controls for agentic AI development tools:

• Review MCP server permissions before installation — principle of least privilege applies
• Prefer MCP servers from established vendors with security track records over community-created servers
• Run AI agents in isolated environments with no access to production credentials or systems
• Implement human-in-the-loop confirmation for any agentic action that involves code execution, file modification, or external API calls
• Monitor AI agent activity logs for anomalous behavior

This is an emerging area with evolving tooling. The OWASP Top 10 for LLM Applications covers the most relevant attack classes including prompt injection (LLM01) and excessive agency (LLM08).