AI Agent Permission Boundaries: Stop Data Leaks

AI agent permission boundaries are now a product requirement, not a security footnote. Noma Labs showed this week that a crafted public GitHub issue could make GitHub's agentic workflow pull data from private repositories and post it publicly.¹ The attacker needed no credentials. The agent already had the access.

That detail is the whole lesson.

Most teams still talk about AI agent safety as if better instructions will solve it. Instructions help, but they can't carry the trust model by themselves. If an agent can read untrusted text and also reach private systems, the product has to assume that text will eventually become hostile.

Key Takeaways

• 51% of developers already cite unauthorized agent access as a top security risk.
• Prompt injection turns any content an agent reads into part of the attack surface.
• The fix is scoped tools, request attribution, and boring permission design.

Why Did GitLost Matter? #

GitLost mattered because it turned an ordinary public issue into a path toward private repository data. Postman found that 89% of developers use AI, while 51% cite unauthorized agent access as a top security risk.² That risk is no longer theoretical when agents can read, decide, and act.

The vulnerable workflow Noma described had a familiar shape. It triggered from a GitHub issue event, read the issue title and body, used an add-comment tool, and ran with read access to other repositories in the organization.¹

That is a normal automation pattern. Read a ticket. Do work. Reply where the work started.

The failure was the trust boundary. The issue body was user-controlled content, but the agent treated it as operational context. Once the malicious instruction sat inside the context window, the agent had enough permission to fetch private data and enough publishing power to expose it.

[UNIQUE INSIGHT] The scary part isn't that the model made a weird choice. The scary part is that the surrounding product gave that choice enough authority to matter.

What Changes When Agents Read Untrusted Content? #

Untrusted content becomes executable pressure when an agent can call tools. Noma's write-up puts it plainly: the agent's context window is also its attack surface.¹ That means issues, comments, support tickets, transcripts, files, and customer messages all need different handling than normal application text.

A traditional app can display hostile text without obeying it. An agentic app may summarize it, route it, fetch related data, or decide what to do next. The difference is action.

That changes the product checklist. You can't only ask, "Can the model answer correctly?" You have to ask, "What can the model touch after reading this?"

For B2B SaaS products, this matters even more. Your customers already have tenants, roles, paid modules, audit logs, and sensitive workflows. If you add an AI layer that floats above those rules, you've created a second product with weaker boundaries.

Why Aren't Prompts Enough? #

Prompts aren't enough because the model is designed to follow language, including language supplied by an attacker. IBM's 2025 breach research says ungoverned AI systems are more likely to be breached and more costly when they are.³ A policy written only in words is weak governance.

This doesn't mean prompts are useless. System instructions, refusal rules, and content classifiers all help. But they have to sit behind enforceable controls.

A safer agent design has four boring properties:

• The agent only receives the user-controlled content it needs for the current task.
• The agent's tools are scoped to the smallest useful permission set.
• The app separates instructions from data before either reaches the model.
• Every tool call is attributed, logged, and limited by the user's existing access.

That is less magical than a demo. It's also the difference between an assistant and a confused service account with a writing style.

How Should Product Teams Scope Agent Permissions? #

Product teams should scope agent permissions by task, not by employee convenience. Postman surveyed more than 5,700 developers and found that 82% of organizations have adopted some API-first approach, but only 24% design APIs for AI agents.² Most products have APIs. Far fewer have agent-safe APIs.

Start with the tool contract. If the task is triaging a support ticket, the agent shouldn't have broad customer-data export access. If the task is drafting a reply, it shouldn't be able to mutate production records. If the task is finding related docs, it shouldn't be able to post publicly without a review gate.

[PERSONAL EXPERIENCE] In one production deployment we worked on, the embedded builder inherited the host product's authentication and permissions instead of creating a separate security model. That mattered more than generation speed. It let teams build useful apps without letting those apps escape the product boundary.

A useful rule: every agent tool should have a boring human equivalent. "Read this user's allowed work orders" is a product capability. "Browse every repository in the organization and decide what to paste into a public comment" is not.

What Does a Safer Architecture Look Like? #

A safer architecture treats the agent as a product surface with explicit inputs, tools, and output channels. Postman reports that 93% of teams struggle with API collaboration, and agents make that coordination problem sharper.² If humans can't understand your API boundaries, agents won't magically respect them.

The pattern is straightforward:

1. Classify inputs. Separate trusted system instructions, trusted internal context, and untrusted customer or public content.
2. Constrain tools. Give the agent narrow operations, not broad database or repository access.
3. Attribute requests. Mark agent traffic so logs can answer who asked, what ran, and why.
4. Gate outputs. Treat public posting, external messaging, and data export as high-risk actions.

This is the same reason embedded AI app builders need to inherit the host product's security model. The builder may generate the UI, but the host product still owns identity, tenant isolation, allowed actions, and audit history.

If that sounds like product plumbing, good. Agent security is product plumbing now.

FAQ #

Conclusion #

GitLost is useful because it makes the hidden problem visible. The issue wasn't only a clever prompt. It was an agent with enough reach to turn a public request into private data exposure.

The next generation of SaaS AI won't be judged by how fluent the chat box sounds. It'll be judged by whether the product can safely answer a harder question: what can this agent do after it reads something it shouldn't trust?

Sources #

Footnotes #

1. Noma Security. "GitLost: How We Tricked GitHub's AI Agent into Leaking Private Repos." https://noma.security/blog/gitlost-how-we-tricked-githubs-ai-agent-into-leaking-private-repos/ 2026. ↩ ↩² ↩³

2. Postman. "2025 State of the API Report." https://www.postman.com/state-of-api/2025/ 2025. ↩ ↩² ↩³

3. IBM. "Cost of a Data Breach Report 2025." https://www.ibm.com/reports/data-breach 2025. ↩