Field noteNov 12, 20266 min read1 source

How We Pen Tested LlamaParse Against the OWASP LLM Top 10

In one minute

LlamaIndex says it tested LlamaParse in production against all ten OWASP Top 10 for LLM Applications risks, alongside standard web-application risks. The four-day greybox exercise found issues the company says it fixed or mitigated, and exposed a practical boundary: a parser can faithfully extract hidden document content, but customers must decide which extracted values are trustworthy before acting on them.

What changed: LlamaIndex details penetration testing of LlamaParse against OWASP LLM risks, adding security-testing findings for its document-parsing tools.

What happened

In a November 11, 2026 engineering post, LlamaIndex describes LlamaParse as a service that turns PDFs, scans and other files into structured data for AI applications. The company says these documents can be sensitive, including financial filings, insurance claims and contracts. Documents may pass through parsing, classification, extraction and indexing, then be read by customer-deployed agents or AI assistants connected through LlamaParse's MCP server. LlamaIndex argues that this makes a whole file, often authored outside the customer’s organization, a more complex attack surface than a form field.

The company mapped the OWASP Top 10 for LLM Applications to possible document-platform risks. Examples included prompt injection in document text, cross-organization disclosure, poisoned index content, unsafe downstream handling of extracted values, excessive agent permissions and unbounded consumption. LlamaIndex says conventional API testing can often check whether an endpoint rejects unauthenticated requests, while model behavior can vary with both the request and the input. Its chosen tester, Trace, performs agentic dynamic application security testing: agents attack a running application, adapt attempts to responses and demonstrate findings with working exploits. Trace staff reviewed findings before delivery, according to the post.

LlamaIndex says it provided Trace with working credentials but no source code or cloud configuration access. The test ran against LlamaParse production, used accounts in separate organizations to test tenant boundaries, and covered the web application, sign-in, public API, parsing, classification, extraction, indexing, retrieval, deployed agents and MCP server. The company reports that the work took four days and covered all ten OWASP LLM risks plus standard OWASP web risks. To exercise end-to-end flows, Trace agents read documentation, created a GitHub repository, wrote Python code, generated sample PDFs, uploaded them and iterated on the code. LlamaIndex says the agents used frontier models through Anthropic’s Cyber Verification Program and OpenAI’s Daybreak program. The authorized test excluded denial-of-service and brute-force attacks and did not retrieve real customer records.

One example involved an invoice with extra lines printed in white on a white page. The visible invoice showed one total and bank account; extraction returned the hidden “corrected” values because the parser read the whole page. LlamaIndex says it fixed or mitigated every issue in the platform itself. It draws a distinction between its responsibility to keep customer data separate, control access and make parsing behave as documented, and the customer’s responsibility to choose which documents enter workflows and indexes, select a parsing tier, and validate results. For high-stakes actions such as payments, the company advises checking extracted values. It also recommends limiting who can add documents to shared indexes and using do_not_cache=true for the most sensitive files.

The post lists enterprise controls including SOC 2 Type II, HIPAA support, EU deployment, encryption at rest and in transit, and SAML or OIDC sign-in with role-based access control. LlamaIndex says uploaded files are cached for 48 hours before deletion, with deletion also available through the API or UI. Deployment options include managed SaaS in North America or the EU, single-tenant SaaS, customer-cloud deployment and self-hosting. The company says it does not train on customer data by default and that its managed-service LLM providers have zero-data-retention and no-training agreements with it.

What the archive adds

New Runtime’s earlier coverage of the draft Document Parsing Protocol described LlamaIndex’s effort to make parsing judgment calls more consistent across systems (Introducing the Document Parsing Protocol, October 30, 2026). This is a new security-testing event, not a restatement: the protocol concerns shared parsing conventions, while this post reports a production security exercise and the boundary between extraction and downstream trust decisions. Earlier New Runtime coverage also examined attacks on Codex’s synchronous controls (Red-teamers found 11 ways for a misaligned agent to slip past Codex's Auto-review, November 2, 2026), a related but distinct security problem.

Trend signal

This supports the view that agent security depends on boundaries around what systems can read and do, not only on instructions given to a model. LlamaIndex’s example shows why a document-processing pipeline needs controls over inputs, tenant access, indexes and downstream actions. The connection is relevant to agent security moves to runtime boundaries, though this case focuses on document pipelines rather than a general-purpose agent runtime.

Why now

LlamaIndex says documents flow through increasingly connected processing paths: extraction and indexing are followed by customer agents and assistants connected through an MCP server. It frames the test as a point-in-time check and notes that each new feature changes the attack surface. That makes the timing relevant to teams adding AI features to systems that already ingest untrusted files.

Why it matters

A parser’s accuracy and a business process’s correctness are not the same thing. If a system extracts every visible or hidden value faithfully, that does not establish which value a payment workflow should trust. Teams that feed supplier invoices, applications, claims or contracts into AI pipelines need to define validation rules where business context exists, rather than assuming the extraction layer can make that decision safely.

LlamaIndex’s account also shows why testing should follow complete workflows. A security review limited to a login screen or API endpoint might miss how a document influences extraction, retrieval, an agent or an MCP-connected tool. The practical lesson is to threat-model the entire path from upload to action, including who can add material to shared indexes and what an agent can do with its results. The reported exercise is vendor-authored, so it is a useful example of method and responsibility boundaries, not independent proof that the product is secure.

Limits and uncertainty

The account is LlamaIndex’s description of its own test. It does not disclose the number or severity of findings, the specific fixes, or a complete independent assessment of the remaining risk. The post says all platform issues were fixed or mitigated, but does not provide enough detail to evaluate those remediations. The hidden-text invoice is an illustrative finding, not a measured estimate of how often such attacks occur. A four-day test is also a point-in-time result; LlamaIndex acknowledges that later features create new attack surface.

Skill to build now

This week, take one document workflow that can trigger a consequential action, such as approving an invoice. Make a test PDF whose visible values conflict with values hidden in the file, then run it through the same parsing and extraction path used in production. Confirm which values emerge and whether the workflow blocks or routes the conflict for human review before payment. Also check who can add files to the relevant shared index. Treat this as a workflow check, not a substitute for a professional security assessment.

Reader takeaway

Treat third-party documents as untrusted input even when the parser is doing exactly what it was designed to do. Parse and extract broadly; enforce trust, access and approval rules where the business process can judge what should happen next.

Evidence / sources

  1. 1llamaindex.ai/blog/how-we-harden-application-testing-for-the-most-sensitive-documents
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicAnthropicExplore the anthropic topic hub.
  2. 02archiveField NotesOpen the latest editorial analysis.
  3. 03sourcesSourcesSee the original sources this archive cites.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract