What Metadata Should Every Marketing RAG File Carry Before It Is Indexed?

Define the ownership, validity, territory, permissions, provenance and review metadata a marketing document needs before entering a private RAG index.

Short answer: Before indexing, every marketing RAG file should identify its stable source, owner, approval state, valid dates, audience, territory, product or campaign scope, confidentiality, permissions, supersession relationship and review trigger. The retrieval layer must preserve and enforce those fields. Metadata that exists only in a spreadsheet—or cannot filter retrieval—is documentation, not a working control.

Private retrieval systems often fail less dramatically than a fabricated answer. They retrieve last quarter’s price, mix two markets, quote an unapproved draft or expose one team’s material to another. The text can be authentic and still be wrong for the question because its scope and status were lost at ingestion.

Chunking and embeddings do not solve that governance problem. The file needs identity and scope before it is split; each chunk needs inherited or resolvable metadata after it is split; the retrieval system must filter or rank using that metadata; and the answer must retain citations back to the approved source. CDM’s dictionary defines the minimum control plane for marketing knowledge.

What the official evidence establishes

The W3C PROV Ontology provides a formal vocabulary for entities, activities and agents and for relationships such as generation, derivation and attribution. A marketing team does not need to implement the entire ontology, but its concepts help distinguish a document, the activity that produced it and the person or system responsible.

NIST’s Generative AI Profile is a voluntary cross-sector risk-management resource. It supports lifecycle documentation and evaluation; it does not prescribe one universal RAG metadata schema. CDM’s field set is an operational synthesis for marketing use, not a NIST requirement.

Vector stores and retrieval frameworks vary in field types, filter semantics and access controls. A field called “confidential” has no protective effect unless the identity and retrieval layers enforce it. The implementation test must therefore follow one document from system of record through chunks, index, retrieval trace and cited answer.

The CDM Retrieval Scope Metadata Dictionary

Use controlled values where possible and reject ingestion when required fields are absent. Record source-level fields once, then verify inheritance at chunk level.

How to run the Retrieval Scope Metadata Dictionary

1. Classify the decisions the corpus supports

List the questions the RAG is allowed to answer and the actions its outputs may influence. Customer-facing claims, internal planning, regulated communications and creative inspiration have different approval and evidence needs. Exclude unsupported purposes at the system boundary rather than relying on a prompt warning.

2. Define controlled vocabularies

Create accepted values for status, territory, audience, product, campaign, classification and owner. Avoid free-text synonyms such as “live”, “final-ish” and “current”. Document whether multiple values are allowed and how a query with missing context should behave.

3. Gate ingestion

Require source ID, version, owner, approval status, validity, classification and citation URI before indexing. Validate dates and references. Quarantine drafts, orphaned documents and files whose permissions cannot be mapped. A human should resolve exceptions rather than silently assigning a permissive default.

4. Prove chunk inheritance

Inspect indexed chunks from headings, tables, footnotes and attachments. Confirm each can resolve the source version, scope and permissions. Keep section or page locators so the answer citation reaches the evidence, not merely a file download. Test whether re-chunking changes metadata coverage.

5. Enforce at retrieval and answer time

Authenticate the requester, derive permitted scope, apply server-side filters and log candidates before generation. Require the answer to identify its supporting source and validity. If required context such as territory is absent, ask for it or return a bounded answer instead of retrieving across every market.

6. Test lifecycle events

Approve a new version, expire an old one, revoke a group, withdraw a claim and change an owner. Verify index update, cache behaviour, retrieval results and answer citations after each event. The metadata model is useful only if those operational changes propagate measurably.

Decision rules

The dictionary is a release gate. Defaults should fail closed for permission and approval, and fail visibly for missing business scope.

Build the evidence record before launch

Treat the schema as a data contract between knowledge owners and retrieval engineers. Store both the definition and proof that the running system enforces it.

• Field name, type, controlled values, definition, owner and whether it is required.
• Source-to-chunk inheritance rule and treatment of tables, attachments and OCR.
• Identity-to-permission mapping and deny-by-default test cases.
• Query-context derivation for territory, audience, product and time.
• Ingestion rejects, quarantined files and approved exceptions.
• Retrieval traces showing eligible, filtered and selected sources.
• Version, supersession, withdrawal and owner-change tests.

Keep confidential values out of shared screenshots and reports. A reviewer can inspect hashes, synthetic examples or access-controlled evidence. The public article describes the control model; it should never expose a client’s corpus, tenant identifiers or permissions.

Separate fact, CDM inference and the operating decision

Worked example: a UK offer retrieved for a US sales answer

A fictional marketing team indexes launch briefs from several regions. A US seller asks for the current introductory discount. The system retrieves a recent UK brief because its wording is similar and no territory field was attached to the chunks. The answer cites a genuine document but presents the wrong market rule.

The team adds stable source IDs, territory, currency, product, approval status and validity dates, then maps the seller’s identity and query context to allowed scopes. The UK brief remains searchable by authorised UK users but is filtered from the US request. A global policy can be used only where its scope explicitly says global.

The correction is not “improve the prompt”. It is an enforceable metadata and identity contract, tested at candidate retrieval. The team also marks the old US brief as superseded and proves that no query returns it.

Limitations and review boundary

Metadata cannot make an inaccurate source true. Owners must still substantiate claims, approve content and correct errors. Filters can also be misconfigured, identity attributes can be stale and free-text values can fragment scope. Test behaviour at the running retrieval layer.

The minimum varies by industry, jurisdiction and system purpose. Legal, records-management, privacy and security teams may require retention, residency, legal-hold or special-category fields beyond this model. Do not copy the dictionary without mapping actual decisions and threats.

Related guides

• How Do You Map Query Fan-Out Without Creating Hundreds of Thin Pages?
• Can AI Agents Complete Your Funnel? How to Audit the Accessibility Tree, DOM and Visual Path

Frequently asked questions

It can propose values, but inferred scope should not become an access or approval control without verification. Territory, permission, status and ownership should come from authoritative systems or accountable reviewers.

Each retrieved chunk must carry or resolve the fields needed for filtering, citation and lifecycle control. Physical duplication is an implementation choice, but losing the relationship to source metadata is not acceptable.

Use an explicit value and review rule, not a blank that silently means forever. For volatile claims, set a conservative review or expiry date and require an owner to renew it.

No. Public structured data helps describe a page to external consumers. Private RAG metadata controls corpus identity, scope, permissions, versioning and retrieval. Some concepts overlap, but purposes and enforcement differ.

Create a tombstone or equivalent status, remove it from eligible retrieval, invalidate relevant caches, retain the audit relationship to replacements, and run fixed queries to prove it no longer grounds answers.

Method and publication status

This prepublication draft was researched from the primary materials linked at the claims they support and checked on 12 October 2026. It requires the named specialist review, live-product retesting where indicated, a real 16:9 editorial image and final URL verification before publication. It is operational analysis, not legal, security or compliance advice.