Intelligence Layer — Technical Reference

Audience: engineers. Non-technical version: overview.md. Back to the Marketing OS index.

Status: ⬜ proposed. Nothing named "intelligence layer" exists in the codebase. One instance of the pattern ships today — the compliance rulebook — and is the reference implementation to generalise from.

---

The pattern to generalise

lib/compliance-intelligence.ts + lib/feed-compliance-agent.ts already implement every property the layer needs:

That last pair is the non-negotiable part. The model proposes; the store adjudicates.

---

The problem it solves

Context assembly is currently duplicated per agent. blog-generator.ts alone does:

const [brand, compliance, competitorBrief, keywords] = await Promise.all([
  resolveBrandContext(tenantId),
  getTenantComplianceRules(tenantId),
  buildCompetitorBrief(tenantId),
  selectBlogKeywords(tenantId, {...}, queuedByUserId),
]);
// then hand-prune each to fit: 6000 / 1200 / 5000 char budgets

Every agent repeats a variant of this, with its own budgets and its own pruning. Nine agents means nine places a brand change can fail to take effect, and nine different answers to "what did the model actually see?"

---

Proposed shape

A single read interface, one write path, versioned.

// proposed — does not exist
interface IntelligenceQuery {
  tenantId: string;
  kinds: KnowledgeKind[];       // which stores to draw from
  subject: { title?, text?, url?, geoState?, category? };
  budget: number;               // total char budget, allocated across kinds
}

interface IntelligenceContext {
  records: KnowledgeRecord[];   // each with id, kind, content, rationale, source
  rendered: string;             // prompt-ready block
  version: string;              // for staleness + reproducibility
  truncated: KnowledgeKind[];   // what did NOT fit — never silent
}

KnowledgeRecord mirrors ComplianceIntelRule: an id that can be cited, content, the reasoning it was authored for, and a source. Every agent's output schema then carries cited_ids, validated against the returned record set the same way attachRuleCitations() does today.

Knowledge kinds

The 🟡 rows mostly need ids and a rationale field rather than new storage. That is the bulk of the migration.

---

Retrieval vs fine-tuning

Decide once, explicitly.

Recommendation: retrieval for facts and rules, permanently. Under RBI scrutiny an unciteable answer is not defensible, and the compliance agent's existing design already accepts this.

Fine-tuning stays open for style — house voice, script cadence — where there is nothing to cite. Even then, prefer few-shot from the layer first.

---

Design constraints, inherited from what works

1. Fail open, never fail closed. No layer → agents fall back to today's direct reads. A retrieval outage must not block the feed, matching every existing agent's behaviour.
2. Truncation is reported, never silent. truncated in the response. The current per-agent prunePromptContext calls drop context invisibly.
3. Deterministic sources outrank model output on conflict, as mergeComplianceFindings() already does.
4. Tenant scope through the owner filter. No new access path — this layer would otherwise become the one place cross-tenant leakage is possible.
5. No PII into prompts. When customer_segment lands, records must carry cohort attributes, never identifiers.
6. Versioned like FEED_ENRICHMENT_VERSION. A context version stamped on every output, so a stale result is detectable — the same staleness signal the blog compliance report uses (reviewedVersion vs draft.version).

---

Migration path

Incremental, each step useful alone:

1  Add id + rationale + source to brand guidelines        → citable brand
2  Wrap existing reads behind one query interface          → no behaviour change
3  Move blog-generator's Promise.all onto it               → one agent proven
4  Add `truncated` reporting                               → stop silent pruning
5  Add choice_event capture (queue/dismiss/publish)         → unblocks Learning
   (partially done for one action: compliance dismissals on blog
    drafts become learned exceptions on the tenant — see the
    compliance agent's technical reference. No event log yet.)
6  Add glossary_term                                        → unblocks Localization
7  Port remaining agents                                    → single context path

Steps 1–4 touch no agent behaviour and are independently shippable. Step 5 is the one that unlocks the highest-leverage proposed agent.

---

Open questions

• Retrieval strategy. Rule counts are small (≤40) and fit whole; keywords and choice events will not. Embeddings, or structured filters plus recency? Start structured — the corpus is not yet large enough to justify a vector store.
• Who curates. The layer's quality becomes the product's quality. Compliance rules have an admin-only editor; the rest have no ownership model.
• Write-back authority. Should the Learning agent write records directly, or propose them for human acceptance? Given every other agent stops short of autonomous action, propose is the consistent answer.
• Conflict resolution between kinds — brand voice says one thing, a compliance rule forbids it. Compliance already outranks brand in the blog master prompt; that precedence needs stating layer-wide.

---

Roadmap source

Deliverable ④ "Marketing OS — Architecture" (page 5); page 1's data layer (IIFL DATA → CRM, Outside world DATA, MANUAL); page 2's target-state list (data enrichment · personalization · intelligence agent · NBA). See ../../roadmap_ref_extracted.md.