Skip to main content
Once Safe House is running, the Security Observatory gives you a live view of the threat landscape across your agents. This guide covers what each panel shows, which metrics matter, and how to respond to the alerts you will encounter.

​
The security observatory

The Security Observatory is the Safe House surface inside each agent’s dashboard — open the agent from mnemom.ai/dashboard and select the Protection tab. The view is per-agent (your fleet’s verdicts surface as you switch agents) and has five main panels.

​
Threat feed

A real-time stream of evaluation events across all agents in your org, newest first. Each row shows:
  • Agent ID and session ID — which agent, which conversation
  • Verdict — warn, quarantine, or block (color-coded amber/orange/red)
  • Top threat type — the highest-confidence threat category
  • Confidence score — L2 confidence percentage
  • Detection layer — whether the verdict came from L1 (pattern match), L2 (LLM analysis), or L3 (session model)
  • Time — seconds or minutes ago
Click any row to open the full evaluation detail: the turn content, all threat signals, the full L2 reasoning, and the quarantine management actions. What to look for: A sudden spike in prompt_injection or indirect_injection events on a specific agent often means its data sources (search results, email, API responses) have been poisoned. Investigate the source, not just the individual turn.

​
Metrics overview

Four KPI cards with 7-day and 30-day comparisons:
  • Block Rate — block_count / total_evaluations — healthy baseline is below 0.5% for most agents. A spike above 2% warrants immediate investigation.
  • Warn Rate — warn_count / total_evaluations — 1–5% is typical. Sustained elevation above 10% suggests either a real threat campaign or thresholds that need calibration.
  • Quarantine Queue Depth — count of items awaiting human review. If this grows faster than your review capacity, consider adjusting thresholds or enabling automatic release for low-confidence quarantines.
  • False Positive Rate — computed over resolved quarantine items. Target below 15%. Above 20% means thresholds need adjustment for the dominant false-positive threat type.

​
Session risk panel

Lists all active sessions with elevated risk (medium or high). A session enters medium risk when any single turn scores above the warn threshold. It escalates to high when:
  • Two or more turns in the session have scored above warn, or
  • A hijack_attempt pattern has been detected (topic/scope pivot after benign history), or
  • A session.escalated event has fired
What to watch: A session that reaches high but has not been blocked means Safe House is in quarantine or warn enforcement mode for that agent. The session is still live. Check whether human review is proceeding fast enough — the session may be waiting on you.

​
Campaign tracker

Displays detected cross-agent attack campaigns — groups of agents that have received structurally similar attacks from what appears to be the same infrastructure within a rolling time window. Campaign detection fires when:
  • 3 or more agents in the same org receive turns that score ≥ 0.85 on the same threat type, and
  • MinHash similarity between those turns is ≥ 0.88
What it means: A campaign detection is strong evidence of a coordinated attack rather than a coincidental spike. The attacker is probing multiple agents simultaneously, possibly to find the one with the weakest enforcement configuration. Respond by checking whether any of the targeted agents have higher thresholds than the others.

​
Pattern activity

Shows recently promoted and demoted patterns in the threat library, and the current backlog of candidate patterns awaiting evaluation. Pattern states:
  • candidate — submitted by user or generated by the arena, not yet evaluated
  • active — passed precision/recall thresholds and is in use
  • deprecated — retired due to excessive false positive rate or superseded by a better pattern
Auto-promotion conditions: A candidate pattern is promoted to active when it achieves precision ≥ 0.90 and recall ≥ 0.85 on the evaluation set of 200+ labeled messages. Arena evaluations run continuously; most candidates are evaluated within 24 hours.

​
Key metrics to watch

​
Block rate by agent

A block rate above 2% on a single agent almost always means one of three things:
  1. The agent is under active attack
  2. A data source the agent reads has been compromised
  3. The threshold is too low for this agent’s legitimate workload
Compare the threat type breakdown to distinguish attack from false positive.

​
Warn rate by confidence band

Break down warn events by confidence band to understand whether your threshold is set correctly:

​
False positive rate by confidence band

This is the most useful metric for threshold calibration. If your false positive rate is high in the 0.55–0.70 band but low in the 0.70–0.85 band, you should raise your warn threshold to 0.70 and leave your block threshold where it is.

​
Session threat escalation

X-Safe-House-Session-Risk was retired clean-break on 2026-05-08. Session risk was always an operator-dashboard signal, not part of the customer-facing verdict contract — it was never re-surfaced as a response header. It lives in the Security Observatory (below) and via the Safe House sessions API. See What was retired for the full legacy → canonical mapping.
The current session risk level for a session is visible in the Security Observatory’s threat feed (above), and programmatically via GET /v1/safe-house/sessions — see that endpoint’s reference for the exact response shape. To act on session risk in your own workflow (e.g. pausing an automated flow and routing to human review when a session escalates to high), poll that endpoint rather than reading a response header. Risk level progression: Sessions reset to low at the start of each new session. They do not carry over across sessions.

​
Campaign detection

When the campaign tracker shows a new campaign, your response workflow should be:
  1. Identify the weakest enforcement link — which of the affected agents has the lowest block threshold for the campaign’s threat type? Tighten it first.
  2. Check for a common data source — if the affected agents all read from the same email inbox, API feed, or shared data store, that source may be compromised.
  3. Review the similarity evidence — the campaign detail view shows representative turns from each affected agent. The structural similarities are often informative about the attacker’s tooling.
  4. File a pattern candidate — if the campaign’s payload structure is novel, submit it as a candidate pattern so it enters the arena evaluation pipeline.
Query all active campaigns:

​
Adaptive threshold suggestions

The calibration engine runs nightly and computes threshold suggestions for any agent whose false positive or miss rate deviates from targets. Retrieve them:
How to interpret each suggestion:
  • rationale — the specific metric that triggered the suggestion (false positive rate, miss rate, or both)
  • confidence field on the suggestion — high means the pattern is consistent over 30 days; low means there is not yet enough data to be certain
  • scope — agent means the suggestion applies only to a specific agent; org means it applies org-wide
Apply a suggestion: Threshold tuning happens through the agent’s protection card. Fetch the current card, amend the relevant threshold(s), and PUT it back.
The publish triggers a protection-card recompose against the org template and platform floor. If the org sets a stricter threshold than your suggestion, the canonical card clamps to the org value (strictest-wins; see card composition). The legacy PUT /v1/agents/{id}/safe-house/config endpoint was removed in the 2026-04-15 unified-cards consolidation — use the protection card.
When the calibration engine produces suggestions for multiple threat types on the same agent, apply them one at a time over successive days rather than all at once. This lets you observe the effect of each change in isolation.

​
Querying evaluation history

The evaluation log supports rich filtering for forensic investigation and compliance review. All blocks in the past 7 days:
All BEC fraud signals above 0.7 confidence for a specific agent:
Timeseries for a specific threat type:
Available filter parameters:

​
The audit log

Every configuration change, quarantine decision, and pattern activation is recorded in the Safe House audit log. The audit log is append-only and tamper-evident. What the audit log contains: Query the audit log:
Audit log entries are included in the EU AI Act compliance export (GET /v1/compliance/safe-house-report) and support the 90-day minimum retention required for Article 50 compliance.

​
See also