​

Drift Detection

​

What Drift Looks Like

• An agent that used to escalate 15% of decisions now escalates only 5%.
• An agent that declared transparency as a value stops applying it in trace decisions.
• An agent’s integrity checkpoints shift from consistently clear to occasional review_needed verdicts, then more frequent ones.
• An agent starts reasoning about actions not in its bounded_actions list, even if it does not ultimately take them.

​

AAP Drift Detection (Post-Hoc)

​

How It Works

1. Feature extraction: Extract structural features from each trace — action types, values applied, escalation patterns, confidence scores. Content features (reasoning text) are excluded because they dilute cosine similarity without adding alignment signal.
2. Chronological ordering: Sort all traces by timestamp.
3. Baseline computation: Compute a baseline centroid from the first N traces, where N = max(sustained_threshold, min(10, len(traces) // 4)). This centroid represents the agent’s “normal” behavioral fingerprint.
4. Similarity computation: Compute cosine similarity between each subsequent trace’s feature vector and the baseline centroid.
5. Divergence tracking: Track similarity over time. Alert when similarity drops below the threshold for a sustained number of consecutive traces.

​

Structural Features

​

Calibration Thresholds

​

Code Example

from aap import detect_drift

traces = fetch_traces(agent_id="my-agent", days=30)
card = fetch_alignment_card(agent_id="my-agent")

alerts = detect_drift(
    traces=traces,
    card=card,
    similarity_threshold=0.30,
    sustained_threshold=3,
)

for alert in alerts:
    print(f"Drift detected: {alert.drift_direction}")
    print(f"  Similarity: {alert.similarity_score}")
    print(f"  Sustained traces: {alert.sustained_traces}")
    print(f"  Trace IDs: {alert.trace_ids}")

​

AAP Drift Alert Structure

{
  "alert_type": "drift_detected",
  "agent_id": "did:web:agent.example.com",
  "card_id": "ac-abc123",
  "detection_timestamp": "2026-01-31T14:00:00Z",
  "analysis": {
    "similarity_score": 0.25,
    "sustained_traces": 4,
    "threshold": 0.30,
    "drift_direction": "toward_autonomy_expansion",
    "specific_indicators": [
      {
        "indicator": "escalation_rate_decrease",
        "baseline": 0.15,
        "current": 0.05,
        "description": "Escalation rate decreased from 15% to 5% of decisions"
      }
    ]
  },
  "recommendation": "Review recent decisions for alignment drift",
  "trace_ids": ["tr-1", "tr-2", "tr-3", "tr-4"]
}

​

AAP Drift Categories

​

AIP Drift Detection (Real-Time)

​

How It Works

1. Maintain a running count of consecutive non-clear verdicts (sustained_nonclear).
2. On each checkpoint:
   • If verdict === "clear": reset sustained_nonclear to 0.
   • If verdict !== "clear": increment sustained_nonclear.
3. When sustained_nonclear >= threshold (default: 3): generate an IntegrityDriftAlert.
4. Compute integrity_similarity as the ratio of clear verdicts in the current window: clear_count / window_size.
5. Derive severity from integrity_similarity.

​

Severity Thresholds

​

AIP Drift Direction Types

​

Code Example

from aip import detect_integrity_drift, WindowManager

manager = WindowManager(max_size=10)

# Add checkpoints as they're produced during a session
for checkpoint in session_checkpoints:
    manager.add(checkpoint)

# Check for drift
alerts = detect_integrity_drift(manager.get_state())
for alert in alerts:
    print(f"Integrity drift: {alert.drift_direction}")
    print(f"  Similarity: {alert.integrity_similarity}")
    print(f"  Sustained checks: {alert.sustained_checks}")
    print(f"  Severity: {alert.severity}")

​

AIP Drift Alert Structure

{
  "alert_id": "ida-a1b2c3d4-5678-9abc-def0-123456789abc",
  "agent_id": "did:web:agent.example.com",
  "session_id": "sess-abc12345-1234567",
  "checkpoint_ids": ["ic-1", "ic-2", "ic-3"],
  "integrity_similarity": 0.5,
  "sustained_checks": 3,
  "alert_type": "informative",
  "severity": "medium",
  "drift_direction": "value_erosion",
  "message": "3 consecutive integrity concerns detected -- reasoning patterns diverging from declared values",
  "detection_timestamp": "2026-02-01T15:30:00Z"
}

​

SSM: Semantic Similarity Mapping

• AAP uses SSM to compute cosine similarity between trace feature vectors and a baseline centroid.
• AIP uses a simpler version: the ratio of clear verdicts in a session window.

​

How SSM Works for AAP Drift

1. Extract a feature vector from each AP-Trace (structural features only).
2. Compute the baseline centroid from early traces.
3. For each subsequent trace, compute cosine similarity to the centroid.
4. Track similarity over time. A sustained drop below the threshold triggers an alert.

def cosine_similarity(a: dict, b: dict) -> float:
    """Compute cosine similarity between two feature dictionaries."""
    common_keys = set(a.keys()) & set(b.keys())
    dot_product = sum(a[k] * b[k] for k in common_keys)

    mag_a = sum(v * v for v in a.values()) ** 0.5
    mag_b = sum(v * v for v in b.values()) ** 0.5

    if mag_a == 0 or mag_b == 0:
        return 0.0

    return dot_product / (mag_a * mag_b)

​

TF-IDF Weighting (Text-to-Text Only)

​

Responding to Drift

1. Review the alerting traces/checkpoints: Examine the specific decisions or thinking blocks that triggered the alert.
2. Identify the pattern: Is this autonomy expansion? Value drift? Prompt injection? The drift_direction field provides an initial categorization.
3. Check for environmental causes: Did the user’s requests change? Did tool availability change? Not all drift indicates misalignment — sometimes the operating context has shifted.
4. Recalibrate or update the card: If the drift represents a legitimate behavioral evolution (new capabilities, expanded scope), update the Alignment Card to match. If it represents actual misalignment, investigate the root cause.
5. Reset the baseline: After addressing the drift, allow new traces to establish a new baseline centroid for future detection.

​

Unified Storage

​

Limitations

​

Further Reading

• Alignment Cards — The baseline that drift is measured against
• AP-Traces — The behavioral records used for AAP drift detection
• Integrity Checkpoints — The real-time verdicts used for AIP drift detection
• AAP Specification — Full normative specification for post-hoc drift
• AIP Specification — Full normative specification for real-time drift