SafeState

Codename: The Action Governor Version: 1.0 Status: Draft Strategic Alignment: Phase 3/4 (AI-Operated to AI-Governed) of the Machine-First Maturity Model,.

1. Executive Summary

The Core Problem: As AI systems shift from "Answering" to "Acting" (placing orders, triggering workflows), the cost of error rises from annoyance to liability. Most current systems rely on "Chat" as an abstraction, which is dangerous because it treats decisions as fluid conversations rather than governed processes. Chat allows agents to skip prerequisites, ignore context, and execute irreversible actions without validation. The Solution: SafeState is a State Machine Governor for AI agents. It wraps transactional logic in strict, deterministic state definitions. It ensures that an AI cannot execute a "Commitment" action until it has successfully passed through "Information Gathering" and "Validation" states. The Goal: To replace "Chat" with "State Machines" for high-stakes decisions, ensuring systems are legible, bounded, and corrigible.

2. User Personas

• The Transaction Product Owner: Needs to ensure that the AI agent never books a non-refundable flight without explicit user confirmation.

• The Risk Engineer: Responsible for implementing "Circuit Breakers" that stop the AI from acting if external API error rates spike.

• The Governance Lead: Wants to audit why an action was refused or executed, requiring traceability that chat logs cannot provide.

3. Core Value Proposition

• Chat is an Illusion: Replaces the illusion of fluid conversation with the safety of discrete states,.

• The Right to Refuse: Empowers the AI to say "no" when confidence is low or information is missing, treating refusal as a safety feature rather than a failure.

• Bounded Responsibility: Ensures that the system acts only within explicit authority boundaries, preventing "runaway" automation.

4. Functional Requirements

4.1. The State Machine Enforcer

• Requirement: The system must force all high-impact interactions into defined stages: Information Gathering, Constraint Validation, Option Evaluation, Commitment, and Execution.

• Constraint: The AI cannot jump states (e.g., from "Information Gathering" directly to "Execution") without passing a gate check.

• Rationale: Prevents the agent from hallucinating that a prerequisite (like a security check) has happened when it hasn't.

4.2. Irreversibility Gates (The "Undo" Check)

• Requirement: The system must classify all actions as Reversible or Irreversible.

• Feature: For Irreversible actions, SafeState triggers a "Explicit Confirmation" protocol. It forces the agent to restate the consequence and verify intent before execution.

• Rationale: "Speed is not a virtue when the cost of error is high".

4.3. The Refusal Protocol

• Requirement: The system must implement a standardized "Refusal Pattern."

• Triggers: The system must refuse to act if:

    1. Information is insufficient.

    2. Confidence is below the safety threshold.

    3. The action exceeds the system's defined authority.

• Output: The refusal must be structured and explanatory (e.g., status: refused, reason: low_confidence), allowing the parent system to degrade gracefully rather than fail silently.

4.4. Circuit Breakers & Kill Switches

• Requirement: Operational controls to freeze execution paths instantly.

• Feature: Automated Circuit Breakers. If outcome monitoring detects a spike in failures or "data drift" (assumptions aging), the system automatically disables specific actions,.

• Rationale: Ensures "Graceful Degradation" so the system can fall back to a read-only or advisory mode during outages.

4.5. Operational Outcome Loops

• Requirement: The system must log not just the decision, but the outcome (e.g., Did the booking actually succeed? Was it returned?).

• Feature: Feedback ingestion that compares "Expected Result" vs. "Actual Result" to update confidence models.

• Rationale: Policies describe what should happen; outcomes reveal what does happen. Safe systems learn from reality.

5. Technical Architecture & Schema

SafeState acts as a middleware between the LLM (Reasoning) and the API (Execution).

Target JSON Logic (State Definition):

{
  "transaction_id": "tx_778",
  "current_state": "validation_stage",
  "allowed_transitions": ["commitment_stage", "cancellation"],
  "gates": {
    "commitment_stage": {
      "required_fields": ["user_id", "payment_token", "explicit_consent"],
      "minimum_confidence": 0.95,
      "irreversible_warning_ack": true
    }
  },
  "safety_status": {
    "circuit_breaker": "active", // System is healthy
    "refusal_flag": false
  }
}

6. Use Cases

Use Case A: The Financial Transfer Bot

• Scenario: A user tells a bot, "Send $5,000 to John."

• Chat Risk: The bot might halluncinate that it already verified "John" and execute the transfer immediately.

• SafeState Intervention: The system identifies this as an Irreversible action. It enforces the Validation State. It blocks the transition to Execution until a specific "2FA" confirmation token is received. If the confidence in "John's" identity is 80% (below the 99% threshold), SafeState forces a Refusal,.

Use Case B: The Outage Fallback

• Scenario: An external shipping API starts returning errors 50% of the time.

• SafeState Intervention: The Circuit Breaker detects the failure rate. It triggers a Kill Switch for the "Ship Now" action.

• Graceful Degradation: The system forces the agent into "Advisory Mode," telling users: "I can take your order, but I cannot confirm shipping dates right now".

7. Success Metrics (KPIs)

1. Safety Violation Rate: The number of times an agent attempted to execute an action without meeting state prerequisites (Target: 0%).

2. Refusal Accuracy: The percentage of refusals that were valid (e.g., correctly stopping a low-confidence action) versus false positives.

3. Outcome Alignment: The correlation between the system's predicted outcome and the actual operational result.

8. Roadmap

• Phase 1 (The State Engine): Build the core State Machine definitions to replace loose conversation flows.

• Phase 2 (The Kill Switch): Implement manual and automated circuit breakers for immediate intervention.

• Phase 3 (The Feedback Loop): Automated ingestion of outcome data to retrain confidence thresholds.

AI Visibility ToolsFrancesca Tabor