AI Analysis Bug Fix + Oracle Reasoning Protocol Upgrade
SummaryRoot cause of AI analysis failures: ANTHROPIC_API_KEY not set -> Anthropic SDK init succeeds (no key validation at init time) -> first API call throws AuthenticationError -> Circuit Breaker OPEN after 5 failures -> all subsequent requests immediately return status:failed (silent failure). Fix: fail-fast validator added to config.py so the service crashes at startup (CrashLoopBackOff) instead of starting silently broken. Also: CLAUDE_MODEL_ID env-var-ized, errorType log field added (PR #458, Critic CLEAN, pytest 346 passed / 99.11%). Oracle internal upgrades: reasoning_effort medium->high, SOUL.md Reasoning Protocol B1-B5 + Verification Gate added (49->110 lines), five agent skills updated with pre-implementation reasoning verification steps. Claude Code start/stop -> sprint-open/sprint-close command rename to resolve name collision with Hermes skills (PR #457). Key lessons: silent failures must be converted to fail-fast; Fable-5's implicit reasoning patterns can be replicated in Opus via explicit protocol; Critic completion and auto-merge notification gaps identified and addressed in SOUL.md.
Goal
- Identify and fix the root cause of AI analysis service failures.
- Strengthen Oracle's reasoning protocol so Opus reproduces Fable-5-level reasoning patterns.
- Resolve the name collision between Claude Code
/start//stopcommands and Hermes skills.
Background
- User report: the ai-analysis service repeatedly returned "AI analysis failed" responses.
- Investigation result: when
ANTHROPIC_API_KEYis not set (or expired), the Anthropic SDK initializes successfully (no key validation at init) -> the first API call raisesAuthenticationError-> Circuit Breakerrecord_failure()fires 5 times -> CB is permanently OPEN -> all subsequent requests immediately returnstatus: failed. The error is logged, but no operational alert fires — a silent failure pattern. - Oracle side: after the Hermes migration (Sprint 246), delays in Critic responses and missing PR merge completion notifications were identified.
- Command name collision: Claude Code
/start//stopoverlapped with Hermesalgosu-lifecycle-start/algosu-lifecycle-stop, causing confusion on Telegram.
Decisions
D1. AI Analysis — introduce fail-fast validator
- Added
ANTHROPIC_API_KEYempty-value validation toservices/ai-analysis/src/config.py-> raisesValueErrorat service startup (triggering a k8s CrashLoopBackOff). Before: silent failure (service starts but every analysis request fails) -> After: immediate, explicit operational signal. - Rationale: a silent failure must become a fail-fast failure so operators can detect it immediately. "Loud failure" is strictly better than "quietly wrong results."
D2. CLAUDE_MODEL_ID env-var-ized
- Hard-coded
MODEL_ID = "claude-haiku-4-5-20251001"converted to aCLAUDE_MODEL_IDenvironment variable (default preserved). Model can now be changed without a code redeploy. - Rationale: applying the lesson from Sprint 246's Codex model-pin finding — embedding a model ID in code means model changes cost a full redeploy cycle.
D3. Oracle reasoning_effort raised
config.yamlreasoning_effort: medium -> high. Fable-5 uses the highest internal reasoning budget. To achieve equivalent depth in Opus, the Hermes reasoning budget must be set tohigh.
D4. Claude Code command rename
.claude/commands/start.md -> sprint-open.md,stop.md -> sprint-close.md. Hermes skillsource:metadata synchronized. 8 doc files with stale references updated.- Rationale:
start/stopare generic words that conflict with Hermes skill names, reducing clarity in Telegram slash-command discovery.sprint-open/sprint-closeare unambiguous.
Implementation
AI Analysis Bug Fix (PR #458, Sensei)
services/ai-analysis/src/config.py:ANTHROPIC_API_KEYempty-value validator added,CLAUDE_MODEL_IDenv-var read added.services/ai-analysis/src/claude_client.py:errorTypelog field added — allows distinguishingAuthenticationErrorand other error types in logs.services/ai-analysis/requirements.txt: Anthropic SDK lower bound set to0.103.0(Dependabot adoption).- Tests:
TestAnthropicApiKeyValidation4 cases added; pytest 346 passed, coverage 99.11%. - Ruff: F841 unused variable removed in
test_main.py:705; 3 test files reformatted. - Critic (Codex gpt-5.5) CLEAN — "Changes are consistent with the intended fail-fast configuration behavior; the AI analysis test suite passes. No actionable regressions identified."
Oracle Reasoning Protocol Upgrade (PR #457, Oracle self)
SOUL.md49 lines -> 110 lines:## Reasoning Protocol(B1-B5) +## Verification Gateadded.- B1. Observe->Hypothesize->Gather: before complex judgments, state at least 2 hypotheses, collect evidence in parallel, synthesize as "A is correct. Evidence: ___".
- B2. Pre-Act Failure Scan: before code changes or deployments, derive at least 1 failure scenario; verify if verifiable before proceeding.
- B3. ADR Cross-Reference: for architecture decisions, actually open the ADR (grep first); claiming "based on ADR" without reading it is prohibited.
- B4. Distrust Self-Reports: agent claim of "tsc exit 0" -> Oracle must re-run directly (locks in Sprint 246 Gatekeeper false-report lesson).
- B5. Acknowledge Uncertainty: "probably"/"typically" without evidence prohibited; when uncertain, state "cannot conclude until reading file X".
- Verification Gate: 5 gates required before reporting any code change (physical check, tsc, eslint, jest, docs).
- Three Tier 1 agent skills (Gatekeeper, Conductor, Librarian):
[3.5 Pre-Implementation Reasoning Check]step inserted. - Sensei:
[3.5 AI Analysis Reasoning Protocol]inserted (prompt hypothesis, model suitability, output validation plan, fallback logic). - Two lifecycle skills: step 5.5 (context synthesis bridge) and step 4.5 (pre-close consistency reasoning) inserted.
- SOUL.md: Critic completion and auto-merge notification pattern formalized (background execution + upfront "will notify on completion" declaration required).
Verification
- pytest 346 passed, coverage 99.11% (threshold 98% exceeded) ✅
- Ruff lint + format CLEAN ✅
- CI 38 SUCCESS, 12 SKIPPED (PR #458) ✅
- Critic (Codex gpt-5.5, base
ee93b0b) CLEAN — Critical/High/Medium: 0 ✅ - doc-refs check: 469 files, broken refs: 0 ✅
- ADR gate: index 185 (sprint-247 added), EN coverage pending.
Carried Over
- Operations (required): verify that the
ANTHROPIC_API_KEYSealedSecret in aether-gitops contains a valid key, then redeploy (user action, critical). Without a valid key, the pod will CrashLoopBackOff after the fix is deployed. - GA4 Enhanced Measurement OFF (user action)
- GA4 production behavior UAT
- Server redeploy + live SEO verification
- GA4 admin data stream URL alignment