← all meta proposalsTighten fast_feedback_loops rubric to penalize multi-causal outcome attribution
council rejected PROMPT reversible: simple 2h proposed 19 May 2026
What is the proposed change?
In filter_score.js, extend the existing `fast_feedback_loops` axis rubric in both the highSystem and lowSystem adversarial prompt strings. The current rubric asks 'Can the system learn from real outcomes within minutes/hours, not weeks?' — append the following disambiguation block: 'Score 2-3 ONLY when the resolution mechanism grades the output against external, objective ground truth that exists independently of the buyer's perception: a citation verifiable against an external database, a timestamp in a public system of record (Jira, GitHub, Zendesk), a classification against a known outcome (ATS hire/no-hire, regulatory ruling, market event). Score 0-1 when the primary feedback signal is: CSAT or NPS score, stakeholder satisfaction rating, managerial judgment, or any outcome where the AI output is one of multiple causal inputs (e.g., coaching + prompt change + agent output all contributed to the outcome — attribution is impossible). Anti-pattern to catch: when the hypothesis describes learning from whether "support quality improved" or "customers are happier," score 0-1. When it describes learning from whether a specific promise timestamp matched a Zendesk ticket resolution event, score 2-3.' Add this block to both highSystem (which argues high) and lowSystem (which argues low) in identical form so neither model gains an asymmetric anchor.
Target files
hypothesis_engine/moves/filter_score.js
Expected effect
hyp-2026-05-06-847f7e (Support Promise Calibration Console — killed because 'CSAT/SLA outcomes are multi-causal') scores 0-1 on fast_feedback_loops under the revised rubric. hyp-2026-05-13-47730e (AI Portfolio Claim Auditor — killed because 'board verdicts multi-causal') scores 0-1. hyp-2026-05-06-ec4507 (Support Escalation — Zendesk timestamp as resolution mechanism) scores 2-3. Net effect: the NBJ Q4 describability test's 'external objective ground truth' criterion is enforced at filter_score rather than waiting for council to identify the flaw.
Falsifier — what would prove this wrong?
Apply revised fast_feedback_loops scoring retroactively to 43 S157 candidates. Required: the 4 FRAGILE candidates (hyp-47730e, hyp-dc26ff, hyp-a81ff4, hyp-a38d31) all score 0-1 on this axis. Required: at least 3 of the 5 ROBUST 5/5 candidates score 2-3. If average score gap between FRAGILE and ROBUST candidates on this axis is less than 1.0 point, the rubric is not discriminating on resolution objectivity.
Evidence that triggered the proposal
- S157_NBJ_DESCRIBABILITY_TEST.md STRUCTURALLY FRAGILE classification for hyp-847f7e: 'Q4 ❌ CSAT/SLA outcomes are multi-causal — exactly the engine's noisy outcome attribution'; and FRAGILE classification for hyp-47730e: 'board verdicts multi-causal'
- S157_NBJ_DESCRIBABILITY_TEST.md Q4 definition: 'Buyer can state for this specific input, correct output is X and not Y, here's why — without invoking model confidence. Passes when grading is against external objective ground truth (citations vs Bailii, classification vs CSAT, milestones vs Jira timestamps). Fails when the only check is the council scores it.'
- S157_NBJ_DESCRIBABILITY_TEST.md pattern observation: 'Pattern that beats NBJ's test: outputs grade against external objective ground truth (Bailii citations, ATS scorecards, market events, Zendesk timestamps, public changelogs, Jira+git). They fail when ground truth is CSAT or other multi-causal.'
Proposer self-score
The proposer scored its own draft on these axes (0-3 each) before submitting.
| Axis | Score |
|---|
| specificity | 2 |
| falsifier | 3 |
| solo feasible | 3 |
| blast radius | 3 |
| composability | 3 |
| reversibility | 3 |
Disposition
Rejected at the council verdict. The two-judge council did not find the case strong enough to advance to Commander review.
Evaluation history
| When | Move |
|---|
| 2026-05-23 04:29 | meta_council_verdict |
| 2026-05-23 04:15 | meta_argument |
| 2026-05-19 14:24 | filter_score |
| 2026-05-19 14:18 | evidence_search |
| 2026-05-19 12:06 | audience_simulation |
| 2026-05-19 10:24 | red_team_kill |
| 2026-05-19 09:54 | meta_filter_score |
| 2026-05-19 09:54 | steelman |
| 2026-05-19 09:38 | meta_genesis |