Presented by
Ed Reif · Capability Architect / Curriculum Developer
Video Explainer
Operator Experience · above the waterline
▶Begin Operator Challenge
✓Validate with SME
Engineering Proof · below the waterline
Five models. One loop.
ADDIE · Bloom · Kirkpatrick · SAM · Gagné
The engine behind this brief. Completion is not capability — we measure readiness.
Open the instrument panel→
Maritime Sensor Fusion · AI Threat Detection · Operator Readiness
Designing for the
Decision, not
the Interface.
A readiness curriculum for the operators who will stand a live data stream and decide, under pressure, what an uncertain contact really is. The software is the vehicle. The decision is the destination.
Format
20-min walkthrough + 10-min Q&A
Constraint
Limited docs · SME interviews · iterative
The Friction Point
One operator.
One ambiguous contact.
An operator sits at the new interface. A contact appears where there should be open water. The acoustic feed registers anomalous cavitation — but the waveform never resolves into a clean profile.
A biological false positive? Or a small uncrewed subsurface asset? The AI will not commit. It hands back a confidence score and waits for a human.
No slide deck teaches this moment. Rehearsal does.
Contact · Bearing 047
SOURCE Acoustic / passive sonar
SIGNAL Anomalous cavitation
PROFILE No clean waveform match
CANDIDATES Biological · UUV (small)
AI VERDICT UNABLE TO AUTO-CLASSIFY
AI Confidence72%
Below the threshold to act. Above the threshold to ignore. The decision falls to the operator.
Thesis · What we are actually building
Completion is not capability.
We don't train clicks. We engineer judgment — and we measure readiness.
The through-line for all five sections
01
Planning & Tracking
5 min
Replace the linear checklist with a continuous Readiness Development Lifecycle.
02
SME Engagement
5 min
Extract how experts think under pressure — not what they happen to know.
03
Design Strategy
5 min
A consequence loop that calibrates the operator against an expert baseline.
04
Sample Concept
5 min
The decision scenario and a live readiness dashboard.
05
Q & A
10 min
Open discussion, edge cases, and how this scales to the swarm.
01 · Project Planning & Tracking
A lifecycle, not a
checklist.
A standard request runs a linear track — analyze, design, develop, ship. That works for content. It breaks against a probabilistic AI system, where the right answer is a calibrated bet, not a fact.
So we replace ADDIE's straight line with a Readiness Development Lifecycle — a loop, engineered backward from the exact moments the system gets tested in the water.
Every stakeholder — engineers, leadership, end users — aligns under one outcome-driven frame. And instead of status reports, leadership watches a readiness dashboard: live visibility into how operators actually perform.
01
Discovery & Judgment Extraction
Map exactly how experts navigate the hard calls — before a single slide is authored.
02
Rapid Environment Generation
Extracted judgments feed straight into high-stress decision environments.
03
Active Operator Testing
Skip passive learning. Prototypes go directly into operator hands to validate performance.
04
Readiness Telemetry
Measure performance against the mission, not against course completion.
Findings feed back into discovery — continuously
01 · Planning & Tracking · Tracking that tells the truth
Status reports lie. Telemetry doesn't.
The risk
The illusion of progress
A green project tracker and a finished module say a course exists. Neither says an operator can hold the line under a swarm.
The shift
Engineered backward
We start at the moment of consequence and design back toward it, so every element traces to a real decision in the water.
The instrument
Readiness dashboard
Leadership sees decision accuracy, latency, and recovery — quantitative evidence in place of a status email.
The lifecycle moves at the speed of the AI it trains against. Treat this as content delivery and you build a gap between the classroom and the mission. We close that gap by refusing to ever open it.
02 · SME Engagement
Don't ask what they
know. Ask how they
decide.
Ask an expert what they know and you'll get technical trivia. The objective is harder and more valuable: capture exactly how they think when the pressure is real.
With limited documentation, the interviews are the source of truth. We pull from three groups, each answering a different question.
Source 01 · Engineers
Where the AI breaks
The failure modes. When does the model go uncertain, and why a 72 is not a 100.
Source 02 · Leadership
What success means
The mission outcomes. The decisions that define a good operator versus a dangerous one.
Source 03 · Operators
Where humans slip
The most frequent field errors. The noise that fools the novice and the anchor the expert trusts.
02 · SME Engagement · The Judgment Extraction Framework
Three gates turn an interview into a decision model.
We don't transcribe opinions. We refine raw observation into the structure of a choice — then build that structure into the simulation as deliberate friction.
01Trigger
Out of all the incoming data, the one signal that says something is wrong. Where attention should snap.
02Filter
In a flood of information, the noise the expert systematically discards to keep focus on what matters.
03Anchor
The single data point that drives the decision to act. Lose it, and the call is a guess.
These triggers and anchors map directly to the mission tasks. We engineer smart friction — the operator must manually filter the noise to reach the anchor, which is exactly how the expertise gets built. We ignore software features in favor of the correct escalation decision.
03 · Instructional Design Strategy
The bet, and the
consequence.
Back at the moment of pressure: the operator makes the call and clicks ESCALATE. The system drops them straight into a continuous consequence loop built from that specific choice.
In a probabilistic world, "correct / incorrect" is useless. So the feedback does something sharper — it isolates the exact signal that mattered, and names the noise that pulled the novice off the anchor.
The loop calibrates the operator's judgment against an established expert baseline, run after run, until the gap closes.
→
Calculate
The operator reads the contact and commits to a call under time pressure.
→
Consequence
The world responds to that exact bet — not to a generic right answer.
→
Isolate the signal
Feedback highlights the anchor that mattered and the noise that distracted.
→
Calibrate
Compare to the expert baseline. Re-enter. Tolerance for uncertainty rises.
Continuous · builds stress tolerance against AI uncertainty
03 · Design Strategy · The principle underneath the method
The interface is the vehicle.
The correct decision is the destination.
Novice
A guess
Reacts to whatever is loudest. Chases the noise. Freezes at 72%.
The loop
A bridge
Exposes the cognitive gaps between the two — then closes them, run by run.
Expert
A calculated bet
Filters fast, finds the anchor, acts inside the window with the uncertainty intact.
04 · Sample Output / Concept · Decision rehearsal — try it
The scenario, and the loop it triggers.
Live Contact · Make the Call
SIGNAL Anomalous cavitation
PROFILE Unresolved waveform
WINDOW 00:00:18 to act
AI Confidence72%
Make a call. The loop responds to your bet — not to a generic right answer.
Consequence Loop · How feedback works
Feedback never says "wrong." It does three things, every time:
A
Names the anchor
The data point that should have driven the call.
B
Flags the noise
What pulled attention away from it.
C
Plots you vs. baseline
Your latency and accuracy against the expert.
04 · Sample Output · The Readiness Dashboard
What leadership
actually watches.
Course completion creates a dangerous illusion of capability. Clicking next does not mean an operator can manage a swarm under pressure.
So the dashboard reports quantitative telemetry against an expert-derived gate — three measures, continuously.
Completion is not capability.
We measure readiness.
Operator Readiness · Telemetry● LIVE
Threat Detection Accuracy88%
Correct anchor identified▲ gate 80%
Decision Latency11.4s
Speed under time constraint↘ approaching gate
False-Alarm Management92%
Recovery from AI uncertainty▲ gate 85%
05 · Q & A · 10 minutes — and the questions I expect
Where this gets pressure-tested.
With no docs, how do you trust the SME signal?
Triangulate across all three sources. Where engineer failure-modes, leadership outcomes, and operator errors agree, that's your anchor. Disagreement is itself a finding.
How does this scale from one contact to a swarm?
The unit of training is the decision, not the screen. The same trigger–filter–anchor structure composes — we raise contact count and degrade comms as readiness gates clear.
What if the expert baseline is wrong?
It's versioned, not sacred. As field telemetry comes back, the baseline updates. The loop trains against the best current model of good judgment, and improves with it.
How fast can you stand up a first prototype?
Discovery and judgment extraction first, then a rough decision environment in operators' hands fast. Active testing beats a polished module nobody has stressed.
The final objective is the systematic engineering of operator judgment — precise decisions under real operational conditions. Thank you. Ready for your questions.