Calibrated Confidence: How MemMesh Scores Predictions Against Reality

Any model can emit a number between 0 and 1 and call it confidence. The hard part — the part that makes a forecast usable in a real decision — is making that number mean what it says. This is calibration, and it's the architectural heart of MemMesh's prediction layer.

What calibration actually means

A predictor is calibrated if, across all the times it says 80%, the event happens about 80% of the time. That's it — but it's a strong constraint. An uncalibrated model can be confidently wrong in a way that quietly poisons every downstream decision, because the consumer trusts the number. Calibration is what lets you act on a 0.82 differently than a 0.55.

An uncalibrated 90% is a lie with a decimal point.

The reconcile loop

MemMesh calibrates by closing the loop against reality. It makes a prediction, waits for the outcome, and reconciles the two — measuring how well the confidence tracked what happened, and adjusting. Because the memory layer already records outcomes as first-class items, this loop is native: every resolved prediction is a data point that sharpens the next one.

// The loop, end to end.
const p = await mm.lattice.predict({ subjectId, target: 'renewal_due' })
// ...later, when the truth is known:
await mm.learning.recordOutcome({ predictionId: p.id, actual: '2026-07-03' })
// → calibration updates; the next forecast is a little sharper

Abstention: knowing when not to guess

Calibration's quiet partner is abstention. When a subject's history is too thin to support a real forecast, the honest move is to decline — return low confidence or no prediction at all, rather than manufacture certainty from nothing. A system that always answers is a system you can't trust when it matters.

Provenance for predictions, too

Just as recalled facts carry a source, forecasts carry the signals they were built from. That means a prediction isn't a black-box number — you can see which parts of the subject's history drove it, which is what makes it defensible to a user, a manager, or an auditor.

• Calibrated — the confidence tracks real outcome frequencies.
• Abstaining — thin signal yields a hedge or a pass, not a fabricated number.
• Provenanced — every forecast shows the signals behind it.

None of this is exotic — it's just the discipline of treating a confidence score as a promise you have to keep. That discipline is the difference between a forecast you demo and a forecast you deploy.