Traditional assessment instruments collapse multi-dimensional human competence into a single static score. We formalize capability estimation as an online Bayesian filtering problem over non-stationary evidence streams. By representing capability belief states as Beta distributions B(α, β) coupled with an entropy-regularized pseudo-count discount factor, we avoid the catastrophic score lock-in observed in conventional moving averages while retaining bounded variance under small sample regimes.
How should we represent capability when our evidence is incomplete?
When assessing a human practitioner, evidence arrives as a non-stationary sequence: resume records, project deliverables, proctored code evaluations, peer feedback, and long idle intervals. Single scores discard epistemic uncertainty. Instead, our belief state maintains a continuous probability density over latent mastery.
Updating the Belief
Test how incoming observable evidence alters the latent capability parameter and narrows the credible interval in real time.
Research insight: Every additional observation updates the posterior belief without asserting unjustified certainty. As α and β grow, the variance shrinks, narrowing the [45%, 91%] interval.
Where the system fails
Documented Failure Modes
Genuine research maturity requires documenting failure modes with the same empirical scrutiny as benchmark successes:
"I managed several teams across production cycles ensuring daily targets were hit."
Diagnosis: The token classifier interprets 'managed' as executive administrative leadership rather than grounding the verb in factory shift operational constraints.
Mitigation: Conditioning entity weights on domain co-occurrence embeddings (e.g. 'production cycles' attenuates corporate leadership classification).
"NC(V) in Electrical Infrastructure Construction with mechanical fitting apprentice modules."
Diagnosis: Vocational programs integrate cross-disciplinary foundational modules. The classifier over-indexed on secondary mechanical tokens, downgrading the primary trade accreditation.
Mitigation: Enforcing strict National Qualification Framework (NQF) regulatory gazette hierarchies during graph projection.
"Candidate with 1 single observation: Score = 100% (1/1 items correct)."
Diagnosis: Arithmetic averages or maximum likelihood estimators report absolute certainty on a single correct response. Our Bayesian formulation maintains broad credible intervals rather than premature confidence.
Mitigation: Informative prior initialization with Minimum Information-Gain regularization.
@techreport{mahandana2026probabilistic,
author = {Mahandana, Daniel and M., Narvin and K., Thabang},
title = {Probabilistic Modelling of Latent Human Capability from Sparse Observation Streams},
institution = {The Sota Lab},
year = {2026},
number = {SOTA-TR-2026-01},
url = {https://sota-lab.lovable.app/research/capability-inference}
}