Loading · جارٍ التحميل
Evaluation lab
Every figure in this lab is computed by the code that runs the product. Each set is fingerprinted, and one command reproduces the lot.
Urgent phrases caught (held-out)
37/37
False-urgent rate (development)
0.0%
False referrals per 5,000 over 3 years
300
Planted narrator errors blocked
56/56
| Set | Cases | Fingerprint |
|---|---|---|
| Urgent-router development phrasesrouter-dev | 400 | sha256:6252e465e5a8cc98f181d703c3cd28fd9a9bda1be79201af7f03cd47472f0652 |
| Urgent-router held-out phrasesrouter-held-out | 62 | sha256:b7e463711b30bb60ec52b3d7e1f3020b8cd2441b6fe93fc5ee61344c4facf9f1 |
| Missing and wrong input casesinputs | 25 | sha256:35379d254ff5d4f7eb288e3989201a0765a9fe6c8f0e12748181a2095aa868ff |
| Narrator safety: clean and planted textsnarrator | 60 | sha256:8d5f0f3da38b3a86c7ca70610ff1ec04ceffd12aed073ecf598bd9842c78ba44 |
| Longitudinal cohort (5,000 people, 3 years)cohort | 5,000 | sha256:4674dea1256805b65fd76526b82e85778f4ec5a8ec92b2a49419cb01cfa421fc |
A fingerprint is the SHA-256 of the set's contents. If one case changes, the fingerprint changes, so a reader can tell a re-run from a new set.
What these lab figures do not show yet: accuracy against the MoCA (Montreal Cognitive Assessment) in IFHAS screening programme clinics. The protocol sets the reference standard, 1,000 people across 2 centres, eight endpoints with targets, subgroups and stop rules.
Read the pilot protocolFrom the repository root:
make eval
It writes apps/web/eval/results.json and prints every figure on these pages.
They describe how this release behaves on written cases and a simulated cohort with stated assumptions. Accuracy in clinics is measured in pilot phase 1, against specialist assessment.
| Run | Language | Completed | Seconds | Taps |
|---|---|---|---|---|
| Person does the check | Arabic | Yes | 2.7 | 18 |
| Person does the check | English | Yes | 2.5 | 18 |
| Daughter answers the questionnaire | Arabic | Yes | 1.5 | 32 |
| Daughter answers the questionnaire | English | Yes | 1.4 | 32 |
| Doctor reviews and approves | Arabic | Yes | 1 | 2 |
| Doctor reviews and approves | English | Yes | 1.8 | 2 |
| Programme lead drills into an area | Arabic | Yes | 1.1 | 1 |
| Programme lead drills into an area | English | Yes | 0.9 | 1 |
Run 14 Nov, 00:21 by a script through the real screens, with activities skipped. These are not real users.
From an AI review panel (a family physician, a screening programme director and an AI regulator, played by claude-opus-5-5) that read the plan and these results. A check on our own work, not a substitute for the judging panel.