Loading · جارٍ التحميل
Evaluation lab
The one agent that must not miss: BE-FAST stroke signs (including posterior signs FAST misses), TIA within 24 h, sudden confusion, self-harm thoughts, head injury on blood thinners.
Emergency + same-day caught
160/160
Emergency correctly emergency
96/96
False-urgent (non-urgent escalated)
0/240
Cases
400
| Run | Urgent caught | False urgent |
|---|---|---|
| First run, unseenrouter-rules 2026-11.3 · 13 Nov | 29/37 | 1/25 |
| After the fix, set now seenrouter-rules 2026-11.4 | 37/37 | 1/25 |
The first run on 62 phrases nobody had tested the rules against missed 8 urgent ones, including two ways of describing self-harm. The rules were widened by pattern, not by pasting the phrases, and the development set still scores 160/160 with 0 false urgent. Because this set has now been seen, a second held-out set is being written by a clinician who has not read the rules.
Missed in the first run
| ID | Input | Lang | Category | Expected | Actual | Failing stage |
|---|---|---|---|---|---|---|
| rc-097 | أمس صار ثقل في إيدي وراح بعد ربع ساعة | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-098 | yesterday my speech was slurred for ten minutes, fine now | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-100 | this morning my arm was numb for a few minutes, it passed | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-101 | أمس صار ثقل في إيدي وراح بعد ربع ساعة. | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-102 | yesterday my speech was slurred for ten minutes, fine now. | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-104 | this morning my arm was numb for a few minutes, it passed. | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-105 | أمس صار ثقل في إيدي وراح بعد ربع ساعة وش أسوي؟ | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-106 | My husband says: yesterday my speech was slurred for ten minutes, fine now | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-108 | My husband says: this morning my arm was numb for a few minutes, it passed | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-109 | زوجتي تقول: أمس صار ثقل في إيدي وراح بعد ربع ساعة | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-110 | yesterday my speech was slurred for ten minutes, fine now, what should I do? | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-112 | this morning my arm was numb for a few minutes, it passed, what should I do? | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-113 | أمس صار ثقل في إيدي وراح بعد ربع ساعة وأنا قلقان | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-114 | Since this morning yesterday my speech was slurred for ten minutes, fine now | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-116 | Since this morning this morning my arm was numb for a few minutes, it passed | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-117 | من الصبح أمس صار ثقل في إيدي وراح بعد ربع ساعة | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-118 | yesterday my speech was slurred for ten minutes, fine now and I am worried | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-120 | this morning my arm was numb for a few minutes, it passed and I am worried | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-121 | أمس صار ثقل في إيدي وراح بعد ربع ساعة الله يعافيك | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-122 | Please help, yesterday my speech was slurred for ten minutes, fine now | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-124 | Please help, this morning my arm was numb for a few minutes, it passed | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-125 | أبوي: أمس صار ثقل في إيدي وراح بعد ربع ساعة | ar-gulf | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-126 | yesterday my speech was slurred for ten minutes, fine now (my father) | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
| rc-128 | this morning my arm was numb for a few minutes, it passed (my father) | en | TIA symptoms resolved < 24 h | same_day | emergency | over-escalation by R-BEFAST, R-TIA24 (safe direction) |
Type a phrase to enable the check. Nothing is saved.
Computed live: every case is run through the same rules the app uses. These are authored phrases and the rules were developed against them, so this is a development set. The held-out suite (hash above in the overview) runs in the backend eval. Resolved TIA phrases currently route to emergency instead of same day: safe, but shown as disagreements.