LongMemEval: 463 of 500 questions answered correctly (92.6%)
Internal test · public dataset
Test conditions:v2.0.4, no built-in model (embedded_models=false). Official scoring rules (±1 day allowed on time questions). Answering and grading used external models; the specific models, test date and number of runs are not stated in the source material.
Limits:Not independently verified. Results vary with the answering model. We can walk you through a re-run during commercial discussions.
For 457 of 470 answerable questions, the needed evidence was in the top 30 results (97.23%)
Internal test · offline retrieval
Test conditions:Same dataset; retrieval only, with no external model or network calls.
Limits:Measures whether evidence was found, not the final answer accuracy.
Hardware-verified on Win11 x86_64, Linux x86_64 and ARM64 RK3588
Internal hardware verification
Test conditions:Before the v2.0.4 release, 7 usage paths were run — portable drive operation, save and search, permissions, evaluation, legacy migration, cross-tool sharing, ARM64 and Win11 hardware — all passed.
Limits:macOS is still in validation; RK3576 and other ARM64 devices are assessed per project.
Release package ≈3.1 MB compressed; server 3.55–4.56 MiB unpacked
Measured package size
Test conditions:v2.0.4. ARM64 fully static server 3.55 MiB (3,724,384 bytes); Linux x86_64 server 4.56 MiB.
Limits:Excludes memory data and attachments.
About 4.70 GB for one million conversation turns
Structural estimate
Test conditions:Extrapolated from the measured text compression ratio and a typical turn length (about 270 turns a day for 10 years).
Limits:Excludes large attachments such as drawings and screen recordings; actual size depends on content.