Skip to content

Benchmark evidence

Context,
under pressure.

Two public evaluations test long-term recall and repository-level software engineering with measurable outcomes.

LoCoMo 90.78 percent accuracy question-answer accuracy
SWE-bench Pro 86.73 percent task resolution tasks resolved with PowerContext on

Two benchmarks. Two different questions.

Memory quality matters twice: first when an agent must recover what happened, then when it must use context to finish real work.

LoCoMo

Can the system remember a long-running conversation?

It measures direct recall, temporal reasoning, multi-step reasoning, and context grounded open-domain answers.

Explore the memory test
SWE-bench Pro

Can an agent turn context into a working patch?

It gives Codex a real repository and issue, then grades the resulting patch with executable tests.

Explore the coding test

LoCoMo tests long-term recall.

The public dataset contains long, multi-session conversations. PowerContext is evaluated on the answerable question set, where facts can be separated by many sessions.

Conversations
10
Scored questions
1,540
Question types
4
841

Single-hop

Recover one fact from the conversation history.

321

Temporal

Reason about dates, order, and duration across sessions.

282

Multi-hop

Connect several facts before producing an answer.

96

Open-domain

Combine conversation evidence with general knowledge.

One run, three ways to carry context.

Switch metrics to compare PowerContext, PowerMem, and placing the entire conversation in the prompt.

+37.88 points above full context
PowerContext90.78%
PowerMem87.79%
Full context52.9%

Higher is better.

12.4x full context took longer
PowerContext1.38 s
PowerMem1.44 s
Full context17.12 s

Lower is better.

93.7% fewer than full context
PowerContextabout 1.65k
PowerMemabout 0.9k
Full context26k

Lower is better.

SWE-bench Pro tests whether context changes the patch.

Each task starts from a real codebase and issue. Codex edits the repository, and the official task tests decide whether the patch resolves the problem.

Same task set731 public v2 repository issues in both arms.
Same modelgpt-5.6-sol with medium reasoning in Codex.
Controlled switchOFF disables plugins. ON enables the installed PowerContext plugin.
PowerContext OFF 602 of 731 tasks resolved with PowerContext off 82.35% resolved
PowerContext ON 634 of 731 tasks resolved with PowerContext on 86.73% resolved
+32 more tasks resolved
In the reported paired run, PowerContext ON improved task resolution by 4.38 percentage points.

The public score index.

Put the published field next to PowerContext, while keeping the test rig visible.

Data checked August 31, 2026

LoCoMo scores on all 1,540 questions

Only results that explicitly disclose the complete 1,540-question scope are included. Reader, judge, and answer-policy differences remain visible.

RankSystemScorePublished protocolEvidence
1 Zep 94.70% 1,540 questions; GPT-5.4 reader and judge Vendor run
2 EverMemOS 94.50% 1,540 questions; lenient shared harness; precomputed retrieval Third-party run
3 XMDB 93.20% 1,540 questions; internally verified Vendor run
4 TrueMemory Pro 93.00% 1,540 questions; three-run mean; lenient judge Open harness
5 Mem0 92.50% 1,540 questions; latest vendor evaluation rig Vendor run
6 PowerContext 90.78% 1,540 questions; 1,398 correct; topical judge Project run
7 Honcho 89.90% 1,540 questions; current full-system result Vendor run
8 Dakera 88.20% 1,540 questions; single pass; no LLM reranker Reproducible vendor run
9 PowerMem 87.79% 1,540 questions; historical project run Project run
10 Memvid 85.65% 1,540 questions; GPT-4o reader; lenient judge Open harness
11 Genesys 85.55% 1,540 questions; ten-run mean; frozen Mem0 protocol Certified vendor run
12 Engram 84.50% 1,540 questions; shared lenient harness Third-party run
13 MemHQ 83.20% 1,540 questions; gpt-4o-mini; partial answers accepted Open harness
14 Logica Mind 72.50% 1,540 questions; Mem0 paper protocol Open harness
15 Supermemory 65.40% 1,540 questions; shared lenient harness Third-party run
One dataset scope, visible test rigs

Every row explicitly reports all 1,540 scored questions. Results with an undisclosed or different question count are excluded. The remaining scores still use different readers, judges, and answer policies, so the rank is a public evidence index rather than an official LoCoMo leaderboard.

Read each result for the question it answers.

The two evaluations share a context theme, but their inputs, outputs, and graders are intentionally different.

Evaluation dimensionLoCoMoSWE-bench Pro
What is testedLong-term conversational recall and reasoningRepository-level issue resolution
InputMulti-session dialogue history and a questionA repository, an issue, and a clean task environment
OutputA grounded natural-language answerA code patch
Primary scoreJudge-rated answer accuracyOfficial executable tests passed

Evidence and methodology

Follow the dataset, paper, harness, and published PowerContext figures from the original sources.

  1. PaperEvaluating Very Long-Term Conversational Memory of LLM Agents

    The ACL 2024 paper that defines LoCoMo and its long-term memory tasks.

  2. Datasetsnap-research/locomo

    The public ten-conversation dataset and annotations.

  3. Benchmarkscaleapi/SWE-bench_Pro-os

    The public benchmark repository and official evaluation path.

  4. HarnessPowerContext evaluation console

    The pinned dataset, OFF and ON arms, isolated runner, and reporting contracts.

  5. ResultsPublished PowerContext benchmark figures

    The current project README values used on this page.

Inspect the system behind the scores.

PowerContext is open source. Review the implementation, evaluation harness, and contracts directly.

View on GitHub