Product benchmarks
Measure the effectof context.
LoCoMo measures factual recall across long conversations. LoCoMo Plus tests whether implicit constraints are remembered and applied. SWE-bench Pro measures whether project context helps Codex resolve more repository issues.
| System | Accuracy | Search p95 |
|---|---|---|
| PowerContext | 90.78% | 1.38 s |
| PowerMem | 87.79% | 1.44 s |
| Full context | 52.9% | 17.12 s |
LoCoMo: retrieve the right context
LoCoMo asks questions about long conversations split across sessions. PowerContext must find the relevant conversation evidence before producing an answer. This run covers 1,540 scored questions in categories 1 through 4.
90.78% answer accuracy
PowerContext answered 1,398 of 1,540 questions correctly, 37.88 percentage points above full-context prompting. Search p95 was 1.38 seconds, with about 1.65k answer tokens per question.
LoCoMo Plus: apply implicit memory constraints
Even when a new question does not resemble an earlier cue, the system must retain and apply the user's state, goals, values, and causal background. These results compare native PowerContext with Jev candidate filtering across three models.
Each model is used for both Memory extraction and answer generation. Jev filters candidates after retrieval; it does not replace either model role.
- Embedding model
- qwen3.7-text-embedding
- 1024 dimensions
These six scores were supplied by the project owner. The corresponding run manifests, summaries, protocol and model/Judge configurations, run hashes and raw outputs are not publicly available for verification. Reproducibility documentation describes the harness, not verification of these scores.
Highest reported score
69.28%
GPT-4o · PowerContext + Jev
Largest score difference
+8.925
GPT-4o-mini · percentage points
| Extraction & answer model | Native PowerContext | PowerContext + Jev | Difference |
|---|---|---|---|
| GPT-4o-mini | 40.855% | 49.78% | +8.925percentage points |
| Qwen3.7-plus | 60.37% | 68.947% | +8.577percentage points |
| GPT-4o | 65.082% | 69.28% | +4.198percentage points |
SWE-bench Pro: turn context into working patches
To measure the effect of PowerContext, we ran the same Codex configuration twice on all 731 public v2 tasks. The two arms differed only in whether PowerContext was enabled.
This is a paired run on a pinned task set, not an official SWE-bench Pro submission. Agent runs are stochastic, so the scores describe these two runs only.
Comparison with published results
Explore published LoCoMo and LoCoMo Plus results alongside the official SWE-bench Pro Public leaderboard. Each entry links to its evaluation configuration and source.
LoCoMo / SWE-bench Pro checked August 31, 2026
15 systems · All 1,540 scored questions
- 1ZepVendor run94.70%
- 2EverMemOSThird-party run94.50%
- 3XMDBVendor run93.20%
- 4TrueMemory ProOpen harness93.00%
- 5Mem0Vendor run92.50%
- 6PowerContextProject run90.78%
- 7HonchoVendor run89.90%
- 8DakeraReproducible vendor run88.20%
- 9PowerMemProject run87.79%
- 10MemvidOpen harness85.65%
- 11GenesysCertified vendor run85.55%
- 12EngramThird-party run84.50%
- 13MemHQOpen harness83.20%
- 14Logica MindOpen harness72.50%
- 15SupermemoryThird-party run65.40%
- Native PowerContext
- 65.082%
- PowerContext + Jev
- 69.28%
These six scores were supplied by the project owner. The corresponding run manifests, summaries, protocol and model/Judge configurations, run hashes and raw outputs are not publicly available for verification. Reproducibility documentation describes the harness, not verification of these scores.
Published Cognitive scores
8 published entries · Verified October 7, 2026- T-Mem74.81%T-Mem paper · Table 3Cognitive; GPT-4.1-mini construction; GPT-4o answers; Gemini-2.5-Flash judge
- HyperMem48.63%T-Mem paper · Table 3Cognitive; baseline experiment in the T-Mem paper
- MemOS32.67%T-Mem paper · Table 3Cognitive; baseline experiment in the T-Mem paper
- Gemini-2.5-Pro26.06%LoCoMo Plus paper · Table 1Cognitive; full context; Gemini-2.5-Flash judge
- GPT-4o21.05%LoCoMo Plus paper · Table 1Cognitive; full context; Gemini-2.5-Flash judge
- A-Mem17.20%LoCoMo Plus paper · Table 1Cognitive; GPT-4o answers; Gemini-2.5-Flash judge
- Mem015.80%LoCoMo Plus paper · Table 1Cognitive; GPT-4o answers; Gemini-2.5-Flash judge
- SeCom14.90%LoCoMo Plus paper · Table 1Cognitive; GPT-4o answers; Gemini-2.5-Flash judge
Rank (UB) is calculated from confidence intervals; overlaps produce ties and skipped positions.
- 1Muse Spark 1.1Meta · mini-swe-agent61.50%±3.10
- 1gpt-5.4 (xHigh)OpenAI · mini-swe-agent59.10%±3.56
- 3Muse SparkMeta · mini-swe-agent55.00%±3.60
- 3claude-opus-4-6 (thinking)Anthropic · mini-swe-agent51.90%±3.61
- 5gemini-3.1-pro (thinking)Google · mini-swe-agent46.10%±3.60
- 5claude-opus-4-5-20251101Anthropic · Scale run45.89%±3.60
- 5claude-4-5-SonnetAnthropic · Scale run43.60%±3.60
- 5gemini-3-pro-previewGoogle · Scale run43.30%±3.60
- 5claude-4-SonnetAnthropic · Scale run42.70%±3.59
- 10gpt-5-2025-08-07 (High)OpenAI · Scale run41.78%±3.49
- 10gpt-5.2-codexOpenAI · Scale run41.04%±3.57
- 10claude-4-5-haikuAnthropic · Scale run39.45%±3.55
- 10qwen3-coder-480b-a35bAlibaba · Scale run38.70%±3.55
- 14minimax-2.1MiniMax · Scale run36.81%±3.55
- 14gemini-3-flashGoogle · Scale run34.63%±3.55
- 16gpt-5.2OpenAI · Scale run29.94%±2.15
- 16kimi-k2-instructMoonshot · Scale run27.67%±3.25
- 18qwen3-235b-a22bAlibaba · Scale run21.41%±2.25
- 19gpt-oss-120bOpenAI · Scale run16.20%±2.67
- 19deepseek-v3p2DeepSeek · Scale run15.56%±2.63
- 21gemma-3-27b-itGoogle · Scale run11.38%±2.15
- 21llama3-1-405b-instructMeta · Scale run11.18%±2.15
- 21glm-4.6Z.ai · Scale run9.67%±2.15
- 24llama4-maverick-17b-instructMeta · Scale run5.24%±1.24
- 25codestral-2405Mistral · Scale run1.51%±1.51
Evaluation methods and sources
Method references and evaluation tools for all three benchmarks. Read models, scoring rules, and data scope together.
LoCoMo
LoCoMo Plus
SWE-bench Pro
Use PowerContext in your workflow
Start with a supported agent or connect an application through the HTTP API. The repository contains the runtime, integrations, and evaluation tools.