Scrugo
⚔️ Comparison 4 products compared EN

Memory management for local Qwen 27B: pi-blackhole, pi-vcc, pi-protokoll, billion-context or transparent-compact?

Memory management for local Qwen 27B: pi-blackhole, pi-vcc, pi-protokoll, billion-context or transparent-compact?
🏆 Our Top Pick

transparent-compact is the best overall choice based on our tests.

📅 Published on
transparent-compact

transparent-compact

9.0 /10
0€
  • ✓ Compaction in 0.01 s: deterministic rendering, zero model calls
  • ✓ 3/3 on the agentic task, 6/6 on fact recall, rule held 3/3
  • ✓ Built-in LLM fallback (/compaction-method summary) + automatic fallback to Pi
  • ✓ 16.4k prompt tokens per request, 68.3% cache hits
  • ✓ 5,739,157 B package (including 5,405,494 B of shell parser in wasm), no postinstall
pi-blackhole

pi-blackhole

8.0 /10
0€
  • ✓ Best measured cache hit rate: 79.4%
  • ✓ Algorithmic compaction in 0.06 s, 6/6 on fact recall
  • ✓ No retention bug across the 4 bench runs
  • ✓ postinstall at install time + 48 dependency packages (tsup, typescript)
pi-protokoll

pi-protokoll

7.0 /10
0€
  • ✓ The cheapest measured prompt: 14.9k tokens per request
  • ✓ 3/3 on the task, 6/6 on recall, cache 73.3%
  • ✓ 7.9 s per compaction (work offloaded to an auxiliary model)
  • ✓ 8 protocols out of 9 written as a catch-up: Qwen does not write the <protocol> block spontaneously
pi-vcc

pi-vcc

6.5 /10
0€
  • ✓ 3/3 on the task, 6/6 on recall, compaction in 0.01 s
  • ✓ 22% of its compactions keep nothing verbatim (retention bug reproduced)
  • ✓ 4 to 7 compactions per turn: the worst cache of the bench (66.7%)
  • ✓ Two distinct lineages share the same tool name (monotykamary, sting8k)

Also available inEnglishFrench(original)

Translated with AI assistance and reviewed by the author.

My Qwen 27B sessions were dying without any error message: empty response, turn cut off in the middle of a tool batch, "EMERGENCY" alerts up to 246% of the window. I audited 248 real sessions, then built a reproducible bench to compare 10 memory-management configurations (compaction, i.e. the automatic summarization of context when the window fills up) under Pi. All of them run on the same model: Qwen 27B INT4 served by QEngine on 2 RTX 3090s, 262k window.

Three results to remember:

  1. Pi's native compaction costs 501 s per compaction at 240k of context (median of my 5 real compactions).
  2. The best remedy is not a better LLM summarizer, but deterministic rendering, with no model call: transparent-compact compacts in 0.01 s.
  3. My previous configuration (billion-context-pi + context-mode) was the worst of the 10: 0/3 on the agentic task and 9 runs out of 9 cut dead.

Vocabulary

TermMeaning in this article
CompactionAutomatic summarization of the history when the context nears the limit
SliceThe old part of the history that compaction replaces with a summary
TurnOne user request and all the agent's work up to its answer (there can be dozens of tool calls)
Mid-turnCompaction triggered in the middle of a turn, not between two turns
Cache hitShare of the prompt already cached server-side: the higher it is, the less gets recomputed
CheckpointThe summary block the extension writes in place of the slice
DeterministicSame input, same output, no model call

Comparison table (12 criteria)

CriterionNative Pipi-vcctransparent-compactpi-protokollpi-blackholebillion-context-pi
Compaction duration (median)4.39 s at 32k · 501 s at 240k0.007 s0.015 s0.00 s + 7.9 s (auxiliary)0.06 s— (native compaction cancelled)
Fact recall (S2)5/6 (83.3%)6/66/66/66/6n/a (0 completed run)
Agentic task (S1)3/33/33/33/32/20/3 (runs abruptly cut off)
Prompt tokens per request18.4k16.8k16.4k14.9k18.5k23.1k
Cache hits72.6%66.7%68.3%73.3%79.4%53.6%
Compactions per turn (S3, ~300 kB)3733not tested0 (no compaction)
Runs abruptly cut off (stopReason length)0/90/90/90/90/49/9
History search toolnonevcc_recallcontext_lookup (sub-agent)protocol_* (after 1st compaction)recallacp_* (unreliable)
Fallback on failureerror bubbles up, no retrynative compaction (but 22% empty compactions)double: /compaction-method summary + automatic fallback to Pimissing protocol if model doesn't write its own algorithmic compactionnone (silent death)
Package: size / postinstall—266,356 B / none5,739,157 B / none221,253 B / none2,356,641 B / yes4,015,065 B / none
Benchmark runs99994 (S1 and S2 only)9 (+ 3 at 64k)
Confidence levelhighmediummedium-high (nothing measured at 262k)mediummedium-low (4 runs, no S3)high (death mode identified, reproduced, explained)

In 30 seconds

You want…Take…WhyGood to know
Compaction with no risk of a dead sessiontransparent-compact (0.12.0)0.01 s per compaction, 3/3 on the task, 6/6 on fact recall, built-in LLM fallback5,739,157 B package (including 5,405,494 B of shell parser in wasm); its author is the author of the study that validates it
The best cache hit ratepi-blackhole (0.5.18)79.4% hits, 0.06 s per compaction, 6/6 on recallpostinstall, 48 dependency packages, only 4 runs on the bench
The fewest prompt tokens per requestpi-protokoll (0.9.10)14.9k tokens per request, the cheapest measured7.9 s per compaction (auxiliary model); 8 protocols out of 9 written as a catch-up on Qwen
Install nothingNative Pi (baseline)it works, 3/3 on the task501 s per compaction at 240k, 5/6 facts recalled, 1 answer out of 3 misaligned

If you are on billion-context-pi or billion-context-pi-lean, go straight to "[The death mode of billion-context-pi]" and then to the migration: the problem is not the quality of the summary, it is a session safety hole.

My final recommendation: remove billion-context-pi, install @foliox/transparent-compact-pi@0.12.0, set reserveTokens to 40,960 and keepRecentTokens to 47,000. The full settings are in the last section.

Why memory is the bottleneck (not the model)

My profile, measured on 248 sessions: 93.4% of the context comes from tool outputs, 5 native compactions in 248 sessions, and p90 = 4 compactions within a single turn (max 14). The context grows fast, and when compaction triggers, it operates on huge slices.

Pi's native compaction is an LLM call: the model summarizes itself. At 240k tokens, that costs a median of 501 s (max 650 s). On the bench, with 33k of context, 4.39 s. The cost follows the size of the slice: ×114 between the two regimes.

fig01 key figures

Of these key figures, only two are discriminating: the duration of native compaction (501 s) and the share of mid-turn compactions (100% of my 5). The rest mainly shows that all the variants capable of finishing the task do finish it.

The bench

ItemValue
HardwareGIGABYTE MZ31-AR0-00, AMD EPYC 7401P 24 cores / 48 threads, 125 GB RAM + 70 GB swap, RTX 3090 Ti (300 W) + RTX 3090 (275 W)
ModelQwen3.8-27B INT4, dual-GPU, declared window 262,144 tokens, maxTokens 32,768
EngineQEngine 1.3.5.4 (container, image pinned by digest)
AgentPi 1.5.2, Node.js 22, Ubuntu 24.04.4
Bench window32,768 tokens (compaction threshold 24,576), plus 6 runs at 64k
Parameterstemperature 0.2, same seed for all variants
Volume12 labels (10 configurations + 2 window variants), 3 scenarios, 86 runs

The 10 configurations: native Pi (baseline), pi-vcc 0.11.2, transparent-compact 0.12.0, pi-protokoll 0.9.10, pi-blackhole 0.5.18, billion-context-pi-lean 0.4.1 (written bcp-lean), my config at the time (billion-context-pi 0.1.83 + context-mode 1.0.169), pi-vcc + context-mode, transparent-compact + context-mode, and a coexistence test pi-vcc + transparent-compact. The 2 window variants are bcp-lean and my config replayed with a 64k window, to check that the death mode really depends on the window size.

The 3 scenarios:

  • S1, agentic task: read a 1,597-line spec (6 reads of ~30 kB), implement, write the tests.
  • S2, planted facts: 12 facts and 1 absolute rule ("never modify config/"), 6 massive reads, an explicit violation attempt, then 6 recall questions.
  • S3, giant turn: 10 consecutive reads (~300 kB cumulative) in a single turn, to force mid-turn compactions.

Breakdown of the 86 runs. 8 configurations have 9 runs each (3 per scenario), for 72. pi-blackhole and the coexistence test have only 4 runs (2 × S1, 2 × S2, no S3). The 6 runs at 64k make 3 per variant (2 × S1, 1 × S3, no S2). Total per scenario: 32 in S1, 28 in S2, 26 in S3.

Each run is isolated: dedicated empty HOME, per-variant config directory, throwaway project reset in git. The bench is reproducible on any OpenAI-compatible server; only the GPU, the throughput (52 tok/s effective here; a 40 s run becomes 4 min on a single 3090) and QEngine are specific.

S1: the agentic task

8 configurations out of 10 succeed at 100%, with total times from 41 to 61 s. The speed differences between them are not significant: with 3 runs, the intervals overlap. What matters is robustness, and there the billion-context group stands out, negatively.

VariantS1CompactionsCompaction duration (s)Cache hitsTotal time
Native Pi3/324.3972.6%44.8 s
pi-vcc3/340.0166.7%56.4 s
transparent-compact3/330.0168.3%46.1 s
pi-protokoll3/330.00 ¹73.3%60.5 s
pi-blackhole2/22.50.0679.4%41.1 s
bcp-lean1/30—52.5%39.5 s
my config (bcp + ctx-mode)0/30—53.6%16.4 s

¹ protokoll's work is done beforehand, in a request to an auxiliary model: 7.91 s measured per compaction in its own entries (22 events).

The total time of my config (16.4 s) is the shortest in the table, but it is a fast give-up, not a performance.

fig06 task time

The most telling graph of the bench shows the context over the course of a long turn: it climbs to the threshold, drops by 30 to 60%, climbs again, and so on (a sawtooth). pi-vcc makes 4 to 5 drops per turn, down to ~3k: it keeps little. transparent-compact makes 3, down to 6-15k. Native Pi makes 2, but each one costs 4.39 s here and ~500 s at my real size. Both billion-context variants have no drop at all: their curve exits through the top of the window. That is the death mode, visible directly.

fig03 context sawtooth

S2: the planted facts and the absolute rule

6 precise facts are asked for at the end of the turn, after 2 to 4 compactions: port 4318, table tickets_v3, version 2.4.1, the owner, the secret value, the rule. Native Pi recalls 5 facts out of 6 (83.3%, i.e. 3 facts lost out of 18 across its 3 runs). All the other completed variants score 6/6.

VariantFacts recalledRule respectedAligned answers
Native Pi5/6 [4–6]3/31/3
pi-vcc6/63/33/3
transparent-compact6/63/33/3
pi-protokoll6/63/33/3
pi-blackhole6/62/22/2
my config—n/an/a (0 runs completed)

The 83.3% / 100% gap remains thin with 3 runs. The baseline's real symptom is alignment: 1 answer out of 3 pairs the right value with the wrong question (port 7 instead of 4318, for example). No extraction-based variant has this flaw over 3 runs.

fig05 fact recall

fig04 rule compliance

Two caveats. The rule holds everywhere (100% on all completed variants): the model refused the violation every time, so this criterion separates no one. And no variant called a history-search tool in this protocol (0/3 everywhere): I cannot measure their precision.

S3: the giant turn

10 massive reads in a single turn (~300 kB). This is the scenario that forces mid-turn compactions, hence my real use case (p90 = 4 per turn). pi-blackhole and the coexistence test have no S3 run.

VariantCompactions"12 sections""port 4318"Total time
Native Pi33/33/344.4 s
pi-vcc73/33/370.0 s
transparent-compact33/33/341.7 s
pi-protokoll33/33/357.0 s
bcp-lean00/32/332.6 s
my config00/30/329.7 s

pi-vcc compacts 7 times per turn: that is the price of the worst cache of the bench (66.7%). Both billion-context variants never compact and lose the information.

The death mode of billion-context-pi

This is the most important result of the bench. It explains abnormal session endings already present in my history (11 EMERGENCY alerts, up to 246% of the window). The sequence is identical on 9 runs out of 9 for my config and 7 out of 9 for bcp-lean, and never appears on the other variants:

  1. The model launches several tools in parallel in a single response (5 bash + 1 read at once in one run). The context jumps from ~10k to 44,698 tokens in a single leap.
  2. billion-context-pi has cancelled native compaction (session_before_compact → {cancel:true}). So Pi does nothing at the end of the tool batch.
  3. The extension's alert arrives too late: "EMERGENCY 242%".
  4. Pi reduces max_tokens to fit in the window, down to 1 token: the response comes out with stopReason: "length" and an empty text.
  5. The emergency compaction is cancelled by the same extension. Pi does not retry. The turn stops silently, the task is abandoned.

At a 64k window, no run is cut (0/3 for each of the two variants): the death mode appears when the context crosses the declared window; it is not a permanent bug. The "last resort" described in the bcp-lean README (truncating large outputs at 95% of the window) does not protect against a parallel tool batch.

The symptom on your side: an empty response at the end of a long turn, with no error message. The fix is the migration described below.

What compaction does to the context

Measured on each compaction (context size after, divided by size before):

VariantCompactionsContext after/beforeAverage summaryEmpty firstKeptEntryId
Native Pi190.38577 tok0
pi-vcc460.73504 tok10 (22%)
transparent-compact250.421,308 tok0
pi-protokoll220.36723 tok0
pi-blackhole90.40692 tok0

pi-vcc reduces the context less (0.73 versus 0.36-0.42) and leaves 22% of its compactions with no message kept as-is: the model then sees only summaries. It is the only retention bug reproduced across the 86 runs.

transparent-compact's summary is bigger (1,308 tok versus 504-723) because it keeps the prose verbatim and discards tool outputs. On compactions of more than 60,000 characters, 99.0% of the summarized slice is tool outputs (3,488,518 characters versus 33,680 of prose). Its checkpoint weighs 3.77× the prose, versus 1.77× for pi-vcc: its size follows the prose, not the tool outputs. With a profile at 93.4% tool outputs, that is the right behavior.

Cost: cache and tokens per request

Each compaction rewrites the start of the prompt, thus invalidates the cache: the more often a variant compacts, the more it pays.

fig07 cost per turn

Cache hits: pi-blackhole 79.4% > transparent + context-mode 76.7% > pi-protokoll 73.3% ≈ native Pi 72.6% > transparent-compact 68.3% > pi-vcc 66.7%. Prompt tokens per request: pi-protokoll 14.9k < transparent-compact 16.4k < pi-vcc 16.8k < native Pi 18.4k < pi-blackhole 18.5k < my config 23.1k.

context-mode is a net cost in my scenarios. With the same compactor, transparent-compact alone takes 46.1 s / 3 compactions / 16.4k tokens / 14 requests, versus 75.8 s / 6 compactions / 20.9k / 19.5 requests with context-mode: +64% time, +28% tokens, twice as many compactions, with no gain on the task or on recall. Its 10 tools cost 7,600 schema tokens on every request, and my scenarios produce no output big enough for its offloading (ctx_execute) to repay that overhead.

This is not a condemnation. My real usage (2,940 ctx_execute calls, 93.4% tool outputs) is precisely the case it targets, and the bench does not test it.

Last caveat: the server counter is unusable (340k declared server-side for 85k client-side, i.e. ~×4, consistent with a multi-GPU count). All token figures come from the client.

Flaws, security and maturity

I checked 7 packages on their tarball, without execution:

PackageSizeFilespostinstallDependencies
pi-vcc 0.11.2266,356 B42nonenone
transparent-compact 0.12.05,739,157 B30nonenone
pi-protokoll 0.9.10221,253 B8nonenone
billion-context-pi 0.1.834,015,065 B62nonenone
billion-context-pi-lean 0.4.13,762,470 B56nonenone
pi-blackhole 0.5.182,356,641 B—yestsup, typescript (48 packages)
context-mode 1.0.1694,249,752 B—yes141 packages

Points of caution:

  • transparent-compact goes from 150,239 B (0.8.8) to 5,739,157 B in 0.11.1. 5,405,494 B come from a single file, shell.wasm, a shell parser compiled to WebAssembly used to reduce bash calls to a single line. It is a parsing binary, not malicious code nor model weights. Real cost: ~5.4 MB on disk, 0 context tokens.
  • context-mode runs a postinstall that modifies configuration files outside its package (plugin repair, MCP JSON sweep). Install it knowingly.
  • None of these packages calls the network at runtime (no API URL in the code, checked by grep).
  • Maturity of transparent-compact: single author, who is also the author of the study publishing its figures (conflict of interest, for you to weigh).
  • Maturity of billion-context-pi: its -lean fork has a different maintainer for the same engine, and a successor (billion-context) already exists, so my config's upstream is end-of-life.
  • pi-vcc carries two distinct lineages (monotykamary, the historical one, and sting8k, the current one) under the same tool name.
  • pi-protokoll is the lightest, but its mechanism requires the model to write a <protocol> block every turn, and Qwen does not do so spontaneously.

The summary (heatmap) normalizes 5 criteria between 0 and 1: speed, fidelity, task, overhead, robustness. It brings out three groups: the candidates (transparent-compact, protokoll, blackhole, transparent + context-mode), the baseline (fidelity 0.83, speed 0.00), and the two billion-context variants (robustness 0.22 and 0.00), disqualified not for the quality of their summaries but because their runs do not reach the end.

fig09 heatmap

The 0-10 scores in the comparison sheet are my editorial judgment, not a measurement: they sum up the bench ranking (speed, robustness, fidelity) and the confidence I have in each result.

The decision, point by point:

  • remove billion-context-pi;
  • add transparent-compact 0.12.0, pinned version;
  • keep context-mode, but put it to the test on real usage;
  • do not install pi-context-prune: it is an LLM summarizer per tool batch, exactly the cost that transparent-compact removes.

Mechanism (verified in the code): it is a hybrid. Pi triggers compaction (threshold) and chooses the cut point; the extension builds the content (deterministic rendering, zero model calls); Pi writes the entry. The fallback is dual: /compaction-method summary switches back to Pi's LLM compaction, and if construction fails, the extension returns undefined and Pi takes over again. To identify these compactions afterwards: details.kind === "mech-compact" in the session JSONL.

// ~/.pi/agent/settings.json
{
  "packages": [
    // … inchangé …
    "npm:context-mode@1.0.169",
    "npm:@foliox/transparent-compact-pi@0.12.0"   // épinglé : pas de mise à jour silencieuse
    // retiré : "npm:billion-context-pi"
  ],
  "compaction": {
    "enabled": true,
    "reserveTokens": 40960,
    "keepRecentTokens": 47000,
    "modelOverrides": {
      "qengine-local/qwen3.8-27b-qengine": { "reserveTokens": 40960, "keepRecentTokens": 47000 }
    }
  }
}
# variables à mettre dans l'environnement de lancement de pi
export MECH_COMPACT_MAX_SUMMARY_PERCENT=30   # plafond du point de contrôle (défaut 25)
export MECH_COMPACT_LOOKUP_TURNS=12          # défaut 25 : trop cher sur un 27B local
export MECH_COMPACT_LOOKUP_MAX_CALLS=3       # garde-fou anti-boucle

Why these values:

  • reserveTokens 40,960 (instead of 24,576): it must cover the output limit (32,768). Otherwise a long generation overflows the window just after the threshold, and Pi catches it through its overflow path instead of a clean compaction. Accepted side effect: the threshold drops from 237,568 to 221,184 tokens, so we compact a bit earlier.
  • keepRecentTokens 47,000 (instead of 40,000): we keep a bit more recent history as-is.
  • Windows under 128k: switch to summary mode, not verbatim. The extension's README itself says it is not suited to this range (the 30% cap of a 32k window is 9,600 tokens, the checkpoint may be truncated there).

Watch out for MECH_COMPACT_LOOKUP_MODEL: by default, it is the session model. The retrieval sub-agent therefore runs the same QEngine, with serialized requests on a single GPU server and a rewrite of the start of the main session's prompt. Three options, in order of preference:

  1. Change nothing and throttle (MAX_CALLS=3, TURNS=12): we pay in latency, not in confidentiality.
  2. Enable mainAgentLookup: the retrieved pages enter the main context, a less-tested mode.
  3. Point to a cheap cloud model: I advise against it, the session history would leave the machine.

The migration, executed and verified on my config:

# 0. sauvegarde
cp ~/.pi/agent/settings.json ~/.pi/agent/settings.json.bak-$(date +%Y%m%d-%H%M%S)

# 1. retirer le compacteur qui annule la compaction native
pi remove npm:billion-context-pi

# 2. installer le remplaçant (version épinglée)
pi install npm:@foliox/transparent-compact-pi@0.12.0

# 3. redémarrer une session, puis vérifier
#    dans la 1re requête : outil "context_lookup" présent, "compress" absent

# 4. après la première compaction
#    attendu : "fromHook":true et details.kind "mech-compact"
#    interdit : "compactor":"pi-vcc"

Rollback in under a minute: pi remove the package, pi install the old one, restore the backup. Nothing is lost: sessions are append-only JSONL, and compactions already written remain readable.

The blind spot: nothing is measured at 262k. The duration is solid, because a deterministic compaction does not depend on the size of the slice. However, the size of the checkpoint at 240k is extrapolated from measured ratios: between 9k and 22k tokens, i.e. 3.5% to 8.5% of the window. The 25% cap should therefore not trigger, but it is an extrapolation, not a measurement. If your checkpoint exceeds ~20% of the window in real usage, raise MECH_COMPACT_MAX_SUMMARY_PERCENT to 35-40 rather than letting truncation bite into the prose: the prose is what carries fidelity.

Additional variants

Two options that do not have their own card in the comparison sheet.

Native Pi (Pi's native compaction, no extension): score 5.0, free

  • 3/3 on the agentic task, 5/6 on fact recall (83.3%), rule held 3/3
  • 4.39 s per compaction at 32k, 501 s at 240k (median, 5 real compactions)
  • 18.4k prompt tokens per request, 72.6% cache hits
  • 1 answer out of 3 misaligned on recall (right value, wrong question)
  • No history-search tool, no fallback: the error bubbles up, no internal retry
  • Nothing to install (bench baseline)

billion-context (successor of billion-context-pi): not tested on the bench, free

  • Successor of billion-context-pi: the upstream of the tested config is end-of-life
  • Same engine as the tested family: the model calls compress and cancels native compaction
  • No fallback: if the model does not compress in time, the request exceeds the window and the run dies (death mode reproduced 9/9 on the family)
  • Footprint of 8,670 tokens confirmed by reading the code (486 for the lean fork)
  • No bench run: these findings are inherited from the family, not measured on this version

Sources and method

Everything "measured" comes from the bench: 86 runs, analysis and aggregation scripts kept, complete sessions (772 MB), 12 isolated installations. The history audit (248 sessions) is a read-only pass. The "n/a" entries on the graphs are non-measurable cells, displayed as such rather than filled in with 0.

What I could not test: drift over a very long session (no 100+ turn run), the precision of the history-search tools (spontaneous usage almost nil), and the per-request prefill (server counters unusable).

The arXiv study 2610.20116 measures 167.8 s per native compaction with Qwen 27B and 0.43 s for transparent-compact: it confirms the order-of-magnitude gap. It tested a different version of pi-vcc (monotykamary lineage), so I do not transpose its figures to mine.

👤 Which product for whom?

**Long sessions on local Qwen 27B (262k window)**

transparent-compact

Deterministic compaction in 0.01 s with no model call, dual LLM fallback, 3/3 on the task and 6/6 on recall. It is the only candidate whose fallback is both tested and built in.

**You want to maximize the cache hit rate**

pi-blackhole

Best measured cache (79.4%) and compaction in 0.06 s, but only 4 runs on the bench: validate before switching, and be aware of its postinstall.

**You want to minimize prompt tokens per request**

pi-protokoll

The cheapest measured (14.9k per request), but 7.9 s per compaction via an auxiliary model and 8 protocols out of 9 written as a catch-up on Qwen.

**You don't want to install anything**

Native Pi

It works, but 501 s per compaction at 240k and 1 answer out of 3 misaligned: acceptable for short sessions, not for long ones.

**You are on billion-context-pi**

Migrate to transparent-compact

Your sessions can be cut dead (9 runs out of 9 on the bench): the extension cancels native compaction with no safety net. 5-minute migration, immediate rollback.

By fabz 🌱🏅🧠🧪

❤️ Like this comparison? Leave a star to support fabz !

💬 Discussions & Questions

0 / 5000

📬 Get the weekly newsletter

Report this content