Free LLM for context compression

Call it with one model id

Point any OpenAI-compatible client at your local Ficelle endpoint and ask for ficelle/auto-compression. Conversation-history compaction.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8646/v1", api_key="ficelle-local")

reply = client.chat.completions.create(
    model="ficelle/auto-compression",
    messages=[{"role": "user", "content": "..."}],
)

What a model needs to be routed here

• Free pricing: a model with any price is never eligible, and paid fallback is hardcoded off.
• Tool calling support.
• At least 64,000 tokens of context by default.
• At least 12,000 completion tokens available.
• Compression quality, checked by Ficelle's compression probe.

How the probe gate decides:

• A model that failed the probe is excluded.
• A model is eligible when it passed the probe, or when its provider's catalog declares the capability.
• A capability Ficelle only inferred from provider defaults must pass the probe first.
• If the gate would exclude every candidate, Ficelle keeps the pool rather than empty it, and labels the served model in the route log so the fallback is visible.

Good for

• Hermes history compaction
• Summarising a long session before handing it to another model
• Shrinking logs before analysis

Worth knowing

• Streams are committed on the first substantive delta, so a compaction that stalls early fails over instead of hanging.
• Read-inactivity budget for this profile: 45 seconds before Ficelle gives up on a silent provider and fails over.

Where the models come from

The pool is built from the free models of the providers you configure. The open core ships OpenRouter and Nous; Pro maintains the broader provider pack. Whether a given provider has a model eligible for this type changes as catalogs change, so Ficelle decides at routing time, not this page.

• OpenRouter
• Nous
• Mistral
• Groq
• Gemini
• NVIDIA
• Cerebras
• SiliconFlow
• Requesty
• OVH Cloud
• SambaNova
• Cloudflare
• Routeway

Questions