Stop paying for the same LLM request twice.
Zecache is a semantic caching API that detects equivalent LLM requests and returns cached responses when it is safe to reuse them — reducing unnecessary provider calls, latency, and cost.
Click a step above to simulate how ZeCache handles requests.
See how initial calls populate the cache and repeat calls serve instant hits without provider cost.
LLMs don't care that two requests mean the same thing. Your cache does.
Different wording often represents the exact same underlying request. A traditional key-value cache sees them as completely separate strings — resulting in repeated LLM calls, unnecessary cost, and extra latency.
Traditional Key-Value Caching
Exact String Hash"How can I reset my password?"
"How do I reset my password?"
"I forgot my password. How can I reset it?"
Zecache Semantic Matching
Exact + Vector + Safety"How can I reset my password?" (MISS → forward & store)
"How do I reset my password?" (HIT → cached response)
"I forgot my password. How can I reset it?" (HIT → cached response)
How Zecache Works
Three simple steps to reduce provider calls without changing your application architecture.
Send your request
Your AI application sends its completion request through Zecache using your existing OpenAI client library configuration.
Zecache checks the cache
Zecache first checks for an exact match and then evaluates semantically similar cached requests against cache safety rules.
Reuse or call provider
If a safe cache match exists, Zecache returns the cached response. Otherwise, it forwards to your LLM provider and stores the completion.
Semantic similarity isn't enough.
Similarity is only the beginning. Zecache evaluates whether a cached response is safe to reuse before returning it to your application.
🔍 Why Cache Safety Checks Matter
"What is the capital city of France?"
"Where is the capital city of France?"
Equivalent query — safe to reuse response.
"What is the capital city of France?"
"What is the capital city of Germany?"
Different entity — must NOT share response.
"What is today's exchange rate?"
"What was yesterday's exchange rate?"
Time-dependent — forward to LLM provider.
✓ Similarity Threshold
Set configurable cosine similarity bounds (e.g. 0.85). Candidates below threshold are safely treated as cache misses.
✓ Model Compatibility
Ensures cached completion responses are only reused across identical or compatible model families.
✓ Namespace Isolation
Isolate cache entries per tenant or user namespace to enforce strict isolation and prevent cross-user data leakage.
✓ TTL & Freshness
Durable expiration policies ensure stale completions expire automatically according to your data freshness needs.
✓ Cache Policy Modes
Select between Exact-Only, Semantic, No-Cache, or Shadow Mode to evaluate cache decisions with zero live traffic risk.
✓ Parameter Verification
Compares temperature, system prompt, and request parameters before approving cache response reuse.
Add caching without rebuilding your AI stack.
Designed around an OpenAI-compatible API surface. Keep using the official client libraries you already know — just update your base URL and API key.
import openai
import os
# Point OpenAI client to ZeCache API Base URL
client = openai.OpenAI(
base_url="https://api.zecache.com/v1",
api_key=os.environ.get("ZECACHE_API_KEY", "cf_live_your_zecache_key")
)
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Explain quantum computing in simple terms."}]
)
print(response.choices[0].message.content)Built for developers building AI applications
Focused infrastructure designed to eliminate redundant LLM calls and simplify semantic caching.
Lower LLM Costs
Avoid paying repeatedly for equivalent requests when a valid cached response already exists in your cache.
Lower Latency
Cached responses avoid expensive LLM provider round trips, returning cached completions significantly faster.
Higher Throughput
Serve frequent repeated requests without sending every individual request to your LLM provider endpoint.
Less Infrastructure
No need to build and operate custom vector storage, embeddings, similarity indexes, TTL rules, and cache invalidation.
Built for repetitive LLM workloads
Ideal for AI applications where users frequently phrase equivalent requests.
Customer Support Bots
Frequent repetitive questions about passwords, billing, and returns.
FAQ & Help Desk Assistants
Common user documentation queries with high semantic overlap.
RAG Knowledge Search
Repeated retrieval questions against identical or similar document contexts.
AI Classification & Extraction
Structured data extraction workloads with recurring prompt structures.
Dedicated developer infrastructure.
Zecache focuses specifically on high-performance semantic caching rather than bloated AI gateways.
General AI Gateways
Often bundled with complex prompt management, model routing, guardrails, and enterprise feature layers you may not need for core caching.
Zecache Caching Engine
Intentionally focused on semantic caching → exact matching, vector similarity evaluation, TTL, and cache safety policies.
We're testing Zecache with real AI workloads.
We're currently validating semantic matching, cache safety, latency, and cost efficiency. We're looking for developers building AI applications who want to test semantic caching in real workloads and help shape the product.