CascadeFlow
lemony-ai/cascadeflowReviewed by AI
Checked by RepoPicks Radar on
What is CascadeFlow?
cascadeflow is a library for Python and TypeScript (plus an n8n node) that sits inside the AI calls of a program you are writing. It asks a small, cheap model first, checks the answer against rules you set, and only sends the question on to a big model when the check fails. It also adds up the cost of each call, can hold a run to a budget, and can refuse a tool call or stop an agent. It talks to OpenAI, Anthropic, Groq, Together, Google, DeepSeek and to local models in Ollama or vLLM, and plugs into LangChain, CrewAI, PydanticAI and Google ADK. You have to write code to use it.
How easy is it to install CascadeFlow?
C · install 1 or 2 tools · Windows
- Windows C
- macOS C
- Linux C
C on Windows, Mac and Linux alike: "pip install cascadeflow" (or "npm install @cascadeflow/core"), a provider key in a .env file, and about ten lines of your own code listing the cheap model and the expensive one. There is nothing to open and no interface; the n8n node is the closest thing to a point-and-click route.
CascadeFlow on GitHub
- Stars
- 3,925on Nov 10, 2026
- Downloads
- 18916 this week
- Growth
- —no gain in the last reading
Stars over time
Facts
- Runs on
- Windows, macOS, Linux
- You need to install
- Python, Node.js
- To build from source
- Python, Node.js
- Disk space
- 200 MB–800 MB · estimate
- Accounts or keys
- At least one model provider: OpenAI, Anthropic, Groq, Together AI, Hugging Face, Google, Azure OpenAI, DeepSeek or OpenRouter (not needed if both models are local, in Ollama or vLLM)
- Cost
- Free, API costs extra · The library is free under the MIT licence. What you pay is the model providers behind it, which is the whole point of the project; running the small model on your own machine with Ollama or vLLM costs nothing.
- License
- MIT (Permissive)
- Can replace
- Portkey, OpenRouter, LangSmith
- Last change on GitHub
- Sep 25, 2026
CascadeFlow risk signals (5)
- Attentionthe cheap models answer is kept when the check passes
By design it "Speculatively executes small, fast models first" and only escalates when validation fails, and that validation is length, "Confidence scoring (logprobs analysis)", format and an optional similarity score with a default "semanticThreshold: 0.5". Whether those thresholds catch a wrong answer in your subject is on you; a reply from the small model is delivered as the final one.
AI review
- Infothe savings numbers come from the maintainers
The README leads with "Cost Savings: 69% (MT-Bench), 93% (GSM8K), 52% (MMLU), 80% (TruthfulQA) savings, retaining 96% GPT-5 quality" and later "Result: 40-85% cost reduction, 2-10x faster responses, zero quality loss". The repository ships examples and tests but no benchmark you can run to reproduce those figures.
AI review
- Infothe cost report uses prices you type in
Prices are hand-written in the configuration, as in ModelConfig(name="gpt-5", provider="openai", cost=0.00562), and the examples print result.total_cost from them. The numbers it reports drift from your real invoice as soon as a provider changes its prices.
AI review
- Infothe published package is older than the code
The newest release is v1.2.0 from 2026-04-02 and pyproject.toml also says version 1.2.0, while the repository kept changing until 2026-09-25. What "pip install cascadeflow" gives you is the April package, not what you read here.
AI review
- Infogateway mode has no password in the instructions
For existing apps the README offers a drop-in gateway: "python -m cascadeflow.server --mode auto --port 8084", with your provider keys loaded. No token or password is mentioned for that port, so anything that can reach it can spend through your keys.
AI review
Evidence is shown in its original language.
What Radar could not check about CascadeFlow
- Whether the savings hold for your own prompts and models
- Whether the quality check spots a wrong answer in your subject before it reaches your users
- The real bill at the end of the month, since the figures it prints come from the prices you entered
- Whether the package on PyPI and npm matches the code in this repository
- The claimed overhead of under 5 milliseconds per step