Toolbox Gateway
A self-hosted platform for tool-using agents: one gateway to OpenAI, Gemini and Bedrock, tools served over MCP, and every step traced with cost.
The problem
When every agent wires an SDK straight into a script, provider keys spread everywhere, costs are invisible, tools can't be reused, and nothing stops an agent that keeps calling tools forever.
How it works
An LLM gateway owns credentials, retries, caching and cost accounting, and translates one provider-neutral tool schema into native function calling for OpenAI, Bedrock Converse and Gemini. Tools live in their own service, reachable through a JSON bridge or native MCP. Agents run a LangGraph loop with concurrent tool calls and a round budget, over HTTP or WebSocket, and a web console shows every model and tool call.
Pipeline
- 01Console
- 02Agent (HTTP / WS)
- 03LLM gateway
- 04Toolbox (MCP)
- 05Trace
Where it's used
- Companies running several agents that need one place for keys, cost and limits
- Sharing internal tools between agents, IDEs and MCP clients
- Comparing providers on cost and latency before committing to one
Highlights
- Native tool calling across three providers, no prompt-parsed tool calls
- Toolbox usable by any MCP client (Claude Desktop, MCP Inspector…)
- Per-step latency, tokens, cost and cache hits; p50/p95 metrics
- Round budget that stops runaway tool loops
Stack
Want something like this for your company?
I build production versions of these systems on your data and your infrastructure.
Related service: Agents & workflow automation
Contact me