AI Requests in One Place: The Case for Running Your Own Proxy
Jared Siegel proposes logging and managing requests to multiple AI models from a single point. Beyond the usage reporting providers offer, he wants to watch the traffic before it reaches the model provider.
On October 18, Jared Siegel argued on his personal blog that AI requests should be funneled through a single proxy you run yourself. A proxy is an intermediary gateway that passes requests between an app and an AI model. Even if you use models from several different companies, routing everything through that one gateway lets you see, in one place, exactly which requests went out.
The proposal is aimed at anyone who wants a detailed record of their own usage, from people paying for AI subscriptions to developers wiring up several models at once. It is the kind of setup you reach for when you run a service that connects multiple models and you want to know which app called which model, and when.
Which app called which model
The approach Siegel describes is to consolidate model requests into a single gateway under your own control. From there, requests are forwarded on to models at various companies while the proxy logs the access key used for the call, the program that sent the request, the model, the time, and the status. Instead of seeing only aggregate usage totals, you can follow the path of each individual request.
The proposal also stresses that the operator can modify the gateway to fit their own needs. Siegel explains that operators can use virtual access keys and adjust the proxy as required. He also says that when a cache miss occurs during coding work, meaning stored data cannot be reused, he tracks down the cause manually in 20 seconds.
How this differs from provider usage reporting
The difference lies in where logging and control happen. OpenAI supports a feature that lets developers look up usage by access key. A proxy you run yourself makes every request pass through a route you manage before it reaches the model provider.
One piece of software that supports this kind of setup is LiteLLM. Its official documentation explains that a proxy you install and operate on your own server supports model connections, virtual keys, budget management, and per-request cost logging. You can attach multiple models, of course, and also manage spending budgets and the cost of each request.
What to check before using it in Korea
Anyone in Korea considering this approach needs to look at the software they choose and the support offered by the model providers they plan to connect to. The features of the proxy software and the actual conditions under which a given model can be used are two separate things to verify. Availability and pricing at the time you plan to use it also have to be checked against the setup you intend to build.
The focus of this proposal is less on how well the models answer and more on how closely you can inspect the process of using them. If you operate a service that stitches together several models, it is worth considering a setup that manages usage and request logs side by side. In weighing whether to adopt it, consider whether it can capture the records you need, what it would cost to run, and how much work it would take to maintain.