DevOps & Observability
Measuring API and Report Request Volume
Measure API and report demand by client, tenant, report type, latency, concurrency, payload and request cost.
On this page
Request volume is the first question in most capacity and rate-limit discussions, and the easiest one to answer badly. A single number such as “40,000 requests a day” hides who sent them, when they arrived, what each one cost, and whether they did anything useful. Measuring volume well means recording every request with enough identity and cost context to explain the traffic, not just count it.
Minimum record
Record one row per request: request_id, correlation_id, authenticated tenant_id, client_id, API key identifier, report type, requested range, arrival and completion times, duration, success or failure, response size, rows, concurrency at arrival, cache outcome, and query duration. Derive tenant and client from authenticated identity, never from values the caller supplies in the request body. Never log an API key secret; the identifier is enough to attribute traffic.
Write this record asynchronously and keep it bounded, so a slow telemetry store cannot slow down or fail report requests. Keep high-cardinality values such as request_id and filter details in the request log rather than in metric labels.
Rates and slices
Measure requests per minute and per hour, then slice by client, API key, tenant, report type, and status. Per-minute rates show bursts and retry storms; per-hour rates show the daily shape of demand and the windows when capacity is actually needed. Keep both, because an hourly average can look comfortable while individual minutes exceed what the platform can serve.
Track active concurrency and queue age alongside the rate. The same requests per minute behave very differently when they arrive evenly than when they arrive in a burst at the top of the hour. Separate interactive JSON pages from large CSV exports, since they have different latency expectations and very different costs.
Heavy consumers
Rank consumers by request count and by measured cost, and compare the two lists. A client that sends many small cached requests may be harmless; a client that sends a few month-long live exports may dominate query time. Look at each heavy consumer’s report types, date ranges, formats, cache hit rate, and error rate before deciding whether its traffic is a problem.
Traffic patterns
Plot volume over time for each heavy consumer and for the platform as a whole. Useful patterns to recognize:
- Business-hours demand: interactive traffic that follows working hours and drops overnight.
- Scheduled jobs: sharp spikes at fixed times, often on the hour, from integrations or batch exports.
- Bursts and retries: short spikes of failed or timed-out requests followed by immediate repeats.
- Steady polling: a flat, constant rate from one client at regular intervals, regardless of time of day.
Scheduled spikes and polling are often the easiest traffic to reshape, because the caller controls the timing.
Useful traffic versus excessive polling
A request is useful when the caller needs a result it does not already have. Excessive traffic asks the same question repeatedly and receives the same answer. To find it, compare requests from the same client, tenant, and report with identical normalized parameters, the same canonical key a report cache would use. Many identical requests within the report’s freshness window, a high cache hit rate for one client, or repeated requests for a report that is still running all indicate polling or repeated calls rather than new demand.
Respond to the pattern rather than to the volume alone. Repeated identical reads can be served from cache. Clients that poll for long-running work can move to an asynchronous pattern where they submit once and check status. Clients that retry immediately after failures need explicit throttling guidance, such as a retry-after interval, instead of timeouts that encourage another attempt.
Request cost
One hundred cheap cached requests can cost far less than one hundred month-long live reports. Model cost using date range, report weight, expected row band, format, query time, reads or scanned bytes, output bytes, and cache outcome. Version and calibrate the model against measured query time; it is an operational estimate, not billing truth.
Capacity and rate-limit decisions
Use the measured rates, concurrency, and cost to set limits deliberately. Base peak capacity on per-minute and concurrency evidence for the busiest windows, not on the daily total. Apply tenant-aware quotas and concurrency limits from authenticated identity, so one consumer cannot exhaust capacity for others. Consider limits by cost as well as by count: a request budget that treats a cached JSON page and a year-long CSV export as equal will be either too strict or too generous.
Route large work to an asynchronous pattern where appropriate, return explicit throttling responses with guidance, and review limits as the traffic patterns change. Re-measure after each change; a new quota or cache rule should show up as a lower repeated-request rate or lower cost per consumer, not merely as fewer requests.
Tradeoffs and failure modes
Detailed request logs improve diagnosis but add storage, privacy, and cardinality costs; keep metrics aggregated and push per-request detail to logs with defined retention. Caching reduces repeated work but introduces freshness questions, so measure cache hit rate per report and per consumer rather than only globally.
Watch for partial outcomes when counting. A report can finish after its caller disconnects, and a cache write can fail after a valid result was returned. Count those outcomes explicitly instead of folding them into success or failure, and do not record an unknown duration or row count as zero.
Operational review questions
Before relying on the numbers, check that a responder can answer these from retained evidence: which client and tenant sent the traffic, which report types and formats dominated, whether the volume was new demand or repeated identical requests, and whether failures triggered retries. Confirm that dashboards distinguish rejected, queued, executing, completed, failed, and cancelled requests. If the evidence cannot separate a growing customer from a misconfigured poller, the measurement is not yet ready to drive capacity or rate-limit decisions.
Was this helpful?
What could be better?
Thanks — that helps me improve Javi Builds.
Continue learning
- Cheat sheetLLM Cost & Token BudgetingEstimate and control LLM workload cost with simple formulas for cost per request, per day and per month, plus token budgets, model tiering, caching, batch workloads and monitoring.Open cheat sheet
- ArticleDesigning an Execution Log for Microsoft Fabric Pipelines and NotebooksDesign a production execution log for Microsoft Fabric pipelines and notebooks using run IDs, correlation IDs, status, row counts, watermarks, errors and rerun-safe patterns.Read article
What should I build next?
I'm building Javi Builds around practical problems data engineers actually face. Tell me what you'd like to see next.
Stay in the loop
Get the Javi Builds updates you actually care about.
Check your inbox
I sent you a confirmation link. Your subscription will be active after you confirm it.