Quick start

Three steps to a limiter

Register a named limiter, add its handler to an HttpClient, and set the rates. Requests now wait for the limiter, and the limiter learns from each response.

1. Add the package from NuGet.
2. Register a limiter and its handler.
3. Configure the rates in units per minute.

Read the getting started guide →

Program.cs

builder.Services.AddAdaptiveRateLimiter("orders-api");

builder
    .Services.AddHttpClient<OrdersClient>(c =>
        c.BaseAddress = new Uri("https://api.example.com/")
    )
    .AddAdaptiveRateLimitHandler("orders-api");

appsettings.json

{
  "AdaptiveRateLimiting": {
    "Limiters": {
      "orders-api": {
        "InitialUnitsPerMinute": 600,
        "MinUnitsPerMinute": 60,
        "MaxUnitsPerMinute": 1200
      }
    }
  }
}

How it works

A feedback loop in front of every request

1. 01

   Wait

   The request waits in a token bucket until its units are free.

2. 02

   Send

   The handler sends the request to the API through HttpClient.

3. 03

   Classify

   A 429, or a 503 with Retry-After, is a throttle. A success that had to wait is a signal to grow.

4. 04

   Adapt

   The controller sets the next rate. A throttle also pauses for Retry-After.

See the bucket, the loop and the pause →

Features

Everything a client needs to share a quota

Retry-After pause

A throttle with Retry-After holds back all requests of the limiter, not only the one that failed.

MaxPause

Pluggable controllers

AIMD (default) and CUBIC are built in. Write your own controller for a different curve.

IRateController

Request cost

A request can cost more than one unit. The estimator tells the cost before the request goes out.

IRequestCostEstimator

Shared budget

Several processes split one quota. Set a static count, or give a live source.

IInstanceCountSource

Metrics

Units, throttles, rejections, wait time, rate, queue and instances, ready for OpenTelemetry.

System.Diagnostics.Metrics

Configuration and DI

Named limiters bind from configuration and reload on change. The binding does not use reflection.

IConfiguration

Rate controllers

Pick the curve that fits your API

AIMD

Default

Adds a fixed step on success and multiplies the rate down on a throttle. Steady and easy to reason about.

Quota used

90%

429s

56

Recovery

8 s

CUBIC

Opt-in

Grows on a cubic curve around the last rate that failed. Uses more of the quota, with more throttles.

Quota used

100%

429s

531

Recovery

27 s

Model numbers: 15 minutes, 2 instances at 2,000 units/min each, quota 3,000 units/min with a 50% drop for 60 s.

Compare the controllers →

Packages

A small core, and an add-on for Microsoft Graph

Nall.RateLimiting.Adaptive

The limiter, the HttpClient handler, the AIMD and CUBIC controllers, DI registration and metrics.

dotnet add package Nall.RateLimiting.AdaptiveNall.RateLimiting.Adaptive.MicrosoftGraph

SharePoint resource-unit costs for Graph requests and $batch, throttled inner $batch responses and CSOM.

dotnet add package Nall.RateLimiting.Adaptive.MicrosoftGraph

See it adapt before you ship it

Run None, Fixed, AIMD and CUBIC against the same quota in your browser. Change the quota, the demand and the number of instances.

Open the simulatorWhy adaptive rate limiting