Skip to content
worldgovdata
Documentation / Core concepts

Core concepts

Rate limits

Three ceilings — per minute, per day, and how many requests may be in flight at once — set by your account tier and reported on every response.

Rate limits and credits are independent. A full balance does not raise your limits, and being rate-limited does not cost you a credit.

The limits

Rate limits by account tier
TierWhoPer minutePer dayConcurrent
AnonymousNo key, or a key on an unverified account301,0004
FreeVerified account, never purchased12020,0008
PaidHas ever completed a purchase600500,00024
  • Limits are counted per key when you send one, and per IP address when you do not. Two keys on one account get their own minute windows.
  • Paid is permanent. An account that has ever completed a purchase stays on the paid tier even at a zero balance — the tier is a trust signal, not a subscription, and throttling you at the moment you are trying to top up would be perverse.
  • Unverified means anonymous. A key on an account that has not confirmed its email address authenticates normally but is limited at the anonymous tier. If a fresh integration is inexplicably slow, check verification first.

Concurrency

The third column is a different kind of ceiling: how many of your requests may be in flight at the same moment. It has no window — a slot frees the instant one of your own requests finishes.

Exceeding it returns 429 like the others, but the response deliberately carries no X-RateLimit-* headers at all: those describe a windowed quota, and a Reset of 0 would tell you to retry at the Unix epoch. Retry-After is the honest answer, and it is short — the wait is however long one of your own in-flight requests takes to finish.

Concurrency refusal · 429

HTTP/1.1 429 Too Many Requests
Retry-After: 1
Cache-Control: no-store

{
  "error": {
    "code": "rate_limited",
    "message": "Too many requests in flight for this key. Wait for one to finish before starting another.",
    "retry_after": 1
  }
}

In practice this only bites when you fan out a large chunked pull. Cap your worker pool at the tier’s concurrency number and it never appears.

The headers

Every response carries your current standing. There is no separate endpoint to poll, and reading these is free.

Rate-limit headers
HeaderExampleMeaning
X-RateLimit-Limit600Requests allowed in the current one-minute window, for your tier.
X-RateLimit-Remaining594Requests left in that window. Reaches 0 just before a 429.
X-RateLimit-Reset1785315600Unix epoch second at which the window resets — a timestamp, not a countdown.
Retry-After37Seconds to wait. Sent on 429 and on 423. Authoritative — prefer it to your own backoff.

What a 429 looks like

Rate limited · 429

HTTP/1.1 429 Too Many Requests
Retry-After: 37
Cache-Control: no-store
X-Request-Id: 0f3a91c4e2b7
X-RateLimit-Limit: 600
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1785315600

{
  "error": {
    "code": "rate_limited",
    "message": "Rate limit exceeded.",
    "retry_after": 37
  }
}

Retry-After is always present on a 429 and is always at least 1 — never 0, because a client honouring Retry-After: 0 turns into a hot loop against the thing the limit was protecting. It is the authoritative number. Prefer it to whatever your own backoff would have computed.

The same value also appears as retry_after in the error body, so a client that only parses JSON is not left guessing.

Backing off correctly

The rule is short. Retry on 429, on any 5xx, and on a transport failure with no response at all. Retry on nothing else: every other 4xx means the request itself is wrong and will fail identically forever, so retrying it only burns your quota faster.

  • Honour Retry-After when present. The server knows exactly when the window opens; your formula is a guess.
  • Exponential otherwise. 1s, 2s, 4s, 8s, 16s — doubling, for 5xx and transport failures where no header exists.
  • Cap the delay. Thirty seconds is plenty. Uncapped doubling turns the eleventh attempt into a seventeen-minute sleep that nobody intended.
  • Add jitter. Not decoration: without it, every client that started together retries together, and the recovery attempt is indistinguishable from the original stampede.
  • Cap the attempts. Five is a reasonable ceiling. A loop with no limit does not fail — it hangs, which is worse.

Retry with backoff

# Retry a GET on 429 and 5xx. Retry-After wins; otherwise 1s, 2s, 4s, 8s, 16s.
wgd_retry() {
  local url="$1" attempt=0 max=5 code wait
  local headers body
  while :; do
    headers="$(mktemp)"; body="$(mktemp)"
    code="$(curl -sS -D "$headers" -o "$body" -w '%{http_code}' \
      -H "Authorization: Bearer $WGD_API_KEY" "$url")" || code="000"

    if [ "$code" -ge 200 ] && [ "$code" -lt 300 ]; then
      cat "$body"; rm -f "$headers" "$body"; return 0
    fi

    if { [ "$code" = "000" ] || [ "$code" = "429" ] || [ "$code" -ge 500 ]; } \
       && [ "$attempt" -lt "$max" ]; then
      wait="$(awk 'tolower($1)=="retry-after:"{gsub(/\r/,"",$2); print $2; exit}' "$headers")"
      [ -n "$wait" ] || wait="$(( 2 ** attempt ))"
      sleep "$(awk -v w="$wait" 'BEGIN{srand(); print w + rand()}')"
      attempt=$(( attempt + 1 ))
      rm -f "$headers" "$body"
      continue
    fi

    cat "$body" >&2; rm -f "$headers" "$body"; return 1
  done
}

These are the retry logic only. Complete clients — auth, typed errors, paged fetch-all — are on Client libraries.

Staying under the limit

Cache

The data behind these endpoints changes on a daily-to-annual cadence, and a request served from your own cache costs no credit and consumes no quota. This is the single highest-leverage thing on this page.

Note where the cache has to live. An authenticated response is private, no-store, because it carries your balance and your remaining quota — put a CDN in front of it and you would be serving one caller’s numbers to another. Anonymous catalogue responses are public, max-age=3600 and genuinely shareable. So: cache inside your application, not in front of it.

Fewer, larger requests

/v1/data takes a comma-separated country list. Twenty-five countries in one request is one request against your minute budget; twenty-five requests is twenty-five. The credit cost is nearly identical and the rate-limit cost is not close.

Do the free work first

Resolve indicator ids, country codes and coverage from the free catalogue endpoints before you start pulling rows. They still count against your request budget, but they cost no credits — and doing them up front means the expensive calls are all correct first time.

Verify, then buy

Verifying your email takes you from 30 requests a minute to 120. A first purchase takes you to 600. Both are step changes, not gradients.

Limits on the website

The account endpoints behind worldgovdata.com carry their own, much tighter, per-IP limits. They exist so that password guessing and signup mail-bombing are expensive. They are listed here for completeness — nothing in an API client should ever meet them.

Per-IP limits on account actions
ActionPer IP address
Sign up5 per hour
Log in20 per 15 minutes
Password reset request5 per hour
Verification email resend5 per hour

Repeated failed sign-ins additionally lock the account for a period and return 423 account_locked with a Retry-After. A valid API key is never locked out this way.

Need more?

The paid tier’s 500,000 requests a day covers essentially every research and product workload we have seen. If yours genuinely needs more, or you need a sustained burst for a one-off migration, talk to us — describe the shape of the job and we will tell you whether the limits or the credits are the binding constraint.