Fetch API Errors and Limits

PacketStream Fetch API error codes and when to retry, which requests are billed, rate, concurrency, and per-site limits, robots.txt, and acceptable use.

Error format

Errors use the same format as the Search API:

{
  "request_id": "7c1e0f9a-4b2d-4e8f-a1c3-5d6e7f8a9b0c",
  "error": {
    "code": "target_error",
    "message": "target returned HTTP 404"
  }
}

Handle errors by code. Messages explain the problem but can change. Every 429 carries Retry-After: 1.

Error codes

Code	HTTP	Meaning	Does retrying help?
`invalid_request`	400	Bad JSON, or a missing, unknown, repeated, mistyped, or out-of-range field. The message names it.	No. Fix the request.
`url_not_allowed`	400	The URL, an address it resolves to, or a redirect target breaks the URL rules. The message never repeats the URL.	No. Use another URL.
`start_index_out_of_range`	400	`start_index` is at or past `total_length`, which the message gives.	No. Use a smaller `start_index`.
`unauthorized`	401	The API key is missing, malformed, revoked, or unknown.	No. Check the key.
`insufficient_balance`	402	Your balance is below $0.0025, or other requests spent it while this one ran.	After you add funds.
`method_not_allowed`	405	Not a POST.	No. Use POST.
`request_too_large`	413	The body is over 4,096 bytes.	No. Shorten it.
`target_error`	422	The site returned a non-2xx status (for example “target returned HTTP 404”), the host does not exist, its TLS certificate is invalid or the handshake failed, or it sent a malformed response.	Usually no. Only when the site’s status was temporary, such as a 429 or 5xx.
`unsupported_content_type`	422	A PDF, image, or other unsupported type or compression.	No.
`javascript_required`	422	The page needs JavaScript to show its content.	No.
`empty_content`	422	The page has no readable text.	No.
`too_many_redirects`	422	More than 5 redirects.	No. Request the final URL instead.
`rate_limited`	429	Your key’s request rate, or the per-site limit for the target, was exceeded.	Yes, after `Retry-After`. Slow down on that site.
`concurrency_limited`	429	Four fetch requests with this key are already running on the API server that received this one.	Yes, once one finishes.
`internal_error`	500	Unexpected server error.	Retry once. If a duplicate charge matters, check your balance first.
`fetch_unavailable`	503	Fetch could not reach the site through the PacketStream network, even after one retry, or failed internally.	Yes, with backoff.
`capacity_exceeded`	503	Fetch capacity is temporarily full.	Yes, with backoff.
`target_timeout`	504	The site was too slow, or the 25-second limit passed.	Sometimes, later.

Billing

A fetch that returns page content costs $0.0025 from your prepaid balance. A request needs at least that much available before it starts.
Every error is free, including the site’s own errors, timeouts, rate limits, and start_index_out_of_range.
Rare exception: a 500 internal_error can follow a charge that went through just before the failure. Your balance is authoritative.
Each part of a long page that you read with start_index is billed separately.

Limits

Rate. Each key may send 5 requests per second, with bursts of 10. Searches, batch items, fetch requests, and MCP tool calls share this limit. Fetch responses carry X-RateLimit-Limit and an approximate X-RateLimit-Remaining. Requests rejected during validation do not use the limit.
Concurrency. Up to 4 fetch requests per key run at once on each API server. Beyond that, fetch returns 429 concurrency_limited. Don’t rely on more than 4 at once.
Per site. Each site (registrable domain) accepts only a few new fetch requests per second, counted across all customers. Beyond that, fetch returns 429 rate_limited. Spread requests to one site over time.
Time. The server finishes or gives up within 25 seconds. Use a client timeout of at least 35 seconds.
Size. A response holds up to max_length characters, at most 500,000. A page is converted up to 3 MiB of body, 50,000 HTML elements, or 8 MiB of Markdown, and page.source_truncated tells you when a limit was hit.

robots.txt and acceptable use

Fetch does not read or obey robots.txt. You, or your agent, decide which pages to fetch.

You must follow the Terms of Service, including section 7, Acceptable use. It allows use of the Services “only for lawful purposes and only where you have any permission required by law, contract, or the operator of a target system.”

Privacy

The API records each fetch request’s ID, outcome, latency, and charge. It does not store the URL, the user_agent, or the page content.