How to Diagnose Residential Proxy Latency Without Blaming the Exit

A residential proxy request that “feels slow” is usually several clocks stacked on top of each other. Treating time_total as a quality score for the exit will send you after the wrong lever: a new country, a new session, more concurrency, or a different vendor.

Diagnose the path first. Split DNS, the TCP/TLS hop to the gateway, destination time to first byte, and body transfer. Then change only the stage that actually moved.

This is a client-side method you can run from the same machine or container as the failing job. It is not a provider ranking and not a latency SLA.

Name the clocks before you time anything

A successful HTTPS request through PacketStream crosses more than one network:

your runtime
  → DNS for proxy.packetstream.io
  → TCP to the gateway
  → TLS to the HTTPS proxy (port 31111)
  → CONNECT / tunnel setup
  → destination DNS (as seen from the residential exit)
  → TCP + TLS to the destination
  → first response byte
  → remaining body

PacketStream automatic rotation selects a residential exit when the client opens a new proxy connection, not when the application constructs another request object. If your HTTP pool reuses a connection, later requests skip most of the gateway handshake. If each attempt opens a new connection, you pay that handshake every time. Those two policies produce different totals even when the exit and destination are unchanged.

Write the unit of work down before the first timed call:

job        = one expected observation
attempt    = one proxy-backed HTTP exchange
connection = one TCP/TLS session to proxy.packetstream.io
success    = destination responded AND the payload passed validation
             AND the observed country matched the requested country

A slow success and a fast 200 with the wrong payload are different incidents.

Freeze the controls

Two timings are comparable only when the following stay fixed—or are reported as changed:

Record the endpoint you actually used. The recommended HTTPS proxy is proxy.packetstream.io:31111. SOCKS5 is proxy.packetstream.io:31113. HTTP on 31112 exists for compatibility and should not be the default for a new harness.

Keep credentials out of the command line. Use PACKETSTREAM_USER and PACKETSTREAM_AUTH_KEY, then put them in a private cURL config instead of expanding them into cURL’s arguments. PacketStream usernames, generated auth keys, and supported targeting/session modifiers use only characters that are safe inside the quoted config value below.

Capture stages, not a single total

cURL can emit a JSON timing record while writing the body somewhere else. Do not mix diagnostics with payload. This setup keeps credentials out of process arguments, creates both files with mode 0600, and removes them when the shell exits:

set -euo pipefail
umask 077

diagnostic_dir=$(mktemp -d "${TMPDIR:-/tmp}/packetstream-curl.XXXXXX")
curl_config="$diagnostic_dir/curl.conf"
body_file="$diagnostic_dir/body"
timing_file="$diagnostic_dir/timing.tsv"
transport_file="$diagnostic_dir/transport.tsv"
trap 'rm -f -- "$curl_config" "$body_file" "$timing_file" "$transport_file"; rmdir -- "$diagnostic_dir"' EXIT

printf 'proxy = "https://proxy.packetstream.io:31111"\nproxy-user = "%s:%s"\n' \
  "$PACKETSTREAM_USER" "$PACKETSTREAM_AUTH_KEY" >"$curl_config"

curl_status=0
curl --disable --config "$curl_config" \
  --fail \
  --silent --show-error \
  --connect-timeout 15 \
  --max-time 45 \
  --output "$body_file" \
  --write-out '%{time_namelookup}\t%{time_connect}\t%{time_appconnect}\t%{time_pretransfer}\t%{time_starttransfer}\t%{time_total}\t%{http_code}\t%{http_connect}\t%{size_download}\t%{remote_ip}\t%{url_effective}\n' \
  'https://ipinfo.io' >"$timing_file" || curl_status=$?

awk -F '\t' '{
  post_wait = "null"
  if (($5 + 0) > 0 && ($5 + 0) >= ($4 + 0)) {
    post_wait = sprintf("%.6f", $5 - $4)
  }
  printf "{\"namelookup\":%s,\"connect\":%s,\"appconnect\":%s,\"pretransfer\":%s,\"starttransfer\":%s,\"post_pretransfer_wait\":%s,\"total\":%s,\"http_code\":%d,\"proxy_connect_code\":%d,\"size_download\":%s,\"remote_ip\":\"%s\",\"url\":\"%s\"}\n", \
    $1, $2, $3, $4, $5, post_wait, $6, $7 + 0, $8 + 0, $9, $10, $11
}' "$timing_file"

IFS=$'\t' read -r _ _ _ _ _ _ destination_code connect_code _ _ _ <"$timing_file"
destination_status=$((10#$destination_code))
proxy_connect_status=$((10#$connect_code))
status_error=0

if (( proxy_connect_status < 200 || proxy_connect_status >= 300 )); then
  printf 'proxy CONNECT returned %03d\n' "$proxy_connect_status" >&2
  status_error=1
fi
if (( destination_status < 200 || destination_status >= 300 )); then
  printf 'destination returned HTTP %03d\n' "$destination_status" >&2
  status_error=1
fi
if (( curl_status != 0 )); then
  printf 'curl exited with status %d\n' "$curl_status" >&2
fi
if (( curl_status != 0 || status_error != 0 )); then
  exit 1
fi

remote_ip in that record is the gateway you connected to, not the residential exit. Inspect the exit from the response body, and treat city or coordinates from an IP database as approximate diagnostic data. Country is the supported targeting boundary.

http_code is the destination response. proxy_connect_code is cURL’s last response to the proxy CONNECT request. This HTTPS-to-HTTPS probe requires a 2xx code from both. It also checks cURL’s exit status. Keep all three checks: cURL documents that --fail is not fail-safe for authentication responses such as 401 and 407.

When no HTTP response exists, cURL writes 000. The formatter converts that to JSON number 0, which means “no status received,” and the status check rejects it. It also emits post_pretransfer_wait: null unless cURL recorded a first-byte timestamp at or after time_pretransfer. The private body file contains successful response content for exit and payload validation; --fail does not retain HTTP error bodies in this recipe.

To isolate connection setup from exit rotation, give both arms the same non-sensitive sticky session. Then compare separate cURL processes with several URLs handled by one long-lived cURL process. --no-keepalive is not a control for this experiment: it disables TCP keepalive probes, not HTTP connection reuse. In the same shell as the private config setup above:

target_url='https://ipinfo.io'
reuse_session=${diagnostic_dir##*.}
case "$reuse_session" in
  (*[![:alnum:]]*|'')
    printf 'temporary directory did not provide an alphanumeric session suffix\n' >&2
    exit 1
    ;;
esac
reuse_auth_key="${PACKETSTREAM_AUTH_KEY}_session-latency${reuse_session}"
printf 'proxy = "https://proxy.packetstream.io:31111"\nproxy-user = "%s:%s"\n' \
  "$PACKETSTREAM_USER" "$reuse_auth_key" >"$curl_config"

validate_transport_records() {
  local records_file=$1
  local failed=0
  local sample new_connections total destination_code connect_code
  local destination_status proxy_connect_status

  while IFS=$'\t' read -r sample new_connections total destination_code connect_code; do
    [[ -n "$sample" ]] || continue
    destination_status=$((10#$destination_code))
    proxy_connect_status=$((10#$connect_code))

    printf 'transport-only sample=%s new_connections=%s total=%s destination_http=%03d proxy_connect=%03d\n' \
      "$sample" "$new_connections" "$total" "$destination_status" "$proxy_connect_status"

    if (( new_connections > 0 && proxy_connect_status == 0 )); then
      printf '%s: new connection has no proxy CONNECT status\n' "$sample" >&2
      failed=1
    elif (( proxy_connect_status != 0 && (proxy_connect_status < 200 || proxy_connect_status >= 300) )); then
      printf '%s: proxy CONNECT returned %03d\n' "$sample" "$proxy_connect_status" >&2
      failed=1
    fi
    if (( destination_status < 200 || destination_status >= 300 )); then
      printf '%s: destination returned HTTP %03d\n' "$sample" "$destination_status" >&2
      failed=1
    fi
  done <"$records_file"

  return "$failed"
}

for attempt in 1 2 3; do
  curl_status=0
  curl --disable --config "$curl_config" \
    --fail --silent --show-error --connect-timeout 15 --max-time 45 \
    --output /dev/null \
    --write-out "fresh-$attempt\t%{num_connects}\t%{time_total}\t%{http_code}\t%{http_connect}\n" \
    --url "$target_url" >"$transport_file" || curl_status=$?

  status_error=0
  validate_transport_records "$transport_file" || status_error=$?
  if (( curl_status != 0 || status_error != 0 )); then
    exit 1
  fi
done

curl_status=0
curl --disable --config "$curl_config" \
  --fail-early --fail --silent --show-error \
  --connect-timeout 15 --max-time 45 \
  --write-out 'shared-%{urlnum}\t%{num_connects}\t%{time_total}\t%{http_code}\t%{http_connect}\n' \
  --output /dev/null --url "$target_url" \
  --output /dev/null --url "$target_url" \
  --output /dev/null --url "$target_url" >"$transport_file" || curl_status=$?

status_error=0
validate_transport_records "$transport_file" || status_error=$?
if (( curl_status != 0 || status_error != 0 )); then
  exit 1
fi

The private temporary directory supplies a random alphanumeric suffix, so each invocation gets its own non-sensitive session label. Each short-lived process starts without a reusable connection. The sticky modifier asks PacketStream to keep those new connections on one exit. In the shared process, new_connections is normally nonzero for the first transfer and 0 when a later transfer reuses the existing transport. cURL reports proxy_connect=000 on a reused transfer because it did not send another CONNECT request. The validator accepts that pair, but it still requires 2xx when new_connections is nonzero and rejects every nonzero CONNECT status outside 2xx.

Redirects, server-directed closes, and protocol negotiation can force another connection. If the sticky exit disconnects, PacketStream fails the session; discard that comparison instead of treating a replacement path as reuse evidence.

--fail-early makes the shared cURL process return nonzero if any transfer fails; otherwise a later successful transfer could hide an earlier HTTP failure in the process exit status.

These are transport-only samples. They discard every body, so they do not prove the expected payload or observed country and must not count as valid application results. To compare useful results, save each body separately and validate its payload and country before including its timing.

A useful derived view:

dns_seconds          = namelookup
gateway_tcp_seconds  = connect - namelookup
gateway_tls_seconds  = appconnect - connect     # HTTPS proxy only
until_first_byte     = starttransfer                # cumulative from start
post_pretransfer_wait = starttransfer - pretransfer # closest cURL TTFB split
body_seconds         = total - starttransfer

Treat those as observations from this runtime, this destination, and this connection policy. They are not a universal performance promise.

What each stage is actually telling you

cURL’s timers were designed for a client talking to one host. An HTTPS proxy inserts another hop, so read them as a split of your path, not as a map of PacketStream internals.

time_namelookup is local DNS for the proxy host. A spike here is almost never the residential exit. Check the resolver in the container, a missing cache, or a blocked lookup for proxy.packetstream.io.

time_connect is TCP to the gateway. Failures or multi-second stalls before authentication usually belong to outbound firewalls, the wrong port, a VPN or corporate proxy, or a client that does not support the selected scheme. Confirm the pairing: HTTPS on 31111, SOCKS5 on 31113.

time_appconnect is TLS to the HTTPS proxy. Certificate errors belong here. Do not disable verification to make the timer look better. Confirm that the library supports an HTTPS proxy—not merely an HTTP proxy used for an HTTPS destination—and that the runtime trust store is current.

time_starttransfer is cumulative. For a typical HTTPS destination it includes the earlier setup represented by time_pretransfer plus the wait for the first response byte. Use time_starttransfer - time_pretransfer as cURL’s closest split for the post-negotiation destination wait, and retain time_starttransfer separately so the record still shows the full time from request start. Neither value maps every internal proxy hop. This is the stage people mislabel as “the proxy is slow” when the inspection URL was fast and the real catalog page is not.

time_total - time_starttransfer is body transfer. Large HTML, JSON, or browser assets dominate this number. If the first byte is prompt and the rest of the clock tracks size_download, you have a bandwidth and payload problem, not a handshake problem.

SOCKS5 on port 31113 is a different contract. time_appconnect will not mean “TLS to PacketStream,” because the client-to-proxy hop is not HTTPS. Compare SOCKS5 with HTTPS only after you have named that difference; do not use a mixed pair as a speed contest.

Run the smallest comparisons that isolate a stage

One timed request is an anecdote. Four short comparisons usually locate the stage:

1. Direct, no proxy, same destination, same runtime. This is the destination plus your local path with no gateway and no residential hop. It is a baseline, not a target PacketStream is supposed to match.
2. Proxied inspection URL such as https://ipinfo.io. This checks DNS, gateway TCP/TLS, authentication, and a small payload.
3. Proxied representative destination — the public page, feed, or API the job actually needs, with the same parser later used in production.
4. Same proxied destination in one reuse-capable cURL process, then in separate cURL processes. Use one sticky session to hold the exit constant when isolating connection setup. Without it, label the comparison fresh rotation versus shared reuse because exit-path changes can also move the total.

Keep the sample modest and report percentiles, not a single mean. A handful of requests will not describe a day of residential supply. If you need a scored experiment with a failure taxonomy and bytes per useful result, that is a benchmark, not a latency triage.

When country targeting is in play, add one more pair: the same destination with and without _country-US (or the ISO code you actually need). A longer path to a farther exit can move starttransfer without meaning the gateway is unhealthy. If the requested country has no available exit, PacketStream fails the request rather than substituting another country—record that as availability, not as latency.

Misreads that waste a day

“Mean total is 1.8s, so the network is bad.” One slow tail event can dominate a crawl that still looks fine on the average. Report p50, p95, and p99, and keep the stage split.

“Every request should be a new IP, and new IPs are slow.” New identities require new connections, and new connections repeat gateway setup. If the job does not need a fresh exit, reuse is doing useful work. If it does, budget handshake time as part of the job, not as a defect. PacketStream does not promise a distinct address on every connection.

“The sticky session is slower.” A sticky session keeps one residential exit for a related workflow. It does not make the path shorter. Continuity depends on the supplying Packeter remaining connected; a disconnect fails the session instead of silently switching identity. Time the workflow, then recover it as a workflow—not by tightening a socket timeout until it flaps.

“City from the IP database proves the hop.” Country is the targeting boundary. Sub-country geography is approximate. Do not chase a surprising city with a latency patch.

“One destination describes the product.” An inspection URL tells you the gateway, credentials, and observed exit work. It does not estimate TTFB, parser yield, or throttling for the real workload.

“More workers will hide the delay.” Extra concurrency on a handshake-bound or destination-bound job lengthens queues and retries without moving first byte. Measure the stage, then apply a concurrency plan that respects it.

Change only the stage you named

Authentication failures, deterministic parser failures, and unsupported configurations should fail immediately. Retry only work that is safe to repeat, with bounded backoff. A latency patch is not a substitute for classifying the error.

If the controlled preflight still cannot reach the gateway, collect a sanitized timing record and the checklist in the troubleshooting docs rather than adding retries.

A compact record you can keep

Store the timing next to the result, not instead of it:

{
  "collected_at": "2026-08-20T15:00:00Z",
  "runtime_region": "your-region",
  "proxy_endpoint": "https://proxy.packetstream.io:31111",
  "connection_policy": "new-connection",
  "requested_country": "US",
  "observed_country": "US",
  "http_code": 200,
  "proxy_connect_code": 200,
  "valid_result": true,
  "namelookup": 0.012,
  "connect": 0.084,
  "appconnect": 0.141,
  "pretransfer": 0.312,
  "starttransfer": 0.640,
  "post_pretransfer_wait": 0.328,
  "total": 0.712,
  "size_download": 318
}

The numbers above are a shape, not a benchmark. Omit credentials, auth-key modifiers that contain sensitive labels, and destination account data.

If valid_result is false, the timing describes a failed attempt. Do not average it into the success path.

What this method will not tell you

It will not tell you whether the residential network is “fast enough” in the abstract. It will tell you which clock moved on this job, from this runtime, through this connection policy.

It will not replace a yield metric. A short request that returns a challenge page, an empty body, or the wrong country is not a win.

It will not pick a country for you. Request the ISO country the dataset actually needs, verify it, and keep unavailable-country failures separate from slow successes.

Once the stage is named, the next document is usually one of: cURL preflight, connection pooling, retries, or a reproducible benchmark. Use the one that matches the lever you are about to pull.