Residential proxies for price monitoring

Check a product page from one country

A price is only useful if you know where it was seen. This Python example opens a sticky session in the target country. It checks the exit country first, then loads the product page through the same exit, and records both:

import os
import secrets
from urllib.parse import quote

import requests

USER = quote(os.environ["PACKETSTREAM_USER"], safe="")
AUTH_KEY = os.environ["PACKETSTREAM_AUTH_KEY"]


def check_price_page(url, country):
    # A new label per check: these requests share one exit.
    label = secrets.token_hex(6)
    password = quote(f"{AUTH_KEY}_country-{country}_session-{label}", safe="")
    proxy = f"https://{USER}:{password}@proxy.packetstream.io:31111"
    proxies = {"http": proxy, "https": proxy}

    exit_info = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=30).json()
    if exit_info.get("country") != country:
        raise RuntimeError(f"wanted {country}, got {exit_info.get('country')}")

    page = requests.get(url, proxies=proxies, timeout=30)
    return {
        "url": url,
        "requested_country": country,
        "observed_country": exit_info["country"],
        "status": page.status_code,
        "html": page.text,
    }


for country in ("US", "DE", "FR"):
    result = check_price_page("https://shop.example/product/123", country)
    print(country, result["status"], len(result["html"]))

Use uppercase two-letter country codes. A code we don’t recognize is ignored, and the request can then leave from any country, which is why the example checks. To keep a valid list, pick the country in the Location menu on the dashboard’s Proxy Access page. See country targeting.

Pull Google Shopping offers instead

If you track products across many shops, Google Shopping may already have the data. One Search API call returns up to 100 offers:

curl -s https://api.packetstream.io/v1/search \
  -H "Authorization: Bearer $PACKETSTREAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "noise cancelling headphones", "type": "shopping", "country": "de", "language": "de"}'

Each offer has a title, the price as Google shows it and the merchant in source. When available, it also has extracted_price, old_price, rating, reviews and delivery. Create the API key on the dashboard’s Search API page. See the Search API docs and the response fields.

Get the price right

• Store the country with the price. Keep the requested and observed country next to every price.
• Watch the page, not just the IP. Shops can also change the price by language, cookies, currency setting, delivery address or login. A country exit does not change those.
• Check before you alert. A price that drops to zero is usually a parsing error or an “out of stock” page. Check suspicious changes before you act on them.
• Go slow. Check each product only as often as you need, and follow each shop’s terms.

Our article Residential proxies for price monitoring covers product matching, normalizing and validation in more depth.

Honest limits

• Country only. You can’t target a city or region. Shops that price by postal code need that code in your request.
• Shops can still block you. Residential exits don’t guarantee access, and we don’t solve CAPTCHAs.
• Shopping results come from Google. They show what Google lists, which can lag behind the shop’s own page.
• You pay for all traffic. Error pages and retries count toward your GB.