Use a proxy with Python requests, httpx and aiohttp

netdash works with any Python HTTP client that accepts an HTTP proxy URL, so there is no SDK to install. This page is the Python requests proxy reference, with the equivalent setup for httpx and aiohttp: authentication, sticky sessions, exit countries and the errors you are likely to meet. The examples run on Python 3.9 and later and use the Scale plan on the Americas gateway, scale-us.netdash.io:2000. Other plans use the same code with a different host and port, listed in the proxy format reference.

Python requests proxy quick start

Copy your username and password from the dashboard (Residential, then Generate), replace USERNAME and PASSWORD, and run:

import requests

proxy = "http://USERNAME-cc-US:PASSWORD@scale-us.netdash.io:2000"
proxies = {"http": proxy, "https": proxy}  # set both, or HTTPS requests bypass the proxy

response = requests.get("https://api.ipify.org?format=json", proxies=proxies, timeout=30)
response.raise_for_status()
print(response.text)

{"ip":"203.0.113.10"}

The script prints the exit IP the website saw, for example {"ip":"203.0.113.10"}.

Set both keys. requests chooses a proxy by the scheme of the URL you request, so with only an http entry, requests to https:// URLs bypass the proxy and leave from your own IP. The value of both keys starts with http://: netdash is an HTTP proxy, and HTTPS websites are reached through a CONNECT tunnel, so TLS still runs end to end between your client and the website.

Proxy authentication in requests

requests reads proxy credentials from the userinfo part of the proxy URL (http://user:password@host:port) and sends them to the gateway as Basic proxy authentication. Your netdash username does two jobs: it identifies your account and carries the connection options, -cc-US for the exit country and, for sticky sessions, -session-{ID}-ttl-{MINUTES}. One username and password work on every residential and mobile plan.

In real code, read the credentials from the environment and build the URL in one place. Save this helper as netdash_proxy.py; the rest of this page imports it.

import os
import secrets
from typing import Optional

NETDASH_USERNAME = os.environ["NETDASH_USERNAME"]
NETDASH_PASSWORD = os.environ["NETDASH_PASSWORD"]


def proxy_url(country: str = "US", session_id: Optional[str] = None,
              ttl_minutes: int = 30, gateway: str = "scale-us.netdash.io:2000") -> str:
    """netdash proxy URL: rotating without session_id, sticky with one."""
    cc = country.upper()
    cc = "GB" if cc == "UK" else cc
    username = f"{NETDASH_USERNAME}-cc-{cc}"
    if session_id is not None:
        username += f"-session-{session_id}-ttl-{ttl_minutes}"
    return f"http://{username}:{NETDASH_PASSWORD}@{gateway}"


def new_session_id() -> str:
    return secrets.token_hex(4)  # 8 characters, letters and digits

Export the two variables in the shell that runs your script, or load them from a .env file that stays out of version control. If a value ever contains characters such as @, : or /, percent-encode it with urllib.parse.quote(value, safe="") before it goes into the URL.

Keep one exit IP for a task

Some tasks need several requests from the same exit IP, for example a page followed by its pagination or a multi-step form you are permitted to use. Add a session ID and a lifetime to the username: reusing the same session ID keeps the same exit IP for the TTL while that IP stays available, and a new session ID starts a new session. The dashboard presets are 15 minutes, 30 minutes, 12 hours and 7 days; rotating and sticky sessions explains how to choose.

A requests.Session keeps cookies and reuses connections, which suits this kind of task. Pass proxies= on each request rather than setting session.proxies: the requests documentation warns that proxies from the environment overwrite Session.proxies.

import requests

from netdash_proxy import new_session_id, proxy_url

sticky = proxy_url("US", session_id=new_session_id(), ttl_minutes=30)
proxies = {"http": sticky, "https": sticky}

with requests.Session() as session:
    for _ in range(2):
        response = session.get("https://api.ipify.org?format=json", proxies=proxies, timeout=30)
        response.raise_for_status()
        print(response.json()["ip"])  # the same exit IP both times

Session IDs use letters and digits only, without hyphens. secrets.token_hex(4) returns 8 characters from 0-9 and a-f, the same length the dashboard generates.

Rotating proxies in Python

A rotating session is the same username without the session part. You don't need a list of proxies or any rotation logic in your code: the gateway assigns exit IPs for you. Requests sent through one HTTPS tunnel share that tunnel's exit IP, so if consecutive requests must share an exit IP, use a sticky session instead of relying on connection reuse.

This loop fetches a short list of pages at a steady rate. When a site answers 429 Too Many Requests, it waits for the Retry-After delay and tries at most twice more; at 403 it stops, instead of retrying until something gets through.

import time

import requests

from netdash_proxy import proxy_url

URLS = [
    "https://example.com/catalog?page=1",
    "https://example.com/catalog?page=2",
    "https://example.com/catalog?page=3",
]
rotating = proxy_url("US")
proxies = {"http": rotating, "https": rotating}


def fetch(url: str) -> requests.Response:
    response = requests.get(url, proxies=proxies, timeout=30)
    for _ in range(2):  # at most two more attempts, only after a 429
        if response.status_code != 429:
            break
        retry_after = response.headers.get("Retry-After", "")
        time.sleep(int(retry_after) if retry_after.isdigit() else 60)
        response = requests.get(url, proxies=proxies, timeout=30)
    return response


for url in URLS:
    response = fetch(url)
    if response.status_code == 403:
        print(f"403 from {url}: stopping. Check the site's terms or use its official API.")
        break
    response.raise_for_status()
    print(url, response.status_code, len(response.text))
    time.sleep(1)  # one request per second to this site

Choose the exit country

The exit country is the -cc- part of the username: an ISO 3166-1 alpha-2 code in uppercase. Use GB for the United Kingdom, not UK (the helper maps it for you). Use the -us gateway for exits in the Americas and the -eu gateway for exits in Europe. Targeting is country-level; see the country targeting syntax for details, and contact support to confirm availability for a specific country.

import requests

from netdash_proxy import proxy_url

for country in ("DE", "GB"):
    proxy = proxy_url(country, gateway="scale-eu.netdash.io:2000")
    response = requests.get("https://api.ipify.org?format=json",
                            proxies={"http": proxy, "https": proxy}, timeout=30)
    response.raise_for_status()
    print(country, response.json()["ip"])

httpx: sync and async clients

httpx takes a single proxy URL with proxy= and uses it for both http:// and https:// requests. The credentials go in the URL, as with requests. The older proxies= argument was deprecated in httpx 0.26 and removed in 0.28, so code that still passes it fails with a TypeError; use proxy=, or mounts= if you need a different proxy per scheme.

For concurrent requests, use AsyncClient and cap the number of requests in flight. netdash sets no per-account concurrency limit, so the limit you set per website is the one that counts.

import httpx

from netdash_proxy import proxy_url

with httpx.Client(proxy=proxy_url("US"), timeout=30) as client:
    response = client.get("https://api.ipify.org?format=json")
    response.raise_for_status()
    print(response.json())

import asyncio

import httpx

from netdash_proxy import proxy_url

URLS = ["https://api.ipify.org?format=json"] * 10


async def fetch(client: httpx.AsyncClient, limit: asyncio.Semaphore, url: str) -> str:
    async with limit:
        response = await client.get(url)
        response.raise_for_status()
        return response.json()["ip"]


async def main() -> None:
    limit = asyncio.Semaphore(5)  # at most 5 requests in flight to this site
    async with httpx.AsyncClient(proxy=proxy_url("US"), timeout=30) as client:
        ips = await asyncio.gather(*(fetch(client, limit, url) for url in URLS))
    print(ips)


asyncio.run(main())

aiohttp

aiohttp takes the proxy per request with proxy=, and the credentials can sit in the proxy URL, so the same helper works. HTTPS websites go through a CONNECT tunnel. Unlike requests and httpx, aiohttp ignores proxy environment variables unless you create the session with trust_env=True.

import asyncio

import aiohttp

from netdash_proxy import proxy_url


async def main() -> None:
    timeout = aiohttp.ClientTimeout(total=30)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        async with session.get("https://api.ipify.org?format=json", proxy=proxy_url("US")) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Environment variables, and why to avoid them

requests and httpx also read HTTP_PROXY, HTTPS_PROXY and NO_PROXY from the environment, so this works without changing any code:

export HTTP_PROXY="http://USERNAME-cc-US:PASSWORD@scale-us.netdash.io:2000"
export HTTPS_PROXY="http://USERNAME-cc-US:PASSWORD@scale-us.netdash.io:2000"
export NO_PROXY="localhost,127.0.0.1"
python3 app.py

import requests

print(requests.get("https://api.ipify.org?format=json", timeout=30).text)

A global HTTPS_PROXY applies to every request the process makes, including calls to model APIs, package registries and your own services. Those requests would go through netdash and count against your balance. Set the proxy on the client that fetches websites instead, and if a library picks up proxy variables you set for something else, turn that off with session.trust_env = False (requests) or trust_env=False (httpx).

Errors and fixes

Next steps

For a complete scraper that fetches, parses, stores and honors robots.txt, follow the complete scraper tutorial. Writing JavaScript as well? See Node.js proxy setup, or browse all proxy integration guides. For first-time setup, start with the quickstart.

• Scrapy proxy setup

  Request meta or a downloader middleware, with the same proxy_url() helper and polite crawl settings.

• Rotating and sticky sessions

  Session IDs, TTL presets and when to rotate instead.

• Web data collection

  Scheduled collection of public pages by country, at a rate each site can handle.

The same client setup runs scheduled rank checks for SEO monitoring.

Stuck? See troubleshooting or email support with your username (without the password) and the error message.