Proxies with Python “Requests” module


Just a short, simple one about the excellent Requests module for Python.

I can"t seem to find in the documentation what the variable "proxies" should contain. When I send it a dict with a standard "IP:PORT" value it rejected it asking for 2 values. So, I guess (because this doesn"t seem to be covered in the docs) that the first value is the ip and the second the port?

The docs mention this only:

proxies – (optional) Dictionary mapping protocol to the URL of the proxy.

So I tried this... what should I be doing?

proxy = { ip: port}

and should I convert these to some type before putting them in the dict?

r = requests.get(url,headers=headers,proxies=proxy)

Answer rating: 332

The proxies" dict syntax is {"protocol":"ip:port", ...}. With it you can specify different (or the same) proxie(s) for requests using http, https, and ftp protocols:

http_proxy  = ""
https_proxy = ""
ftp_proxy   = ""

proxyDict = { 
              "http"  : http_proxy, 
              "https" : https_proxy, 
              "ftp"   : ftp_proxy

r = requests.get(url, headers=headers, proxies=proxyDict)

Deduced from the requests documentation:

method – method for the new Request object.
url – URL for the new Request object.
proxies – (optional) Dictionary mapping protocol to the URL of the proxy.

On linux you can also do this via the HTTP_PROXY, HTTPS_PROXY, and FTP_PROXY environment variables:

export HTTP_PROXY=
export FTP_PROXY=

On Windows:

set http_proxy=
set https_proxy=
set ftp_proxy=

Thanks, Jay for pointing this out:
The syntax changed with requests 2.0.0.
You"ll need to add a schema to the url:

Get Solution for free from DataCamp guru