How to authenticate in Django via urllib2?

Let's say you want to write a python script, using urllib2 to access some @login_required protected typical django website, with django.contrib.auth authentication and default csrf protection. In order to do this, you have to first login, and then keep the cookie with session id you received. Also, you need to keep in mind that csrf will raise a 403 if you would try to POST to the page without csrf_token in your POST data. First you have to do some initial setup that will later allow you to use cookies:

import urllib, urllib2
import cookielib

cj = cookielib.CookieJar()

opener = urllib2.build_opener(
    urllib2.HTTPCookieProcessor(cj), 
    urllib2.HTTPHandler(debuglevel=1)
)

HTTPHandler(debuglevel=1) part is actually optional, in this case it would just print more data that you might be interested in (headers mostly). Now you use your shiny new opener to open a login page and grab a csrf token from it:

from lxml import html
login_form = opener.open(login_url).read()
csrf_token = html.fromstring(login_form).xpath(
    '//input[@name="csrfmiddlewaretoken"]/@value'
)[0]

Then you create dictionary with username, password and newly grabbed token:

# make values dict
values = {
    'username': username,
    'password': password,
    'csrfmiddlewaretoken': csrf_token,
}

# Convert to params
params = urllib.urlencode(values)

Then you can login:

login_page = opener.open(login_url, params)
print login_page.read()

And check out some sites that needs authentication:

staff_page = opener.open(staff_url)
print staff_page.read()

Enjoy!