Sending JSON in Requests - cat /dev/brain

This is the first in what I hope will be a series of explorations of advanced and lesser known features in requests.

Recently, the requests team released version 2.4.3. In this version, we had a very significant new feature released that was contributed by Carol Willing. It has been requested many times over in the last couple years, and we finally decided to accept it. You can now use json as a parameter to any requests call to have the library handle sending the data as JSON for you. Previously, you had to do:

import json
import requests

r = requests.post('https://httpbin.org/post',
                  data=json.dumps({'my': 'json'}),
                  headers={'Content-Type': 'application/json'})

This is extremely annoying for especially simple calls like the one above. After Carol's pull request, all that you need to do is:

import requests

r = requests.post('https://httpbin.org/post', json={'my': 'json'})

But does it really work? YES! Let's look at that response we just received

import pprint

pprint.pprint(r.json())
# {u'args': {},
#  u'data': u'{"my": "json"}',
#  u'files': {},
#  u'form': {},
#  u'headers': {u'Accept': u'*/*',
#               u'Accept-Encoding': u'gzip, deflate',
#               u'Connect-Time': u'1',
#               u'Connection': u'close',
#               u'Content-Length': u'14',
#               u'Content-Type': u'application/json',
#               u'Host': u'httpbin.org',
#               u'Total-Route-Time': u'0',
#               u'User-Agent': u'python-requests/2.4.3 CPython/2.7.8 Darwin/13.3.0',
#               u'Via': u'1.1 vegur',
#               u'X-Request-Id': u'2688e310-4c50-4384-837e-4bc9f90471da'},
#  u'json': {u'my': u'json'},
#  u'origin': u'192.168.1.1',
#  u'url': u'https://httpbin.org/post'}

The important things to note is that it properly set the Content-Type header and that it was validly formatted JSON data.

Custom Content-Type Headers

At this point, though, you may be concerned about using this with custom Content-Type headers. Let's say you're using a service that follows JSON API and requires that you send a specifically formatted Content-Type:

s = requests.Session()
s.headers['Content-Type'] = 'application/vnd.myapi+json'

r = s.post('https://httpbin.org/post', json={'my': 'json'})
pprint.pprint(r.json())
# {u'args': {},
#  u'data': u'{"my": "json"}',
#  u'files': {},
#  u'form': {},
#  u'headers': {u'Accept': u'*/*',
#               u'Accept-Encoding': u'gzip, deflate',
#               u'Connect-Time': u'1',
#               u'Connection': u'close',
#               u'Content-Length': u'14',
#               u'Content-Type': u'application/vnd.myapi+json',
#               u'Host': u'httpbin.org',
#               u'Total-Route-Time': u'0',
#               u'User-Agent': u'python-requests/2.4.3 CPython/2.7.8 Darwin/13.3.0',
#               u'Via': u'1.1 vegur',
#               u'X-Request-Id': u'129dc6d2-ccaa-4b40-8049-e74db26d178f'},
#  u'json': {u'my': u'json'},
#  u'origin': u'72.160.200.100',
#  u'url': u'https://httpbin.org/post'}

You'll notice that requests did not overwrite the user-specified header. (The same would be true if you passed the header via the headers parameter.) This is a feature, but for users who are upgrading, it could present a way to shoot themselves in the feet.

If a user is using a Session with a Content-Type header that specifies something like application/xml, they might expect that by passing the json parameter, it would override the existing value. It will not. We do not inspect the existing header value and we will not forcibly change it for the user. In light of this, I strongly advise users to be careful that they be careful when upgrading any code to use this new feature. Double-check that everything works and then celebrate how it simplifies your code.

Cheers!