{"id":3330,"date":"2020-02-13T20:00:55","date_gmt":"2020-02-13T20:00:55","guid":{"rendered":"https:\/\/www.askpython.com\/?p=3330"},"modified":"2020-02-13T20:04:35","modified_gmt":"2020-02-13T20:04:35","slug":"python-json-module","status":"publish","type":"post","link":"https:\/\/www.askpython.com\/python-modules\/python-json-module","title":{"rendered":"Python json Module"},"content":{"rendered":"\n

Before we dive into the Python JSON module, let’s understand what JSON is. The JSON<\/strong> (JavaScript Object Notation) is a standardized format that allows exchanging of data across the internet. <\/p>\n\n\n\n

Since this has become the standard for any information exchange through the internet, it makes sense for any Python application to send and receive data using this format.<\/p>\n\n\n\n

Python’s built-in json<\/strong> module is the interface that converts Python objects into JSON objects.<\/p>\n\n\n\n

In this tutorial, let’s take a look at some of the most commonly used methods in the json module.<\/p>\n\n\n\n

\n\n\n\n

Format of a JSON object<\/h2>\n\n\n\n

Before going into the module details, let’s understand what a JSON object consists of.<\/p>\n\n\n\n

This is actually very similar to a Python dictionary, where you have a set of {Key: value}<\/strong> pairs. The only small difference is that a JSON object has an opening and closing curly parenthesis.<\/p>\n\n\n\n

Given below is a simple example of a JSON<\/strong> object<\/p>\n\n\n

\n{\n    "name": "John",\n    "age": 42,\n    "married": True,\n    "qualifications": ["High School Diploma", "Bachelors"]\n}\n<\/pre><\/div>\n\n\n
The JSON object can consist of various attributes, including strings, integers or even lists.<\/p>\n\n\n\n
Now that we know what a JSON object is made of, let’s look at the Python json<\/strong> module’s methods.<\/p>\n\n\n\n
\n\n\n\n
Import the Python json module<\/h2>\n\n\n\n
Python already has the json<\/strong> module ready with it, so there is no need to install using pip.<\/p>\n\n\n\n
To import this module, simply type<\/p>\n\n\n
\nimport json\n<\/pre><\/div>\n\n\n
json.dumps() – Construct a JSON object<\/h2>\n\n\n\n
We can encode a Python object into a JSON object using the json.dumps()<\/code> method.<\/p>\n\n\n\n
You can think of dumps()<\/code> as serializing the Python object into a Python JSON object and returning a string. This is needed if you wish to transfer data across the internet.<\/p>\n\n\n\n
The encoded data is mentioned in the below table, for different Python objects.<\/p>\n\n\n\n
Python<\/th>JSON<\/th><\/tr><\/thead>
dict<\/strong><\/td>object<\/td><\/tr>
list<\/strong>, tuple<\/strong><\/td>array<\/td><\/tr>
str<\/strong><\/td>string<\/td><\/tr>
int<\/strong>, float<\/strong>, int<\/strong>– & float<\/strong>-derived Enums<\/td>number<\/td><\/tr>
True<\/strong><\/td>true<\/td><\/tr>
False<\/strong><\/td>false<\/td><\/tr>
None<\/strong><\/td>null<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
This takes any Python object that can be serialized as an argument and returns a string.<\/p>\n\n\n\n
Format:<\/p>\n\n\n
\njson_object = json.dumps(serializable_object)\n<\/pre><\/div>\n\n\n
Here, serializable_object<\/code> is a Python object, such as a list, string, etc, that can be serializable. It cannot be a function\/lambda, etc.<\/p>\n\n\n
\nimport json\n\npython_object = ['Hello', 'from', 'AskPython', 42]\n\njson_object = json.dumps(python_object)\n\nprint(type(json_object), json_object)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\n<class 'str'> ["Hello", "from", "AskPython", 42]\n<\/pre><\/div>\n\n\n
This method will raise a TypeError<\/code> if the object is not serializable.<\/p>\n\n\n
\n>>> import json\n>>> a = lambda x : x * 2\n>>> a(2)\n4\n>>> json.dumps(a)\nTraceback (most recent call last):\n    raise TypeError(f'Object of type {o.__class__.__name__}\nTypeError: Object of type function is not JSON serializable\n<\/pre><\/div>\n\n\n
Sorting Keys of a Dict<\/h3>\n\n\n\n
If we’re passing a Python dictionary<\/a> into json.dumps()<\/code>, we can specify another parameter sort_keys<\/code>, which will make the Python json object have sorted keys.<\/p>\n\n\n
\nimport json\n\ndict_obj = {1:"one", 20: "twenty", 5:"five"}\n\njson_obj = json.dumps(dict_obj, sort_keys = True)\n\nprint(json_obj)\n<\/pre><\/div>\n\n\n
Output<\/p>\n\n\n
\n{"1": "one", "5": "five", "20": "twenty"}\n<\/pre><\/div>\n\n\n
Our output indeed has sorted keys. <\/p>\n\n\n\n
NOTE<\/strong>: the numbers are converted into strings since it is encoded into JSON. It will be properly deserialized back into integers on using appropriate methods.<\/p>\n\n\n\n
Pretty Printing Python JSON objects<\/h3>\n\n\n\n
We can use the indent<\/code> parameter of json.dumps()<\/code> to specify the indentation level. Usually, indent = 4<\/code> will make the output look really good.<\/p>\n\n\n
\nimport json\n\ndict_obj = {1:"one", 20: "twenty", 5:"five"}\n\njson_obj = json.dumps(dict_obj, sort_keys = True, indent = 4)\n\nprint(json_obj)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\n{\n    "1": "one",\n    "5": "five",\n    "20": "twenty"\n}\n<\/pre><\/div>\n\n\n
json.dump() – Dump into a file<\/h3>\n\n\n\n
We can also dump an object into a file, if you wish to use it later, using another method json.dump()<\/code>.<\/p>\n\n\n\n
Format<\/strong>:<\/p>\n\n\n
\njson.dump(data, file_object)\n<\/pre><\/div>\n\n\n
The json.dump()<\/code> method takes in data and writes it to a file object.<\/p>\n\n\n\n
So you can open a new file and write to that file object using json.dump()<\/code><\/p>\n\n\n
\nimport json\n\npython_object = ['Hello', 'from', 'AskPython', 42]\n\nwith open("sample.json", "w") as wf:\n    json.dump(python_object, wf)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\nuser@AskPython $ cat sample.json\n["Hello", "from", "AskPython", 42]\n<\/pre><\/div>\n\n\n
As you can see, the Python object was indeed dumped to the file.<\/p>\n\n\n\n
Now, let’s take that JSON object we showed in the first example and store it into a file.<\/p>\n\n\n
\nimport json\n\njson_object = {\n    "name": "John",\n    "age": 42,\n    "married": True,\n    "qualifications": ["High School Diploma", "Bachelors"]\n}\n\nwith open("sample.json", "w") as wf:\n    json.dump(json_object, wf)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\nuser@AskPython $ cat sample.json\n{"name": "John", "age": 42, "married": true, "qualifications": ["High School Diploma", "Bachelors"]}\n<\/pre><\/div>\n\n\n
\n\n\n\n
Deserialize JSON objects<\/h2>\n\n\n\n
Similar to encoding a Python object into a JSON object, we can also do the reverse, by converting a JSON object into a Python object. This is called deserialization<\/strong>.<\/p>\n\n\n\n
We can do this using the methods json.loads()<\/code> and json.load()<\/code>, similar to json.dumps()<\/code> and json.dump()<\/code>.<\/p>\n\n\n\n
json.loads()<\/h3>\n\n\n\n
This converts a json object encoded using json.dumps()<\/code> back into a Python object.<\/p>\n\n\n
\nimport json\n\npython_object = ['Hello', 'from', 'AskPython', 42]\n\nencoded_object = json.dumps(python_object)\n\ndecoded_object = json.loads(encoded_object)\n\nprint(type(decoded_object), decoded_object)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\n<class 'list'> ['Hello', 'from', 'AskPython', 42]\n<\/pre><\/div>\n\n\n
We have successfully obtained our old list object back!<\/p>\n\n\n\n
json.load() – Deserialize from a file<\/h3>\n\n\n\n
This performs the reverse operation of json.dump()<\/code>, by converting the json object back from a file, into a Python object.<\/p>\n\n\n\n
Let’s take our sample.json<\/code> file, and get back the data using this method.<\/p>\n\n\n
\nimport json\n\nwith open("sample.json", "r") as rf:\n    decoded_data = json.load(rf)\n\nprint(decoded_data)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\n{'name': 'John', 'age': 42, 'married': True, 'qualifications': ['High School Diploma', 'Bachelors']}\n<\/pre><\/div>\n\n\n
Indeed, we have again got back our old JSON<\/em> object, that we stored in the file!<\/p>\n\n\n\n
Now that we’ve covered the most commonly used methods of this module, let’s go to the next step: Creating our own JSON Encoder!<\/p>\n\n\n\n
\n\n\n\n
Create our own JSON Encoder<\/h2>\n\n\n\n
The json<\/code> module uses an encoder called json.JSONEncoder<\/code>, which uses the rules in the table given above to encode Python objects.<\/p>\n\n\n\n
However, it does not encode all Python objects and depending on the problem we face, we may need to write our own JSON Encoder to encode those objects in a special way.<\/p>\n\n\n\n
To do that, we must write our custom Encoder Class. Let’s call it MyEncoder<\/code>. This must extend<\/a> the json.JSONEncoder<\/code> class, to add to its existing features.<\/p>\n\n\n\n
For this demonstration, we’ll take numpy arrays and convert them to Python JSON objects. Now the json module by default cannot handle numpy arrays so if you try to convert a numpy array without our extended class, you’ll get a TypeError:<\/p>\n\n\n
\nTypeError: Object of type ndarray is not JSON serializable\n<\/pre><\/div>\n\n\n
Let’s write this class to serialize and encode a numpy array into json objects as well, by converting it into a Python list, in our default()<\/code> handler method.<\/p>\n\n\n
\nimport json\nimport numpy as np\n\nclass MyEncoder(json.JSONEncoder):\n    # Handles the default behavior of\n    # the encoder when it parses an object 'obj'\n    def default(self, obj):\n        # If the object is a numpy array\n        if isinstance(obj, np.ndarray):\n            # Convert to Python List\n            return obj.tolist()\n        else:\n            # Let the base class Encoder handle the object\n            return json.JSONEncoder.default(self, obj)\n\n\n# Numpy array of floats\na = np.arange(1, 10, 0.5)\nprint(type(a), a)\n\n# Pass our encoder to json.dumps()\nb = json.dumps(a, cls=MyEncoder)\nprint(b)\n\n<\/pre><\/div>\n\n\n
We finally encode it, by passing the class name to the cls<\/code> parameter of json.dumps()<\/code>.<\/p>\n\n\n\n
So, the encoding call will be:<\/p>\n\n\n
\njson_object = json.dumps(python_object, cls=MyEncoder)\n<\/pre><\/div>\n\n\n
Output<\/strong><\/p>\n\n\n
\n<class 'numpy.ndarray'> [1.  1.5 2.  2.5 3.  3.5 4.  4.5 5.  5.5 6.  6.5 7.  7.5 8.  8.5 9.  9.5]\n[1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5]\n<\/pre><\/div>\n\n\n
Indeed, our custom encoder can now convert numpy arrays into JSON objects! We have now completed our first complex encoder.<\/p>\n\n\n\n
You can extend this functionality to write different encoders for your specific use case!<\/p>\n\n\n\n
\n\n\n\n
Conclusion<\/h2>\n\n\n\n
In this article, we learned how we can use Python’s json<\/code> module to do various operations involving JSON objects.<\/p>\n\n\n\n
\n\n\n\n
References<\/h2>\n\n\n\n