You are here because when you try to load and parse a JSON file with multiple JSON objects in Python, you received an error. json.decoder.JSONDecodeError: Extra data error. The reason is that the json.load() method can only handle a single JSON object.
Further Reading:
- Solve Python JSON Exercise to practice Python JSON skills
The file is invalid if it contains more than one JSON object. When you try to load and parse a JSON file with multiple JSON objects, each line contains valid JSON, but as a whole, it is not a valid JSON as there is no top-level list or object definition. We can call JSON a valid JSON only when there is a top-level list or object definition.
For example, you wanted to read the following JSON file, filter some data, and store it into a new JSON file.
{"id": 1, "name": "Ault", "class": 8, "email": "ault@pynative.com"}
{"id": 2, "name": "john", "class": 8, "email": "jhon@pynative.com"}
{"id": 3, "name": "josh", "class": 8, "email": "josh@pynative.com"}
{"id": 4, "name": "emma", "class": 8, "email": "emma@pynative.com"}
If your file contains a list of JSON objects, and you want to decode one object one-at-a-time, we can do it. To Load and parse a JSON file with multiple JSON objects we need to follow below steps:
- Create an empty list called
jsonList - Read the file line by line because each line contains valid JSON. i.e., read one JSON object at a time.
- Convert each JSON object into Python
dictusing ajson.loads() - Save this dictionary into a list called result jsonList.
Let’ see the example now.
Output:
Started Reading JSON file which contains multiple JSON document Printing each JSON Decoded Object 1 Ault 8 ault@pynative.com 2 john 8 jhon@pynative.com 3 josh 8 josh@pynative.com 4 emma 8 emma@pynative.com
So What Do You Think?
I want to hear from you. What do you think of this article? Or maybe I missed one of the ways to Parse multiple JSON objects from a file, Either way, let me know by leaving a comment below.
Also, try to solve the Python JSON Exercise to have a better understanding of Working with JSON Data in Python.
Thank you for your tutorial! My situation is exactly how you describe but with one addition: there is a single timestamp that tells the date the data was gathered. So in addition to those 4 data lines you have, there is one additional line in the file: { “timestamp”: “2024-12-10T13:45:00.000Z” }. I need to pass this data on and would prefer to send valid JSON.
I have two questions:
1. In your tutorial you state the file is not valid JSON, though each line is. If we wanted your example to be valid (I want to match a schema), how would the data change? An array or something?
2. How is something like this timestamp added to a schema that has this list of tuples?
Thanks again!
I have a schema and test data to match. If anyone gets here and would like to see them, hit reply and I should be notified so I can get it to you!
Hey, please help. This is my data, and it is an invalid Json, is there any way to get all the values of the data
{ "data": { "blah": "blah", "blah": "blah", "blah": "blah" }, "metadata": { "blah": "blah", "blah": "blah", "blah": "blah" } }{ "data": { "blah": "blah", "blah": "blah", "blah": "blah" }, "metadata": { "blah": "blah", "blah": "blah", "blah": "blah" } }I would love to see the code to parse this as well, several commands write json out like this for multiple objects.
How would you read the json file of it was supported by brackets and then divide it into smaller json file .
How would you achieve this if we have multiple json separated by space not by line.
Thanks a lot, it saved my time..
Hi.. It worked for Me. Thanks for sharing
Hey how would I process if my json is in below format:
[{"id": 1, "name": "Ault", "class": 8, "email": "ault@pynative.com"} {"id": 2, "name": "john", "class": 8, "email": "jhon@pynative.com"}] [{"id": 3, "name": "josh", "class": 8, "email": "josh@pynative.com"} {"id": 4, "name": "emma", "class": 8, "email": "emma@pynative.com"}]It is like multiple lists and inside there are multiple dictionaries
This is in JSON format . kindly convert it to object dictionary with loads
hi this code worked for me and it helped
thank you
I want to know how you can convert multiple jason file to single jason file and insert in dynamo db.
Jason files picked from s3 bucket.
if our data is in below format:
{ "id": 1, "name": "Ault", "class": 8, "email": "ault@pynative.com" } { "id": 2, "name": "john", "class": 8, "email": "jhon@pynative.com" }it throws errors. Can anyone put light on this why is it so?
Useful method – thanks for your post
i have a doubt in my code regarding getting multiple inputs from the user and while storing i got an error
hello, thanks for your tutorial. After applying the code, i got the following error:
JSONDecodeError: Extra data: line 1 column 806 (char 805)
What do you advise?
Thanks and best regards,
hey, Can you please provide me your code.
Great post! Helped a lot. Thank you : )
I’m glad it helped you, Manthan Admane.
Reading a .json file to create a data frame is difficult. There might be syntax or version issues but whatever solutions different experts have suggested, none of them works. Shall appreciate highly if you give an example of reading a multiobject .json file for creating a dataframe.
Thanks for this explanation, I am currently facing similar problem, but it’s more difficult because the file has over 1 million lines of json object. How do I do these without using memory?
You have multiple objects on ONE file? OR one OBJECT that needs a million lines to store? Big difference…
If it’s the former, you can piecemeal it and likely only load the objects you need (1 at a time)… Or better yet, put them in to separate files in a special folder. Look through items in that folder. Boom no memory issues.
If it’s the latter, then yeah you are right. Loading that much into memory, not the best idea. You need to see if you really need 1 million lines of data for ONE OBJECT.. You can always write your own serialize() and deserialize() functions, so you only write the data you need to a file.
Hi, I have a single JSON file with multiple objects, but each object is split over multiple lines (so it is not like in the example above, where one object is in one line). It looks like this:
Any advice on how to load this into Python, or at least on how to split this file into individual-object files?