JsonMend is a robust Ruby gem designed to repair broken or malformed JSON strings. It is specifically optimized to handle common errors found in JSON generated by Large Language Models (LLMs), such as missing quotes, trailing commas, unescaped characters, and stray comments
Integrating Large Language Models (LLMs) into software workflows often requires structured data output. While prompting an LLM to "return JSON" is a common pattern, models are probabilistic text generators, not strict serialization engines. They frequently treat JSON syntax as a loose suggestion rather than a rigid standard.
Standard JSON.parse is fragile when facing the chaotic output of an LLM. Common failure modes include:
- Hallucinated Syntax: LLMs often include trailing commas, code comments (
//or#), single quotes, or Python-style literals (True,None) that break standard JSON parsers - "Chatty" Wrappers: Models frequently wrap JSON in Markdown code blocks (
json ...) or include conversational preambles (Here is the data you requested: ...), turning valid data into invalid syntax errors - Truncation: JSON is verbose. Output limits often cut off the response mid-stream, leaving unclosed brackets and braces
JsonMend acts as a middleware layer between the messy text output of an LLM and your Ruby application. It aggressively parses, cleans, and repairs the raw string—handling truncation, stripping garbage text, and normalizing syntax—to ensure you get usable structured data instead of a JSON::ParserError
- Lenient Parsing: Handles single quotes, slanted quotes (
“...”), and unquoted keys or values - LLM Error Correction: Fixes common LLM "hallucinations" like truncated JSON, missing closing brackets/braces, and
...placeholders - Comment Stripping: Automatically removes JavaScript-style (
//,/* */) and Ruby/Shell-style (#) comments - Flexible Output: Can return either a repaired JSON string or a native Ruby Hash/Array
- Heuristic Repair: Includes advanced logic to merge dangling arrays and handle duplicate keys by splitting objects
Add this line to your application's Gemfile:
gem 'json_mend'And then execute:
$ bundle installOr install it directly:
$ gem install json_mendThe primary method is JsonMend.repair. By default, it returns a valid JSON string
require 'json_mend'
# Missing closing brace and unquoted key
broken_json = '{"name": "John", "age": 30, city: "New York"'
JsonMend.repair(broken_json)
# => '{"name":"John","age":30,"city":"New York"}'
bad_json = 'Here is the data: ```json {"id": 1} ``` check it out'
JsonMend.repair(bad_json)
# => '{"id":1}'
bad_json = '{"part": 1} {"part": 2}'
JsonMend.repair(bad_json)
# => '{"part":2}'If you want to work with the data immediately, pass return_objects: true to get a Hash or Array
result = JsonMend.repair('{"items": [1, 2, 3,]}', return_objects: true)
# => {"items" => [1, 2, 3]}After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment
To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release
The gem is available as open source under the terms of the MIT License