[Tag Review] - JSON Schema

[etiennenoel]

From: w3ctag/design-reviews#1093 (comment)

While JSON Schema is the de facto standard for describing available tools and expected output when dealing with language models, this would be the first time this format is used at web-platform scale. User agents and operating systems may need to implement JSON Schema support first, which could have unforeseen security implications and may create new and unforeseen attack vectors. Please consider the long-term implications of this choice. Will it work across all platforms? What additional costs will be incurred by having a dependency on JSON Schema?

Please note that JSON Schema hasn't been formally standardized and there seems to be various dialects (see https://json-schema.org/tools). How does the community plan to deal with that when standardizing?

[sushraja-msft]

Structured outputs support the 2020-12 version of JSON schema.

With regards to the stability of the spec for JSON schema
https://json-schema.org/blog/posts/stable-json-schema seems to indicate that 2020-12 is good for production use "When we last published a specification, we used the IETF's Internet-Draft Publication Process. This meant that our documents were required to retain the "draft" moniker, even though we considered them fully fledged and ready for use in production systems"

Finally, I did an LLM scan through of the library that implements structured outputs support in Chromium
https://docs.google.com/document/d/1JWsu8lemNxrFrTGdB4YK2BV1HOWbTQjckJgmcM3zIeU/edit?usp=sharing

The results need fact checking but call out the parts of JSON schema that are supported. For aspects that are not supported, we currently throw an exception.