Correct support for avc3, hev1?

[dalecurtis]

• Apparently avc3 and hev1 are specific forms for writing the AVC and HEVC bitstreams.
• AVC1 and AVC3 don't differ much beyond the codec string, but HEV1 has a array_completeness value which should be set to 1 if parameter sets are expected to change.
• WebCodecs VideoEncoder requires explicit config changes, so no matter what we always emit AVC1/HVC1 compliant streams.

The WebCodecs codec registry currently reports support for both of these forms. Should we remove that? Should we require implementations to set array_completeness=1 for HEV1 content even if we don't allow in-band parameter set changes?

@Djuffin @sandersdan

[StaZhu]

I think if its hev1/avc3, WebCodec should also include sps/pps/vps into the EncodedVideoChunk, so I dont need to extract codec description to get these parameter sets if they are changed.

[Djuffin]

I think if its hev1/avc3, WebCodec should also include sps/pps/vps into the EncodedVideoChunk

It looks more like a muxing issue rather than an encoding issue. VideoEncoder already provides two ways to get SPS, PPS, and VPS (avc and annexb). If a container needs these parameters to be in the bitstreams, it can either

• choose annexb
• extract them from the description

[dalecurtis]

Eugene reminded me offline that we always emit a new description when the config changes. So I think it would be acceptable for us to say that we never generate avc3/hev1 and instead always generate avc1/hvc1. We can either remove the avc3/hev1 codec strings or say something in the registry that they are treated as avc1/hvc1.

[aboba]

I would remove the avc3/hev1 codec strings rather than treating them as avc1/hvc1.

[StaZhu]

My feeling is that the options we provide, namely annex-b and avc / hevc, are also matters for the muxer. In principle, WebCodecs only needs to offer the option annex-b. After all, avc / hevc can be converted from annex-b, and applications can write codec_description based on sps/pps/vps on their own.

Providing the avc / hevc option is more like an option that makes it easier to use for developers without requiring too much additional post processing.

Based on the above logic, it seems that if WebCodecs is willing to supports the hev1 and avc3 mimetypes, I would be more inclined to understand that this provides a more convenient way so that developers don't need to implement the extraction logic of SPS/PPS/VPS and modification of the description on their own.

[padenot]

Convenience was the driving factor here indeed.