Currently, sniff fails to detect valid CSV files that other qsv commands can successfully open.
This is because, sniff is powered by the qsv-sniffer crate, which is a qsv-optimized fork of the unmaintained csv-sniffer crate.
The csv-sniffer crate uses the Viterbi algorithm to sniff and infer CSV metadata and there are fairly common CSV configurations where it fails.
Describe the solution you'd like
Remove the qsv-sniffer crate altogether and just use the csv crate to get CSV metadata.
Describe alternatives you've considered
There was a discussion to create a clone of python's csv-sniffer in #1719 , but that's a non-trivial project by itself
Currently,
snifffails to detect valid CSV files that other qsv commands can successfully open.This is because,
sniffis powered by the qsv-sniffer crate, which is a qsv-optimized fork of the unmaintained csv-sniffer crate.The csv-sniffer crate uses the Viterbi algorithm to sniff and infer CSV metadata and there are fairly common CSV configurations where it fails.
Describe the solution you'd like
Remove the qsv-sniffer crate altogether and just use the csv crate to get CSV metadata.
Describe alternatives you've considered
There was a discussion to create a clone of python's csv-sniffer in #1719 , but that's a non-trivial project by itself