join: failing GNU tests

[jtracey]

Since it seems like it could be useful to have this information somewhere, here are the remaining failing GNU tests for join:

• 3c: no support for joins with a null field separator, -t "\0" (join: add support for -t '\0' #2881)
• 10b: broken comparisons with empty fields (join: fix default check order behaviour #2908)
• sp-vs-blank: \f (form feed) seems to be interpreted as a newline (join: operate on bytes instead of Strings #2851)
• 8-bit-t: 8-bit characters aren't valid unicode, cause a panic (first half: join: operate on bytes instead of Strings #2851, second half: join: add support for non-unicode field separators #2902)
• bigfield1 and bigfield2: very large integers cause parsing as a field number to fail ( join: "support" field numbers larger than usize::MAX #2882)
• invalid-j: test expects the invalid -j error to trigger before the missing args error (patched upstream, update GNU coreutils version in GnuTests workflow #2919)
• chkodr-4 and chkodr-7: GNU has relaxed order checks by default; see info join -n "General options in join" (join: fix default check order behaviour #2908)
• chkodr-5, chkodr-5b, chkodr-5c, and header3: slight differences in "not sorted" error message (join: print unsorted line in error message #2899)
• chkodr-5d and chkodr-5e: missing additional error messages (join: fix default check order behaviour #2908)
• z1, z4, z5, z6: -z flag (null-terminated lines) isn't supported (join: add support for -z option #2880)

[jfinkels]

I looked into the first item, labeled (3c). I created a file tests/fixtures/join/fields_2_null_sep.txt that is exactly like fields_2.txt but with a null character instead of a space:

$ cat fields_2_null_sep.txt | tr "\0" "N"
1Na
2Nb
3Nc
4Nd
5Ne
6Nf
7Ng
8Nh
9Ni

Here is the GNU output:

$ join tests/fixtures/join/fields_1.txt tests/fixtures/join/fields_2_null_sep.txt -t "\0"
1^@a
2^@b
3^@c
5^@e
8^@h

Here's ours:

$ ./target/debug/coreutils join tests/fixtures/join/fields_1.txt tests/fixtures/join/fields_2_null_sep.txt -t "\0"
join: multi-character tab \0

This is happening because the input argument \0 is interpreted as two bytes, a literal '\' followed by a literal '0', instead of as a single null byte. Perhaps there is a way to get the clap argument parsing library to respect the escape sequence in the input argument?

[jtracey]

I briefly looked into it when filing this, and saw something similar was accomplished in #2517

[jtracey]

I suppose I should also mention: the thing I was investigating when I last looked at this was:

GNU has --nocheck-order on by default

since it seemed like a simple fix, but it actually seems like GNU doesn't just have that flag enabled by default, they do some other weird, semi-undocumented behavior. I can't recall what it is off the top of my head right now, but IIRC it's visible from the existing GNU tests.

[jtracey]

Once all the open PRs are merged, all the GNU tests for join are passing (note that the CI won't reflect this until the workflows are updated to use the fixed invalid-j test).

[tertsdiepraam]

Excellent! Could you open a PR to update the workflow?