Passthrough on short files fails

[cgbur]

Describe the bug
When trying to use the decompression passed through functionality on files of 0, 1, or 2 bytes the decompressor fails to read enough bytes to understand the context and quits instead of just copying the bytes to the output. This means if you have small files and are trying to use zstdcat for all of them, the very small files will fail and not be written to the output.

To Reproduce

echo | zstd -df
echo 1 | zstd -df
echo 12 | zstd -df
echo 123 | zstd -df

echo | zstdcat
echo 1 | zstdcat
echo 12 | zstdcat
echo 123 | zstdcat

output

zstd: /*stdin*\: unknown header 
zstd: /*stdin*\: unknown header 
zstd: /*stdin*\: unknown header 
123

Expected behavior
Zstd and gzip support decompression passthrough where the bytes will simply be copied to the output as-is when the following conditions are met:

1. if a file is present that is not in a known format
2. the -f flag is present
3. the output will be written to stdout

example

❯ echo hello there | zstd -df
hello there

gzip on the failure cases produces the correct output

echo | gzip -df
echo 1 | gzip -df
echo 12 | gzip -df
echo 123 | gzip -df

1
12
123

Screenshots and charts
Linux x86 - self installed from repo

[terrelln]

Thanks for the report, we'll fix this for the next release!

[terrelln]

Basically, we read 4 bytes, then if we don't see any magic numbers we recognize, we will go into pass-through mode if we are writing to stdout and -f is passed:

zstd/programs/fileio.c

Lines 2385 to 2386 in 466e13f

	} else if ((prefs->overwrite) && !strcmp (dstFileName, stdoutmark)) { /* pass-through mode */
	return FIO_passThrough(&ress);

The fix is to also go into pass-through mode if the file is less than 4 bytes and if pass-through mode is enabled (in #3211 we'll talk about changing the conditions for enabling pass-through mode, but not consider it for this issue). The relevant checks are here:

zstd/programs/fileio.c

Lines 2341 to 2352 in 466e13f

	if (ress.readCtx->srcBufferLoaded==0) {
	if (readSomething==0) { /* srcFile is empty (which is invalid) */
	DISPLAYLEVEL(1, "zstd: %s: unexpected end of file \n", srcFileName);
	return 1;
	} /* else, just reached frame boundary */
	break; /* no more input */
	}
	readSomething = 1; /* there is at least 1 byte in srcFile */
	if (ress.readCtx->srcBufferLoaded < toRead) {
	DISPLAYLEVEL(1, "zstd: %s: unknown header \n", srcFileName);
	return 1;
	}

[cgbur]

Thanks I'll take a stab at opening a PR with a fix!