fix(pgwire): fix memory corruption from malformed wire messages

[bluestreak01]

Summary

Several code paths in io.questdb.cutlass.pgwire trusted length and dimension fields read from the wire without validating them. In the pre-auth handshake this allowed a network attacker to drive pointer arithmetic to positions outside the receive buffer; in the post-auth extended query protocol it allowed an authenticated client to wrap the flat length of an array to a value smaller than the shape implied.

Pre-authentication

• PGCleartextPasswordAuthenticator.processInitMessage accepted any signed int as msgLen. Negative values passed the msgLen > availableToRead check and flowed into processStartupMessage, where msgLimit = recvBufStart + msgLen pointed before the receive buffer. recvBufReadPos = msgLimit then moved the read cursor backwards, and compactRecvBuf invoked Vect.memmove(recvBufStart, recvBufReadPos, recvBufWritePos - recvBufReadPos) with a source pointer outside the allocated buffer.
• PGCleartextPasswordAuthenticator.processPasswordMessage had the same class of issue with its msgLen.
• Both handlers now require msgLen to sit between the protocol minimum and the capacity of the receive buffer; out-of-range values terminate the connection via PGMessageProcessingException.
• DefaultPGCircuitBreakerRegistry.cancel compared the index with < instead of <=, so a CancelRequest with pid == circuitBreakers.size() indexed one slot past the logical end of the ObjList. The method also read circuitBreakers.getQuick(idx) and cb.getSecret() outside the registry spin lock, while add/remove/close all hold it. The cancel path now runs the whole lookup, secret check and cb.cancel() under the same lock. It additionally rejects the -1 sentinel that clear() installs, closing the window between clear() and the next init() during which the slot held no real secret.

Post-authentication

• PGNonNullBinaryArrayView.addDimLen and PGNonNullVarcharArrayView.addDimLen computed flatViewLength *= dimLen on signed ints without overflow checking. Dimensions such as [65537, 65537] wrapped flatViewLength to 131073 while shape still claimed ~4.3B elements. setPtrAndCalculateStrides then validated the on-wire byte count against the wrapped length and passed. Both methods now use Math.multiplyExact.
• PGPipelineEntry.setBindVariableAsArray and setBindVariableAsVarcharArray did not check that valueSize stayed non-negative as it was decremented, and did not reject negative dimensions or dimensionSize. They now validate valueSize before each field read, reject non-positive dimension counts and negative dimension sizes, and translate ArithmeticException from addDimLen into a PGMessageProcessingException.

Tradeoffs

• Well-formed clients already send lengths within the validated ranges, so there is no observable behavior change for legitimate traffic.
• The pre-auth checks add two comparisons on the startup/password paths; those paths are not performance-sensitive.
• The per-dimension checks in setBindVariableAsArray run once per dimension per Bind. Arrays typically have few dimensions, and this is not on a hot per-row path.
• Moving the cancel path under the registry spin lock means cb.cancel() now runs while the lock is held. close() already calls cb.cancel() under the same lock, so the contention pattern is unchanged.
• The -1 sentinel rejection in cancel is conservative: a real random secret equal to -1 (1 chance in 2^32) would be rejected. That is an acceptable cost to eliminate the clear/init window.

Test plan

• mvn -pl core -Dtest='DoubleArrayParserTest,VarcharArrayParserTest,PGSecurityTest' test
• mvn -pl core -Dtest='PGArraysTest,PGVarcharArrayBindVariablesTest' test
• mvn -pl core -Dtest='PGJobContextTest' test
• mvn -pl core test (full suite)
• Manual: connect via pgjdbc and psql to confirm normal auth and query flow

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e17b5305-5756-4964-b592-87607ff12d28

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch vi_pg_fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[RaphDal]

The fixes are looking good, tests should be made understandable

[mtopolnik]

[PR Coverage check]

😍 pass : 42 / 49 (85.71%)

file detail

	path	covered line	new line	coverage
🔵	io/questdb/cutlass/pgwire/PGNonNullBinaryArrayView.java	4	6	66.67%
🔵	io/questdb/cutlass/pgwire/PGPipelineEntry.java	18	23	78.26%
🔵	io/questdb/cutlass/pgwire/PGCleartextPasswordAuthenticator.java	6	6	100.00%
🔵	io/questdb/cutlass/pgwire/DefaultPGCircuitBreakerRegistry.java	12	12	100.00%
🔵	io/questdb/cutlass/pgwire/PGNonNullVarcharArrayView.java	2	2	100.00%