Skip to content

Implemented the binary wire format#2394

Merged
fulghum merged 1 commit into
mainfrom
daylon/binary-wire-tests
Mar 2, 2026
Merged

Implemented the binary wire format#2394
fulghum merged 1 commit into
mainfrom
daylon/binary-wire-tests

Conversation

@Hydrocharged

@Hydrocharged Hydrocharged commented Feb 27, 2026

Copy link
Copy Markdown
Collaborator

Fixes:

In a nutshell, we did not respect a client's wishes when they requested that the results for some columns are returned in their binary format (obtained from their send function). Instead, we had hardcoded that some types, in certain circumstances, would only return their binary format, while others would strictly return their text format. This PR implements the binary wire format for every type that is currently supported. This also fixes numerous bugs that were hidden due to our incorrect behavior.

In addition to the above fixes, this PR introduces a new test harness for analyzing our compatibility with the wire messages returned directly to Postgres clients. Thousands of lines of new tests have been added that ensure we're returning the exact correct results for a given set of messages (not just queries but the actual client messages), and the test harness allows for quick iteration when comparing what a Postgres server sends versus a Doltgres server, so we can ensure that we're returning the same bytes, not just the same overall structure.

@Hydrocharged Hydrocharged force-pushed the daylon/binary-wire-tests branch 3 times, most recently from f0d13b2 to e52522b Compare March 2, 2026 09:12
@Hydrocharged Hydrocharged requested a review from fulghum March 2, 2026 09:12
@Hydrocharged Hydrocharged marked this pull request as ready for review March 2, 2026 09:13
@Hydrocharged Hydrocharged requested a review from tbantle22 as a code owner March 2, 2026 09:13
@Hydrocharged Hydrocharged force-pushed the daylon/binary-wire-tests branch from e52522b to aeaf9c4 Compare March 2, 2026 09:14
@Hydrocharged Hydrocharged removed the request for review from tbantle22 March 2, 2026 09:14
@dolthub dolthub deleted a comment from github-actions Bot Mar 2, 2026
@dolthub dolthub deleted a comment from github-actions Bot Mar 2, 2026
@github-actions

github-actions Bot commented Mar 2, 2026

Copy link
Copy Markdown
Contributor
Main PR
covering_index_scan_postgres 554.61/s 586.65/s +5.7%
index_join_postgres 120.65/s 124.32/s +3.0%
index_join_scan_postgres 176.22/s 182.69/s +3.6%
index_scan_postgres 11.15/s 11.06/s -0.9%
oltp_point_select 2356.77/s 2367.29/s +0.4%
oltp_read_only 1679.19/s 1693.84/s +0.8%
select_random_points 110.44/s 112.02/s +1.4%
select_random_ranges 434.23/s 443.97/s +2.2%
table_scan_postgres 10.87/s 10.87/s 0.0%
types_table_scan_postgres 4.93/s 5.00/s +1.4%

@github-actions

github-actions Bot commented Mar 2, 2026

Copy link
Copy Markdown
Contributor
Main PR
Total 42090 42090
Successful 17648 17651
Failures 24442 24439
Partial Successes1 5575 5575
Main PR
Successful 41.9292% 41.9363%
Failures 58.0708% 58.0637%

${\color{lightgreen}Progressions (3)}$

psql

QUERY: SELECT 1 AS one ; SELECT warn('1.5') ; SELECT 2 AS two ;
QUERY: SELECT 3 AS three ; SELECT warn('3.5') ; SELECT 4 AS four 
QUERY: SELECT 1 AS one ; SELECT warn('1.5') ; SELECT 2 AS two ;

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@fulghum fulghum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Very cool that we can easily run the new test harness against a Postgres server and confirm that our expected wire messages are correct.

@fulghum

fulghum commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

A customer pinged us about needing this change, so I'm going to go ahead and merge it in and get a release out for them.

@fulghum fulghum merged commit a95de96 into main Mar 2, 2026
17 checks passed
@fulghum fulghum deleted the daylon/binary-wire-tests branch March 2, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment