Description
When using the extended query protocol (binary format), DoltgreSQL sends an empty (0-byte) binary payload for NUMERIC/DECIMAL column values. PostgreSQL's binary format for NUMERIC requires at least 2 bytes for the ndigits header (a u16), so any typed client that implements the standard PostgreSQL binary NUMERIC decoder will panic or error when it tries to read the header from an empty buffer.
Reproduction
Using Rust with sqlx 0.8 and rust_decimal::Decimal:
#[derive(sqlx::FromRow)]
struct Example {
amount: Option<rust_decimal::Decimal>, // NUMERIC column
}
let row = sqlx::query_as::<_, Example>("SELECT amount FROM my_table")
.fetch_one(&pool)
.await?;
This panics with:
thread 'tokio-runtime-worker' panicked at bytes-1.11.1/src/lib.rs:170:5:
advance out of bounds: the len is 0 but advancing by 2
The panic originates in the bytes crate because sqlx's NUMERIC binary decoder calls buf.advance(2) to read the ndigits u16 header, but the buffer is empty.
Expected behavior
DoltgreSQL should send NUMERIC values using the standard PostgreSQL binary format:
- 2 bytes:
ndigits (number of digit groups)
- 2 bytes:
weight
- 2 bytes:
sign
- 2 bytes:
dscale
ndigits * 2 bytes: digit data
Or, if the value is NULL, it should be indicated at the protocol level (not as an empty non-NULL payload).
Workaround
We've implemented a DoltDecimal wrapper that forces text-mode decoding instead of binary, similar to our workarounds for ENUM binary framing issues:
pub struct DoltDecimal(pub rust_decimal::Decimal);
impl<'r> sqlx::Decode<'r, sqlx::Postgres> for DoltDecimal {
fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
let text = <&str as sqlx::Decode<sqlx::Postgres>>::decode(value)?;
let d = text.parse::<rust_decimal::Decimal>()?;
Ok(DoltDecimal(d))
}
}
Environment
- DoltgreSQL: latest (
dolthub/doltgresql:latest Docker image)
- Client: Rust sqlx 0.8 with
PgConnectOptions and statement_cache_capacity(0)
- Affected columns: Any
NUMERIC/DECIMAL column read via the extended query protocol
Description
When using the extended query protocol (binary format), DoltgreSQL sends an empty (0-byte) binary payload for
NUMERIC/DECIMALcolumn values. PostgreSQL's binary format for NUMERIC requires at least 2 bytes for thendigitsheader (au16), so any typed client that implements the standard PostgreSQL binary NUMERIC decoder will panic or error when it tries to read the header from an empty buffer.Reproduction
Using Rust with sqlx 0.8 and
rust_decimal::Decimal:This panics with:
The panic originates in the
bytescrate because sqlx's NUMERIC binary decoder callsbuf.advance(2)to read thendigitsu16 header, but the buffer is empty.Expected behavior
DoltgreSQL should send NUMERIC values using the standard PostgreSQL binary format:
ndigits(number of digit groups)weightsigndscalendigits * 2bytes: digit dataOr, if the value is NULL, it should be indicated at the protocol level (not as an empty non-NULL payload).
Workaround
We've implemented a
DoltDecimalwrapper that forces text-mode decoding instead of binary, similar to our workarounds for ENUM binary framing issues:Environment
dolthub/doltgresql:latestDocker image)PgConnectOptionsandstatement_cache_capacity(0)NUMERIC/DECIMALcolumn read via the extended query protocol