ArrayIndexOutOfBoundsException in partial updates after ADD COLUMN

[fresh-borzoni]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

0.9.0 (latest release)

Please describe the bug 🐞

After ADD COLUMN, partial updates on existing rows crash with ArrayIndexOutOfBoundsException. ThePartialUpdater is created with the latest schema (4 fields) but reads old rows from RocksDB that were stored with the previous schema (3 fields). At field index 3, GenericRow.fields[3] throws on a 3-element array.

Reproduce: insert a row, ADD COLUMN, partial update the same row - crash.

Solution

Use column IDs to align rows to the latest schema before the merger sees them

Are you willing to submit a PR?

• I'm willing to submit a PR!

[wuchong]

See the comment #2594 (review)

[wuchong]

@fresh-borzoni , could you also open a pull request against release-0.9 branch? I think this is also a critical issue for 0.9 version and should be included in 0.9.1.

[luoyuxia]

Close since we already pick to 0.9