fix(index): prune stale label list entries during updates

[terapyon]

Summary

With stable row IDs enabled, a LABEL_LIST index can match an updated row against its previous label after index optimization, even though the stored value is correct. This change applies old_data_filter to existing label postings and list-level null rows before merging replacement data.

Original scenario and impact

This was found while evaluating stable row IDs for _row_last_updated_at_version in an embedded LanceDB application that uses a list<string> column for access-control labels. The reporting application's production tables do not enable stable row IDs; the reproductions use synthetic data.

In the original 50-row example, changing one row's acl from ["Anonymous"] to ["Manager"] correctly reduced the results for array_has_any(acl, ['Anonymous']) from 50 to 49. After table.optimize(), the count returned to 50, including the updated row with its current ["Manager"] value. This occurred with both update and merge_insert, and with both filtered scans and vector searches using prefiltering.

Applications that rely on this predicate to enforce access restrictions, without independently checking the current permissions, could therefore return a row after its previous access label has been revoked. This is a query-correctness defect with potential application-level authorization impact. The reproductions do not establish a bypass of built-in authentication or a production disclosure.

The behavior was reproduced with LanceDB 0.37.1 and 0.38.0, and with source-built Lance at 2602724cf6256ff8f55571805fdc5b0614d706b8.

Minimal reproduction

The problem reduces to two rows, one list column, one LABEL_LIST index, and an update to one row. A BTREE index, vector search, and file compaction are not required; optimize_indices() alone reproduces it.

import lance
import pyarrow as pa


def test_old_label_is_removed(tmp_path):
    ds = lance.write_dataset(
        pa.table({"labels": [["old"], ["keep"]]}),
        tmp_path,
        enable_stable_row_ids=True,
    )
    ds.create_scalar_index("labels", "LABEL_LIST")
    old = "array_has_any(labels, ['old'])"
    ds.update({"labels": "['new']"}, where=old)
    assert ds.to_table(filter=old).num_rows == 0

    ds.optimize.optimize_indices()

    assert ds.to_table(filter=old, use_scalar_index=False).num_rows == 0
    assert ds.to_table(filter=old).num_rows == 0

Before the fix, the final assertion fails with 1 == 0: the indexed query returns the row containing ["new"], while the forced scan correctly returns no rows. The corresponding test with stable row IDs disabled passes.

Cause and fix

LabelListIndex::update received but discarded old_data_filter, allowing obsolete memberships to survive index updates. Pass that filter through to the existing bitmap builder and also apply it to the old list_nulls bitmap. Prune old state before adding replacements, since replacement rows may reuse the same stable row IDs.

This addresses the existing correctness limitation documented in #8840. It preserves the streaming build approach and does not change the public API or index file format.

The Python regression covers stable row IDs enabled and disabled across two fragments. The Rust regression covers row-ID and fragment filters, null elements, null lists, empty lists, and preservation of unchanged rows.

Validation

• Python LabelList tests: 11 passed.
• Rust LabelList tests: 18 passed.
• Additional local investigation scripts: all 32 comparison cases and all eight original Lance probe configurations passed with the patched Lance build.
• Rust formatting, Ruff checks for the changed Python file, and commit hooks passed.

The full workspace test suite, workspace-wide Clippy, and full Python lint target have not been run. Post-fix validation used the locally built Lance library; the released LanceDB wheels were not modified.

Existing indices

This prevents stale memberships from being retained during future updates. It does not automatically repair an index that already contains them. In the affected synthetic dataset, opening it with the patched library still returned the stale match; rebuilding the index with create_scalar_index("acl", "LABEL_LIST", replace=True) restored the correct result.

Please consider whether the conditional authorization impact warrants critical-fix treatment and a backport.

Reported and fixed by Manabu TERADA (@terapyon).

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The update now applies the live-row filter to both per-label postings and the separate null-list bitmap before replacement rows are merged. This is the right incremental fix, with coverage for stable row IDs, physical row addresses, null/list edge cases, and the optimize path.

Already-stale LABEL_LIST indexes are not repaired by upgrading. Deployments that may have optimized such an index after updates should rebuild it once after adopting this fix; otherwise stale matches can persist.

[terapyon]

@westonpace Can you review this PR? I made it for related #8840

[Xuanwo]

Thank you for this fix!