Skip to content

[Stream Load][Optimize] Filter partitions by where header when generate stream load plan #5666

@liutang123

Description

@liutang123

When doing stream load, user can set partitions or temporary_partitions to specifies which partitions to import data into.

Sometimes, user don't known the partition names. FE will send all partitions info to BE for starting stream load. If some partitions(not the load partition) has bad tablet, FE will throw tablet xxx has few replicas: 1 and the load will fail.

proposal

Filter the partition names by where header.

As shown below:
Client send a stream load http request witch where header.
curl -XPUT -H 'where:dt=1' http://FE:port/api/db_name/table_name/_stream_load
If there are 3 partitions
p1 [0, 1)
p2 [1, 2)
p3 [2,3) bad partition
FE should only sends partition p2's info to BE.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions