branch-4.0: [feature](iceberg) Support Partition Evolution DDL for Iceberg Tables (#57972)#58538
Merged
yiguolei merged 2 commits intoapache:branch-4.0from Dec 2, 2025
Merged
Conversation
…apache#57972) This PR implements support for partition evolution in Iceberg tables, allowing users to dynamically modify table partition strategies without rewriting data files. This is a metadata-only operation that maintains multiple partition spec versions. Apache Iceberg supports partition evolution, which enables changing partition strategies on existing tables without data migration. Doris, as a query engine for Iceberg, needs to support SQL syntax for partition evolution operations to provide users with flexible partition management. - **Add Partition Field**: Add new partition field to existing partition specifications - **Drop Partition Field**: Remove partition field from existing partition specifications - **Replace Partition Field**:Replace partition field from existing partition specifications with new partition field 1. **Metadata-only operation**: Partition evolution only updates table metadata, no data files are rewritten 2. **Backward compatibility**: Historical data retains original partition specs, new data uses new partition specs 3. **Syntax compatibility**: Follows Spark SQL ALTER TABLE syntax for consistency ```sql -- use optional AS keyword to specify a custom name for the partition field ALTER TABLE table_name ADD PARTITION KEY partition_transform [AS key_name]; -- example ALTER TABLE prod.db.sample ADD PARTITION KEY bucket(16, id); ALTER TABLE prod.db.sample ADD PARTITION KEY truncate(4, data); ALTER TABLE prod.db.sample ADD PARTITION KEY year(ts); -- use optional AS keyword to specify a custom name for the partition field ALTER TABLE prod.db.sample ADD PARTITION KEY bucket(16, id) AS shard; ``` ```sql ALTER TABLE table_name DROP PARTITION KEY partition_transform｜key_name; -- example ALTER TABLE prod.db.sample DROP PARTITION KEY catalog; ALTER TABLE prod.db.sample DROP PARTITION KEY bucket(16, id); ALTER TABLE prod.db.sample DROP PARTITION KEY truncate(4, data); ALTER TABLE prod.db.sample DROP PARTITION KEY year(ts); ALTER TABLE prod.db.sample DROP PARTITION KEY shard; ``` ```sql -- use optional AS keyword to specify a custom name for the partition field ALTER TABLE table_name REPLACE PARTITION KEY key_name WITH partition_transform [AS key_name]; -- example ALTER TABLE prod.db.sample REPLACE PARTITION KEY ts_day WITH day(ts); -- use optional AS keyword to specify a custom name for the new partition field ALTER TABLE prod.db.sample REPLACE PARTITION KEY ts_day WITH day(ts) AS day_of_ts; ``` | Transform | Syntax | Example | |-----------|--------|---------| | bucket | `bucket(N, column)` | `bucket(16, id)` | | truncate | `truncate(N, column)` | `truncate(10, name)` | | year | `year(column)` | `year(ts)` | | month | `month(column)` | `month(ts)` | | day | `day(column)` | `day(ts)` | | hour | `hour(column)` | `hour(ts)` | | identity | `column` | `category` |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
morningman
approved these changes
Dec 1, 2025
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
yiguolei
approved these changes
Dec 2, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bp: #57972