Release Note 3.0.8

[gavinchou]

Behavior Changes

• When using ranger / LDAP, creating users in Doris is no longer prohibited #50139
• The nested attribute of variant is disabled by default. To enable it when creating a table, you need to first execute the following command in the session variable: set enable_variant_flatten_nested = true #54413

New Features

Lakehouse

(No specific content)

Asynchronous Materialized Views

(No specific content)

Query Optimizer

• Support MySQL's GROUP BY WITH ORDER syntax #53037

Query Execution

(No specific content)

Semi-structured Data Management

(No specific content)

Storage

(No specific content)

Improvements

Data Ingestion

• Optimize the flushing strategy when memory is insufficient (#52906, #53909, #42649, #54517)
• S3 Load and TVF support accessing publicly readable objects without AK/SK (#53592, #54040)

Primary Key Model

(No specific content)

Storage

(No specific content)

Storage-Compute Decoupled

• When the cache space is sufficient, the rowset generated by base compaction can be written to the file cache (#53801, #54693)
• Optimize the ALTER STORAGE VAULT command, the type attribute can be automatically inferred without explicit specification (#54394, #54475)

Lakehouse

(No specific content)

Asynchronous Materialized Views

(No specific content)

Query Optimizer

• Point queries will be planned to have only one fragment to improve the execution speed of point queries #53541

Query Execution

• Improve the performance of unique key tables in point queries #53948

Inverted Index

• Optimize the additional resource consumption of common default tokenizers when writing non-tokenized indexes #54666

Permissions

(No specific content)

Others

(No specific content)

Bug Fixes

Data Ingestion

• Fix the enclose parsing error when using multi-character column separators (#54581, #55052)
• Fix the issue where S3 Load progress is not updated in a timely manner (#54606, #54790)
• Fix the error when loading JSON boolean types into INT columns (#54397, #54640)
• Fix the issue of missing error URL return in Stream Load (#54115, #54266)
• Fix the issue where group commit is blocked after an exception is thrown in schema change #54312

Primary Key Model

(No specific content)

Storage

(No specific content)

Storage-Compute Decoupled

(No specific content)

Lakehouse

• Fix the issue of JDBC SQL pass-through parsing failure in some cases #54077
• Fix the failure issue when writing to iceberg tables with decimal partitions #54557
• Fix the query failure issue of Hudi table Timestamp type partition columns in some cases #53791

Asynchronous Materialized Views

(No specific content)

Query Optimizer

• Fix the issue of incorrectly using colocate join in some self-join scenarios #54323
• Fix the potential result error when select distinct is used with window functions #54133
• Provide more user-friendly error messages when lambda expressions appear in unexpected positions #53657

Permissions

• Fix the issue of incorrectly checking the permissions of base tables in views when querying external views #53786

Query Execution

• Fix the issue that IPV6 type cannot parse IPV4 type data #54391
• Fix the stack overflow error when parsing IPV6 type #53713

Inverted Index

(No specific content)

Complex Data Types

• BE supports selecting the simdjson parser that matches the instruction set at startup #52732
• Fix the incorrect type inference caused by data type conflicts in variant nested data types #53083
• Fix the default value filling issue for variant nested top-level nested array data #54396
• Prohibit building indexes on variant types in the cloud #54777
• Fix the issue of generating empty index files when writing data that does not meet the index conditions after creating an inverted index for variant #53814

Others

schema-change

• Fix the issue where new tablets are empty when cleaning up failed SC tasks (#53952, #54064)
• Rebuild bucket columns in the original order (#54024, #54072, #54109)
• Prohibit deleting bucket columns #54037
• Support automatic retry in case of network errors (#54419, #54488)
• Avoid deadlocks on tabletInvertedIndex (#54197, #54996)

[gavinchou]

行为变更

• 当使用 ranger / LDAP 时，不再禁止在 Doris 中创建用户 #50139

• variant 在默认情况下会关闭 nested 属性，若需在建表时开启，需先在 session variable 中执行以下命令：set enable_variant_flatten_nested = true#54413

新特性

Lakehouse

异步物化视图

查询优化器

• 支持 MySQL 的 GROUP BY WITH ORDER 语法 #53037

查询执行

半结构化数据管理

存储

其他

改进

导入

• 优化内存不足时的下刷策略 (#52906, #53909, #42649, #54517)

• S3 Load 和 TVF 支持无 AK/SK 访问公开可读的对象 (#53592, #54040)

主键

存储

存算分离

• 当缓存空间充足时，base compaction 生成的 rowset 可以写入文件缓存 (#53801, #54693)

• 优化 ALTER STORAGE VAULT 命令,type属性可以自动推导, 无需显式制定 (#54394, #54475)

Lakehouse

异步物化视图

查询优化器

• 点查查询会被规划为只有一个 fragment，以提升点查的执行速度 #53541

查询执行

• 提升 unique key 表在点查时的性能 #53948

倒排索引

• 优化不分词索引写入时常见默认分词器的额外资源消耗 #54666

权限

其他

缺陷修复

导入

• 修复在使用多字符列分隔符时，enclose 解析错误的问题 (#54581, #55052)

• 修复 S3 Load 进度更新不及时的问题 (#54606, #54790)

• 修复 JSON 格式布尔类型加载到 INT 列时的错误 (#54397, #54640)

• 修复 Stream Load 缺失错误 URL 返回的问题 (#54115, #54266)

• 修复在 schema change 抛出异常后 group commit 被阻塞的问题 #54312

主键

存储

存算分离

Lakehouse

• 修复部分情况下使用 JDBC SQL 透传解析失败的问题 #54077

• 修复写入 decimal 分区的 iceberg 表失败的问题 #54557

• 修复某些情况下 Hudi 表 Timestamp 类型分区列查询失败的问题 #53791

异步物化视图

查询优化器

• 修复在部分自关联场景中，错误使用 colocate join 的问题 #54323

• 修复 select distinct 与窗口函数一起使用时可能导致的结果错误 #54133

• 当 lambda 表达式出现在非预期位置时，提供更友好的报错 #53657

权限

• 修复查询外部视图时，错误检查视图中基表权限的问题 #53786

查询执行

• 修复 IPV6 类型不能解析 IPV4 类型数据的问题 #54391

• 修复 IPV6 类型解析时出现栈溢出的错误 #53713

倒排索引

复杂数据类型

• BE 支持启动时选择符合指令集的 simdjson parser #52732

• 修复 variant nested 数据类型在数据类型冲突情况下导致的错误类型推断 #53083

• 修复 variant nested 顶层嵌套 array 数据默认值填充问题 #54396

• 禁止 variant 类型在 cloud 上 build index #54777

• 修复 variant 创建倒排索引后写入不符合索引条件的数据时生成空索引文件的问题 #53814

其他

schema-change

• 修复在清理失败的 SC 任务时新 tablet 为空的问题 (#53952, #54064)

• 按原有顺序重建 bucket 列 (#54024, #54072, #54109)

• 禁止删除 bucket 列 #54037

• 网络错误时支持自动重试 (#54419, #54488)

• 避免在 tabletInvertedIndex 上的死锁 (#54197, #54996)