Skip to content

[AMORO-3531] Support build project with JDK 8, 11 - #3568

Merged
xxubai merged 17 commits into
apache:masterfrom
xxubai:deprecate-java8
May 29, 2025
Merged

[AMORO-3531] Support build project with JDK 8, 11#3568
xxubai merged 17 commits into
apache:masterfrom
xxubai:deprecate-java8

Conversation

@xxubai

@xxubai xxubai commented May 16, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why are the changes needed?

Close #3531.

Brief change log

  • Add jdk 8, 11 ci workflow to make sure the project can be builded
  • Prefer to use JDK 11 in the image.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (docs)

@github-actions github-actions Bot added type:docs Improvements or additions to documentation module:mixed-trino trino module for Mixed Format type:infra type:build and removed module:mixed-trino trino module for Mixed Format labels May 16, 2025
@xxubai
xxubai force-pushed the deprecate-java8 branch from 0e60d08 to 28a4632 Compare May 20, 2025 07:19
@github-actions github-actions Bot added module:ams-server Ams server module module:mixed-flink Flink moduel for Mixed Format module:mixed-spark Spark module for Mixed Format labels May 20, 2025
@xxubai
xxubai force-pushed the deprecate-java8 branch from 5297b4a to d838544 Compare May 21, 2025 15:06
@github-actions github-actions Bot removed module:mixed-flink Flink moduel for Mixed Format module:mixed-spark Spark module for Mixed Format labels May 21, 2025
xxubai
xxubai commented May 21, 2025
View reviewed changes
Comment thread pom.xml
<iceberg.version>1.6.1</iceberg.version>
<paimon.version>0.9.0</paimon.version>
<hive.version>3.1.1</hive.version>
<hive.version>3.1.3</hive.version>

@xxubai xxubai May 21, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refer to #3532 (comment)

Comment thread amoro-ams/src/test/java/org/apache/amoro/server/TestDefaultOptimizingService.java
SerializationUtil.simpleDeserialize(task.getTaskInput());
TableOptimizing.OptimizingInput input2 =
SerializationUtil.simpleDeserialize(task2.getTaskInput());
Assertions.assertEquals(input2.toString(), input.toString());

@xxubai xxubai May 21, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make changes here because RewriteFilesInput is not equals after serialized via JDK9+. The MixedTable can not keep idential bytes

@xxubai xxubai changed the title [AMORO-3531] Support build project with JDK 8, 11 and 17 [AMORO-3531] Support build project with JDK 8, 11 May 21, 2025
@xxubai

xxubai commented May 21, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I find JettyServerThreadPool and table-explorer-scheduler-0 will insert a duplicate table runtime when create a table via rest catalog.

There are two threads insert the same table runtime record:

  1. DefaultTableService#exploreInternalCatalog in the thread table-explorer-scheduler-0
  2. DefaultTableService#onTableCreated in the thread JettyServerThreadPool

The error logs see: https://github.com/apache/amoro/actions/runs/15165798529/job/42643175684?pr=3568#step:5:27267

@xxubai
xxubai requested review from czy006, klion26 and zhoujinsong May 21, 2025 16:23
@xxubai
xxubai force-pushed the deprecate-java8 branch from d838544 to abeb34f Compare May 22, 2025 02:55
@github-actions github-actions Bot added the module:mixed-spark Spark module for Mixed Format label May 22, 2025
@xxubai
xxubai requested review from Aireed and baiyangtx May 22, 2025 06:20
@Aireed

Aireed commented May 22, 2025

Copy link
Copy Markdown
Contributor

how to specify jdk version when user compile?

@xxubai

xxubai commented May 22, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

how to specify jdk version when user compile?

-Pjava8 or -Pjava11.
You can see the scripts in the CI workflow

klion26
klion26 reviewed May 26, 2025
View reviewed changes

@klion26 klion26 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! This can push we forward to support higher version of iceberg.

left some comments inline, please take a look when you're free.

Comment thread .github/workflows/core-hadoop2-ci.yml Outdated
Comment thread .github/workflows/core-hadoop3-ci.yml Outdated
Comment thread .github/workflows/docker-images.yml Outdated
Comment thread .github/workflows/docker-images.yml Outdated
Comment thread amoro-format-mixed/amoro-mixed-spark/v3.2/amoro-mixed-spark-3.2/pom.xml
@xxubai
xxubai force-pushed the deprecate-java8 branch from db23180 to 5fdadd6 Compare May 27, 2025 13:26
@xxubai
xxubai requested a review from klion26 May 27, 2025 14:53
klion26
klion26 approved these changes May 29, 2025
View reviewed changes

@klion26 klion26 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work, LGTM

@xxubai
xxubai merged commit b3f11aa into apache:master May 29, 2025
@xxubai
xxubai deleted the deprecate-java8 branch May 29, 2025 08:24
xxzhky pushed a commit to xxzhky/amoro that referenced this pull request Sep 7, 2025
@xxubai zhengsheng.du
[AMORO-3531] Support build project with JDK 8, 11, 21 (apache#3568)
02edf5d 
* Update CI configuration to support JDK 8, 11, and 17

* Update CI configuration to use actions/setup-java@v4 and actions/checkout@master

* Update CI configuration to use JDK setup and Maven versions for improved compatibility

* Revert CI configuration with comments

* Update hive.version to 3.1.3 in pom.xml for compatibility

* add extra java args

* Replace './mvnw' with 'mvn'

* revert

* Refactor assertions to compare task ID only

* Update pom.xml to enhance Java arguments for testing

* remove jdk 17

* exclude slf4j-log4j12 confilct

* Refactor CI configuration to simplify JDK setup and remove unnecessary parameters

* Add Java version profile

* remove

* fix trino compile

(cherry picked from commit b3f11aa)
xxubai added a commit to xxubai/amoro that referenced this pull request Oct 14, 2025
@xxubai
[AMORO-3531] Support build project with JDK 8, 11 (apache#3568)
ddc0e7b 
* Update CI configuration to support JDK 8, 11, and 17

* Update CI configuration to use actions/setup-java@v4 and actions/checkout@master

* Update CI configuration to use JDK setup and Maven versions for improved compatibility

* Revert CI configuration with comments

* Update hive.version to 3.1.3 in pom.xml for compatibility

* add extra java args

* Replace './mvnw' with 'mvn'

* revert

* Refactor assertions to compare task ID only

* Update pom.xml to enhance Java arguments for testing

* remove jdk 17

* exclude slf4j-log4j12 confilct

* Refactor CI configuration to simplify JDK setup and remove unnecessary parameters

* Add Java version profile

* remove

* fix trino compile

(cherry picked from commit b3f11aa)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@klion26 klion26 klion26 approved these changes
@Aireed Aireed Aireed approved these changes
@zhoujinsong zhoujinsong Awaiting requested review from zhoujinsong
@czy006 czy006 Awaiting requested review from czy006
@baiyangtx baiyangtx Awaiting requested review from baiyangtx

Assignees

No one assigned

Labels

module:ams-server Ams server module module:mixed-spark Spark module for Mixed Format type:build type:docs Improvements or additions to documentation type:infra

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Improvement]: Deprecate Java 8

3 participants

@xxubai @Aireed @klion26