Skip to content

[feature](fe) Add information_schema role mappings table#61998

Closed
CalvinKirs wants to merge 23 commits into
apache:masterfrom
CalvinKirs:feature/role-mapping-system-table
Closed

[feature](fe) Add information_schema role mappings table#61998
CalvinKirs wants to merge 23 commits into
apache:masterfrom
CalvinKirs:feature/role-mapping-system-table

Conversation

@CalvinKirs

@CalvinKirs CalvinKirs commented Apr 1, 2026

Copy link
Copy Markdown
Member

feature Add information_schema role mappings table

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add information_schema.role_mappings so administrators can inspect authentication integration role mapping definitions and audit fields through SQL.

Release note

Add information_schema.role_mappings for inspecting role mapping metadata.

Check List (For Author)

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Move AuthIntegration role mapping into the successful authentication runtime path so granted roles are computed once during authentication and downstream authenticators only propagate the result.

### Release note

None

### Check List (For Author)

- Test: Attempted `./run-fe-ut.sh --run org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest`, `./run-fe-ut.sh --run org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest`, `./run-fe-ut.sh --run org.apache.doris.mysql.authenticate.AuthenticatorManagerTest`
    - Unit Test (blocked by existing FE build issues: concurrent cache restore collisions, then fe-core pattern generator failure `NoClassDefFoundError: org.apache.doris.nereids.pattern.generator.javaast.TypeBound`)
- Behavior changed: Yes (role mapping is now computed in `AuthenticationIntegrationRuntime` immediately after successful authentication, while `AuthenticationIntegrationAuthenticator` only propagates granted roles)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary: Define the approved design for porting CREATE/DROP ROLE MAPPING into the current OIDC branch, removing property-backed production role mapping, and adding the minimum MySQL OIDC login bridge needed to reach the existing OIDC plugin.

### Release note

None

### Check List (For Author)

- Test: No need to test (design document only)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Port CREATE ROLE MAPPING and DROP ROLE MAPPING into FE metadata, switch runtime evaluation to metadata-backed role mappings, advertise the MySQL capability flags required by OIDC clients, and propagate authenticated roles into session privilege checks so OIDC logins receive mapped roles end to end.

### Release note

Support metadata-backed role mappings for authentication integrations and apply mapped roles during OIDC MySQL login.

### Check List (For Author)

- Test: FE unit tests
    - ./run-fe-ut.sh --run org.apache.doris.nereids.parser.RoleMappingParserTest,org.apache.doris.authentication.RoleMappingMgrTest,org.apache.doris.authentication.AuthenticationIntegrationMgrTest,org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest,org.apache.doris.mysql.MysqlCapabilityTest,org.apache.doris.mysql.MysqlHandshakePacketTest,org.apache.doris.mysql.MysqlAuthPacketTest,org.apache.doris.mysql.authenticate.password.ClearPasswordResolverTest,org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest,org.apache.doris.authentication.handler.AuthenticationServiceTest,org.apache.doris.authentication.handler.OidcAuthenticationServiceTest,org.apache.doris.mysql.privilege.AuthTest
- Behavior changed: Yes (OIDC login now consumes metadata-backed role mappings and session-mapped roles)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Prevent FE from throwing a null pointer when a client closes the MySQL connection during handshake, and preserve the original OIDC token when password auth falls back to authentication_chain so the fallback authenticator can still use the client-provided token.

### Release note

Improve FE MySQL authentication error handling and OIDC fallback behavior.

### Check List (For Author)

- Test: No test run (per user request; added FE unit tests but did not execute them)
- Behavior changed: Yes (handshake failures now return a clear error instead of a null pointer; OIDC fallback reuses the original token)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Re-enable edit log persistence for create, alter, and drop operations in AuthenticationIntegrationMgr so authentication integration metadata changes participate in FE journal replay again.

### Release note

None

### Check List (For Author)

- Test: No need to test (per user request in this session)
- Behavior changed: Yes (authentication integration DDL now writes edit log records again)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add temporary FE-side logging around MySQL OIDC auth packet decoding and token bridging so we can determine whether the incoming mysqlsh token still contains a length-encoded prefix or is being transformed before OIDC validation.

### Release note

None

### Check List (For Author)

- Test: No need to test (debug-only instrumentation requested by user; user asked not to run tests)
- Behavior changed: Yes (adds temporary INFO logs for OIDC MySQL authentication flow only)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Move MySQL OIDC auth-packet parsing out of ClearPasswordResolver and make packet-aware credential resolution explicit for integration and plugin authenticators.

### Release note

None

### Check List (For Author)

- Test: No test run (per user request)
    - No test execution requested by the user in this session
- Behavior changed: Yes (direct plugin-based OIDC auth now consumes credentials from the initial MySQL auth packet through an explicit resolver path)
- Does this need documentation: No
… role mapping

 The core changes are:

  1. Add MySQL protocol-layer support for OIDC login
     - bridge OIDC credentials carried in the MySQL authentication packet into Doris authentication requests
     - allow OIDC-based external login to enter the FE authentication pipeline through the MySQL protocol path

  2. Add ROLE MAPPING for built-in authorization
     - introduce role mapping so external authenticated identities can be mapped to Doris built-in roles
     - support using external identity attributes/groups to drive internal authorization decisions

  3. Integrate the authentication feature modules into FE
     - wire the authentication plugin/integration modules into the FE runtime
     - connect external authentication, JIT/external login, and built-in role authorization into one complete flow

  With these changes, Doris can accept OIDC-based external logins from the MySQL protocol entry, authenticate them through the integrated authentication modules, and apply Doris-native authorization
  through ROLE MAPPING.

  ### Release note

  Support OIDC-based external login through the MySQL protocol and add ROLE MAPPING for Doris built-in authorization.
…mapping-oidc

# Conflicts:
#	fe/fe-core/src/main/java/org/apache/doris/persist/meta/PersistMetaModules.java
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Remove the OIDC authentication plugin module and the role-mapping OIDC bridge design spec from this branch so the PR only keeps the MySQL protocol credential bridge and role-mapping related FE integration work.

### Release note

None

### Check List (For Author)

- Test: No test run (per user request)
    - No test execution requested in this session
- Behavior changed: Yes (this branch no longer submits the OIDC plugin module or the design spec; MySQL protocol-layer OIDC credential adaptation remains)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Address review blockers in the OIDC login and role mapping integration by including the MySQL auth packet OIDC credential extractor in fe-core, fixing role mapping metadata image persistence, aligning lock ordering between authentication integration and role mapping managers, avoiding null role materialization in Auth, using the command context consistently for DROP ROLE MAPPING privilege checks, and making evaluator cache refresh atomic.

### Release note

Fix role mapping metadata persistence and stabilize fe-core OIDC login integration.

### Check List (For Author)

- Test: No test run (per request)
    - No unit test or regression test was executed
- Behavior changed: Yes
    - Role mappings are now included in image save/load and fe-core OIDC credential extraction is committed with the protocol bridge
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Preserve safe high-level authentication failure context when all configured authenticators fail, while keeping sensitive credential failures generic.

### Release note

Improve final authentication error reporting for plugin and integration fallback failures while preserving generic access-denied behavior for sensitive credential errors.

### Check List (For Author)

- Test: FE unit test
    - ./run-fe-ut.sh --run org.apache.doris.mysql.authenticate.AuthenticatorManagerTest#testAuthenticateReportsOperationalFailureSummaryWhenAllAttemptsFail+testAuthenticateKeepsAccessDeniedWhenAnyAttemptHasSensitiveFailure
- Behavior changed: Yes (service/configuration authentication failures now surface a high-level operational error when no sensitive failure occurred)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add a design spec for Codex-driven dependency-check automation that runs outside Maven plugin configuration, classifies direct versus transitive dependencies, upgrades through centralized version controls, validates each upgrade batch with Maven build, and writes short-term exceptions to an external suppression file.

### Release note

None

### Check List (For Author)

- Test: No need to test (design doc only)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add the implementation plan for the external Codex-driven dependency-check workflow, including test-first scaffolding, scan/report parsing, dependency-tree analysis, artifact generation, and suppression management.

### Release note

None

### Check List (For Author)

- Test: No need to test (plan doc only)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add information_schema.role_mappings so administrators can inspect authentication integration role mapping definitions and audit fields through SQL.

### Release note

Add information_schema.role_mappings for inspecting role mapping metadata.

### Check List (For Author)

- Test: FE Unit Test; Regression test attempted
    - FE Unit Test: ./run-fe-ut.sh --run org.apache.doris.service.FrontendServiceImplTest
    - FE Unit Test: ./run-fe-ut.sh --run org.apache.doris.catalog.SchemaTableTest
    - Regression test: ./run-regression-test.sh --run -d auth_p0 -s test_role_mapping_system_table (blocked locally because the default 127.0.0.1:9030 cluster did not include CREATE ROLE MAPPING syntax)
- Behavior changed: Yes (adds information_schema.role_mappings)
- Does this need documentation: No
@Thearas

Thearas commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@CalvinKirs CalvinKirs changed the title [WIP]Feature/role mapping system table [feature](fe) Add information_schema role mappings table Apr 1, 2026
@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@CalvinKirs
CalvinKirs marked this pull request as draft April 1, 2026 08:27
@hello-stephen

Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.48% (1798/2291)
Line Coverage 64.16% (32268/50296)
Region Coverage 65.01% (16173/24878)
Branch Coverage 55.46% (8616/15536)

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 72.85% (475/652) 🎉
Increment coverage report
Complete coverage report

@doris-robot

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.89% (20030/37869)
Line Coverage 36.48% (187978/515233)
Region Coverage 32.73% (145840/445618)
Branch Coverage 33.89% (63874/188458)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.49% (27256/37088)
Line Coverage 57.17% (293652/513684)
Region Coverage 54.47% (244970/449731)
Branch Coverage 56.11% (106052/189024)

@CalvinKirs CalvinKirs closed this Apr 2, 2026
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 3, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61998

Problem Summary: The role_mappings information_schema table was added on the FE side, but BE still fell back to SchemaDummyScanner for SCH_ROLE_MAPPINGS, so end-to-end SQL queries could not fetch any rows.

### Release note

None

### Check List (For Author)

- Test: Attempted BE unit test and static check
    - Unit Test: Added SchemaRoleMappingsScannerTest
    - Attempted: ./run-be-ut.sh --run --filter=SchemaRoleMappingsScannerTest.* -j4 (blocked by local OpenMP/OpenBLAS/Faiss build environment issues)
    - Static check: git diff --check
- Behavior changed: Yes (information_schema.role_mappings now has a BE scanner path instead of falling back to SchemaDummyScanner)
- Does this need documentation: No
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 3, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61998

Problem Summary: Fix clang-format violations in the role mappings BE scanner and its unit test so the formatter check passes.

### Release note

None

### Check List (For Author)

- Test: clang-format --dry-run --Werror on the touched BE files; git diff --check rich/master...HEAD
    - Unit Test / No need to test (format-only change)
- Behavior changed: No
- Does this need documentation: No
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 7, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61998

Problem Summary: The role_mappings information_schema table was added on the FE side, but BE still fell back to SchemaDummyScanner for SCH_ROLE_MAPPINGS, so end-to-end SQL queries could not fetch any rows.

### Release note

None

### Check List (For Author)

- Test: Attempted BE unit test and static check
    - Unit Test: Added SchemaRoleMappingsScannerTest
    - Attempted: ./run-be-ut.sh --run --filter=SchemaRoleMappingsScannerTest.* -j4 (blocked by local OpenMP/OpenBLAS/Faiss build environment issues)
    - Static check: git diff --check
- Behavior changed: Yes (information_schema.role_mappings now has a BE scanner path instead of falling back to SchemaDummyScanner)
- Does this need documentation: No
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 7, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#61998

Problem Summary: Fix clang-format violations in the role mappings BE scanner and its unit test so the formatter check passes.

### Release note

None

### Check List (For Author)

- Test: clang-format --dry-run --Werror on the touched BE files; git diff --check rich/master...HEAD
    - Unit Test / No need to test (format-only change)
- Behavior changed: No
- Does this need documentation: No
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants