Skip to content

Conversation

@asolntsev
Copy link
Contributor

@asolntsev asolntsev commented Dec 29, 2025

User description

💥 What does this PR do?

Fixes AssertJ's asserts for java.util.Maps and java.util.Sets.

🔧 Implementation Notes

The problem is that AssertJ's method isEqualTo doesn't always work for Sets and Maps because it assumes strict order of elements:

  1. assertThat(MAP).isEqualTo(Map.of(...)) // can fail time-to-time because of different order of elements
  2. assertThat(SET).isEqualTo(Set.of(...)) // can fail time-to-time because of different order of elements

In both cases, we have to replace isEqualTo by containsExactlyInAnyOrderEntriesOf or similar method.

💡 Additional Considerations

🔄 Types of changes

  • Bug fix (backwards compatible)

PR Type

Bug fix


Description

  • Replace AssertJ isEqualTo with order-agnostic assertions for Maps and Sets

  • Fix flaky tests that fail due to element ordering assumptions

  • Improve PDF validation in BrowsingContextTest with content verification

  • Simplify test setup code using modern Java collection literals

  • Add proper resource management with try-with-resources blocks


Diagram Walkthrough

flowchart LR
  A["AssertJ isEqualTo<br/>for Maps/Sets"] -->|Replace with| B["containsExactlyInAnyOrder<br/>Entries/Elements"]
  C["Flaky test assertions<br/>due to ordering"] -->|Fixed by| B
  D["Test code cleanup<br/>ArrayList/HashMap"] -->|Simplify to| E["List.of/Map.of<br/>Set.of"]
  F["Resource leaks<br/>JsonInput"] -->|Add| G["try-with-resources<br/>blocks"]
Loading

File Walkthrough

Relevant files
Bug fix
18 files
ProxyTest.java
Fix List assertion to use containsExactly                               
+2/-2     
ChromeOptionsTest.java
Replace Map assertions with order-agnostic checks               
+18/-19 
FirefoxProfileTest.java
Fix Set assertion and add resource cleanup                             
+9/-7     
ConcatenatingConfigTest.java
Replace Set assertions with containsExactlyInAnyOrder       
+2/-3     
InternetExplorerOptionsTest.java
Fix Map capability assertion with order-agnostic check     
+3/-4     
PointerInputTest.java
Replace Map assertion with containsExactlyInAnyOrderEntriesOf
+2/-2     
WheelInputTest.java
Replace Map assertion with containsExactlyInAnyOrderEntriesOf
+2/-2     
JsonInputTest.java
Add try-with-resources for resource management                     
+134/-113
JsonOutputTest.java
Replace Map/List assertions with order-agnostic checks     
+48/-27 
JsonTest.java
Fix Map and List assertions for unordered collections       
+6/-6     
NewSessionPayloadTest.java
Replace Set/List assertions with order-agnostic checks     
+9/-12   
RemoteLogsTest.java
Simplify Set creation and fix assertion                                   
+4/-15   
RemoteWebDriverUnitTest.java
Replace List assertion with containsExactly                           
+2/-4     
W3CHandshakeResponseTest.java
Fix Map assertion with order-agnostic check                           
+5/-3     
UrlTemplateTest.java
Replace Map assertions with containsExactlyInAnyOrderEntriesOf
+6/-5     
ByChainedTest.java
Simplify test setup and fix assertions                                     
+84/-128
DefaultElementLocatorTest.java
Replace List assertion with containsExactly                           
+2/-3     
ExpectedConditionsTest.java
Replace List assertions with containsExactly                         
+4/-6     
Enhancement
1 files
BrowsingContextTest.java
Improve PDF validation with content verification                 
+5/-1     
Miscellaneous
1 files
SetNetworkConditionsTest.java
Add missing test method and imports                                           
+11/-0   
Tests
2 files
GeckoDriverServiceTest.java
Split and clarify timeout builder tests                                   
+9/-5     
DesiredCapabilitiesTest.java
Improve Map entry assertion clarity                                           
+4/-2     
Cleanup
2 files
DecoratedOptionsTest.java
Simplify Set creation with Set.of                                               
+1/-3     
DecoratedWebDriverTest.java
Simplify Set creation with Set.of                                               
+1/-3     

@asolntsev asolntsev self-assigned this Dec 29, 2025
@selenium-ci selenium-ci added C-java Java Bindings B-devtools Includes everything BiDi or Chrome DevTools related B-support Issue or PR related to support classes labels Dec 29, 2025
@asolntsev asolntsev added this to the 4.40.0 milestone Dec 29, 2025
@selenium-ci
Copy link
Member

Thank you, @asolntsev for this code suggestion.

The support packages contain example code that many users find helpful, but they do not necessarily represent
the best practices for using Selenium, and the Selenium team is not currently merging changes to them.

After reviewing the change, unless it is a critical fix or a feature that is needed for Selenium
to work, we will likely close the PR.

We actively encourage people to add the wrapper and helper code that makes sense for them to their own frameworks.
If you have any questions, please contact us

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Dec 29, 2025

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

🔴
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status:
Misnamed lifecycle method: The method annotated with @AfterEach is named like a test and does not describe teardown
intent, reducing readability and self-documentation.

Referred Code
@AfterEach
void testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag() {
  driver.get(pages.simpleTestPage);
  WebElement img = driver.findElement(By.id("validAnchorTag"));
  String attribute = img.getAttribute("href");
  assertThat(attribute).isEqualTo(appServer.whereIs("icon.gif"));
}

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@asolntsev asolntsev added I-defect Something is not working as intended I-enhancement Something could be better and removed Review effort 3/5 labels Dec 29, 2025
@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Dec 29, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix test annotation

Change the @AfterEach annotation to @Test for the
testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag method to
ensure it runs as a standalone test.

java/test/org/openqa/selenium/bidi/emulation/SetNetworkConditionsTest.java [123-129]

-@AfterEach
+@Test
 void testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag() {
   driver.get(pages.simpleTestPage);
   WebElement img = driver.findElement(By.id("validAnchorTag"));
   String attribute = img.getAttribute("href");
   assertThat(attribute).isEqualTo(appServer.whereIs("icon.gif"));
 }
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies a significant bug where a test method is mis-annotated as @AfterEach, which would cause incorrect test execution.

Medium
High-level
Remove unrelated test from PR

The new test
testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag in
SetNetworkConditionsTest.java is unrelated to the PR's scope. It should be moved
to a more suitable test file or a different PR.

Examples:

java/test/org/openqa/selenium/bidi/emulation/SetNetworkConditionsTest.java [123-129]
  @AfterEach
  void testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag() {
    driver.get(pages.simpleTestPage);
    WebElement img = driver.findElement(By.id("validAnchorTag"));
    String attribute = img.getAttribute("href");
    assertThat(attribute).isEqualTo(appServer.whereIs("icon.gif"));
  }

Solution Walkthrough:

Before:

// In 'SetNetworkConditionsTest.java'
public class SetNetworkConditionsTest extends JupiterTestBase {
  // ... tests for network conditions

  @AfterEach
  void testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag() {
    driver.get(pages.simpleTestPage);
    WebElement img = driver.findElement(By.id("validAnchorTag"));
    String attribute = img.getAttribute("href");
    assertThat(attribute).isEqualTo(appServer.whereIs("icon.gif"));
  }
}

After:

// In 'SetNetworkConditionsTest.java'
public class SetNetworkConditionsTest extends JupiterTestBase {
  // ... tests for network conditions

  // The unrelated @AfterEach test is removed.
}

// In a more appropriate test file (e.g., WebElementAttributeTest.java)
class WebElementAttributeTest extends JupiterTestBase {
    @Test
    void testShouldReturnAnAbsoluteUrlWhenGettingHrefAttributeOfAValidAnchorTag() {
        driver.get(pages.simpleTestPage);
        WebElement img = driver.findElement(By.id("validAnchorTag"));
        String attribute = img.getAttribute("href");
        assertThat(attribute).isEqualTo(appServer.whereIs("icon.gif"));
    }
}
Suggestion importance[1-10]: 6

__

Why: The suggestion correctly identifies a completely unrelated test added to SetNetworkConditionsTest.java, which degrades code organization and PR clarity.

Low
Learned
best practice
Validate decoded bytes safely

Avoid converting arbitrary PDF bytes to a UTF-8 String; validate the decoded
bytes directly (header/footer bytes) and fail clearly if Base64 decoding throws.

java/test/org/openqa/selenium/bidi/browsingcontext/BrowsingContextTest.java [518-519]

 byte[] pdf = Base64.getDecoder().decode(printPage);
-assertThat(new String(pdf, UTF_8).trim()).startsWith("%PDF-").endsWith("%%EOF");
+assertThat(pdf).startsWith("%PDF-".getBytes(UTF_8)).endsWith("%%EOF".getBytes(UTF_8));
  • Apply / Chat
Suggestion importance[1-10]: 5

__

Why:
Relevant best practice - Add explicit validation/guards at integration boundaries (e.g., decoding/format checks) instead of assuming input is valid and text-decodable.

Low
  • Update

The problem is that AssertJ's method `isEqualTo` doesn't always work for Sets and Maps because it assumes strict order of elements:
1. `assertThat(MAP).isEqualTo(Map.of(...))`
2. `assertThat(SET).isEqualTo(Set.of(...))`

In both cases, we have to replace `isEqualTo` by `containsExactlyInAnyOrderEntriesOf` or similar method.
After the simplification, it became obvious that some of ByChainedTest tests don't really test anything. They create a bunch of lists (e.g. `elems345`) which are not used. And only the empty list of elements is loaded and asserted.

Seems that this complex/misleading test logic in ByChainedTest was
introduced in commit c4861b8 (some "patch from BenChambers"). Needs to be reviewed and fixed.
@asolntsev asolntsev force-pushed the fix/asserts-for-maps-and-sets branch from 9220164 to e0ab2e3 Compare December 29, 2025 18:36
now we check that the resulting content at least _looks like PDF_.
@asolntsev asolntsev force-pushed the fix/asserts-for-maps-and-sets branch from e0ab2e3 to a267dae Compare December 29, 2025 18:55
@asolntsev asolntsev merged commit d867b37 into SeleniumHQ:trunk Dec 29, 2025
11 checks passed
@asolntsev asolntsev deleted the fix/asserts-for-maps-and-sets branch December 29, 2025 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-devtools Includes everything BiDi or Chrome DevTools related B-support Issue or PR related to support classes C-java Java Bindings I-defect Something is not working as intended I-enhancement Something could be better

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants