Skip to content

Conversation

@mbien
Copy link
Member

@mbien mbien commented Apr 24, 2025

(e.g JsTypedTextInterceptorTest#testDisabledSmartQuotes had a high failure rate)

situation:

  • OptionsUtils caches some Preference values and updates those asynchronously via a listener (Preferences event Thread is not EDT even though it has a deceivingly similar name when looked at through the debugger)
  • tests expect values to be immediately available after writes to Preferences (and also that clear() isn't delayed)

fix:

  • disable async preferences tracking during tests, write values manually via OptionsUtils instead of preferences.
  • update all OptionsUtils copies so that the cached values are made volatile since they are shared between threads and fixed lazy init code which used a CAS block as critical section

part of #8183

@mbien mbien added JavaScript [ci] enable web job and extra JavaScript tests (webcommon/javascript2.editor) tests labels Apr 24, 2025
@mbien mbien added this to the NB27 milestone Apr 24, 2025
@mbien
Copy link
Member Author

mbien commented Apr 24, 2025

testDisabledSmartQuotes1: org.netbeans.modules.javascript2.editor.JsDeletedTextInterceptorTest fails too (same cause). We probably have to check usages and put this into a base class.

@mbien mbien marked this pull request as draft April 24, 2025 19:52
@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from 09f9738 to d5b991f Compare April 24, 2025 20:25
@mbien mbien changed the title Fix race condition in JsTypedTextInterceptorTest Fix race condition in JavaScript TextInterceptorTests Apr 24, 2025
@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from d5b991f to 1b7528a Compare April 24, 2025 20:27
@neilcsmith-net
Copy link
Member

Curious! Are you sure sync() is not just taking long enough to mask the race condition? Is it guaranteed that events will have been fired by its return?

That OptionsUtils class should probably protect the caching map for concurrent access?

Maybe there should be method in OptionsUtils to flush its cache instead? Or even set keys through it. It's not API anyway as far as I can see.

@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from 1b7528a to 72d0fcd Compare April 25, 2025 14:46
@mbien
Copy link
Member Author

mbien commented Apr 25, 2025

is not just taking long enough to mask the race condition?

yes I had that in mind too since this is the case for all race conditions. If you make the code slow enough they disappear. It is difficult to proof that this is not the case here too - given that sync is surprisingly slow.

I will close this PR later and isolate the three tests to wrap them in a retry script via #8450

@mbien mbien added the do not merge Don't merge this PR, it is not ready or just demonstration purposes. label Apr 25, 2025
@mbien
Copy link
Member Author

mbien commented Apr 25, 2025

That OptionsUtils class should probably protect the caching map for concurrent access?

it stores it in fields, but those should be volatile at the very least, yeah. I won't change anything unless it also fixes the problem completely, since changing the timing could only obfuscate the issue.

Maybe there should be method in OptionsUtils to flush its cache instead? Or even set keys through it. It's not API anyway as far as I can see.

this wouldn't be sufficient. Since nothing knows what is still in the event queue at this point. I observed the clear event arriving after the Preference write but before the write event arrived. The "happens before" promise doesn't exist if the event thread itself can't be flushed.

@mbien
Copy link
Member Author

mbien commented Apr 25, 2025

the fact that this was copied around everywhere:

find . -type f -iname OptionsUtils.java
./webcommon/javascript2.editor/src/org/netbeans/modules/javascript2/editor/options/OptionsUtils.java                                                                                                                               
./php/php.twig/src/org/netbeans/modules/php/twig/editor/ui/options/OptionsUtils.java
./php/php.editor/src/org/netbeans/modules/php/editor/options/OptionsUtils.java
./ide/db.sql.editor/src/org/netbeans/modules/db/sql/editor/OptionsUtils.java

doesn't really motivate me to work on this at all tbh.

@neilcsmith-net
Copy link
Member

That OptionsUtils class should probably protect the caching map for concurrent access?

it stores it in fields, but those should be volatile at the very least, yeah.

hmmm, I totally misread where the storage was at a glance! 😆 But yes, possibly volatile ...

Maybe there should be method in OptionsUtils to flush its cache instead? Or even set keys through it. It's not API anyway as far as I can see.

this wouldn't be sufficient. Since nothing knows what is still in the event queue at this point.

I was only thinking of the test requirements, not a general fix. Enough to ensure the setting in the test works synchronously.

Another option might be to add a temporary second preference listener (assuming they're called in the order they're added) and listen for the required event(s) to propagate?

@mbien mbien added PHP [ci] enable extra PHP tests (php/php.editor) web [ci] enable web job SQL labels Apr 26, 2025
@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch 2 times, most recently from 5262616 to bd4e182 Compare April 26, 2025 00:32
@mbien
Copy link
Member Author

mbien commented Apr 26, 2025

another attempt:

  • disabled the async preferences updater during tests, the values are set directly
  • made all cached values volatile to fix the OptionsUtils incarnations

(verified with ~400 local test runs)

lazy init via CAS block is also incorrect, but lets see if the tests work first before making more changes.

@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from bd4e182 to 9d611a2 Compare April 26, 2025 02:28
@mbien mbien removed the do not merge Don't merge this PR, it is not ready or just demonstration purposes. label Apr 26, 2025
@mbien mbien marked this pull request as ready for review April 26, 2025 16:26
@mbien mbien requested a review from neilcsmith-net April 26, 2025 16:39
@mbien
Copy link
Member Author

mbien commented Apr 30, 2025

planing to merge maintenance PRs like this one here soon.

@mbien mbien added the CI continuous integration changes label May 5, 2025
@mbien
Copy link
Member Author

mbien commented May 5, 2025

rebasing, planning to merge it once green

@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from 9d611a2 to e63ea4c Compare May 5, 2025 20:22
@mbien
Copy link
Member Author

mbien commented May 5, 2025

reverted a test too much. Will update in a moment.

(e.g JsTypedTextInterceptorTest#testDisabledSmartQuotes and the
JsDeletedTextInterceptorTest equivalent had a high failure ratse)

situation:
 - OptionsUtils caches some Preference values and updates those
   asynchronously via a listener (Preferences event Thread is not EDT)
 - tests expect values to be immediately available after written to
   Preferences (and also that clear() isn't delayed)

fix:
 - disable async preferences tracking during tests, write values
   manually
 - update all OptionsUtils copies so that the cached values are made
   volatile since they are shared between threads and fixed lazy init
   code
@mbien mbien force-pushed the fix-js-typedtextinterceptor-test branch from e63ea4c to e7a32d0 Compare May 5, 2025 22:27
@mbien mbien merged commit dfc692e into apache:master May 5, 2025
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI continuous integration changes JavaScript [ci] enable web job and extra JavaScript tests (webcommon/javascript2.editor) PHP [ci] enable extra PHP tests (php/php.editor) SQL tests web [ci] enable web job

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants