Remove legacy TheSchwartz ESN workers and ESN_OVER_SQS tunable#3520
Merged
zorkian merged 4 commits intodreamwidth:mainfrom Feb 15, 2026
Merged
Remove legacy TheSchwartz ESN workers and ESN_OVER_SQS tunable#3520zorkian merged 4 commits intodreamwidth:mainfrom
zorkian merged 4 commits intodreamwidth:mainfrom
Conversation
$LJ::ESN_OVER_SQS has been 1 in production, meaning 100% of ESN events route through DW::TaskQueue (SQS). The four legacy TheSchwartz-based ESN workers are consuming no work. This removes the legacy path entirely so ESN always uses the modern DW::Task::ESN::* pipeline. - DW::TaskQueue dispatch() now unconditionally calls fire_task for events - LJ::Event::fire_job() removed; process_fired_events() rewritten to drain DW::TaskQueue synchronously (used by tests) - LJ::ESN: removed schwartz_capabilities(), process_fired_events(), jobs_of_unique_matching_subs(), and all four LJ::Worker::* packages - Deleted 10 legacy worker scripts from bin/worker/ - Removed esn-legacy entries from workers.json and dwtool CategoryOrder - LocalDisk::receive() gains optional wait_secs param (default 10) so process_fired_events can do non-blocking receives Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
schwartz_role was used by the legacy TheSchwartz ESN workers (removed in the previous commit) to route certain event types to the mass job database. No code calls these methods anymore. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These TheSchwartz tables were used by the mass ESN workers to process high-volume events on a separate job database. No code reads from or writes to them anymore — all ESN goes through DW::TaskQueue/SQS. The tables will remain in existing databases until manually dropped. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zorkian
force-pushed
the
remove-legacy-esn-workers
branch
from
713f6a8 to
91f9995
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
$LJ::ESN_OVER_SQShas been set to1in production, meaning 100% of ESN events route throughDW::TaskQueue(SQS). The four legacy TheSchwartz-based ESN workers have been consuming no work.This removes the legacy path entirely so ESN always uses the modern
DW::Task::ESN::*pipeline.Perl changes:
DW::TaskQueue::dispatch()now unconditionally callsfire_taskforLJ::Eventobjects,removing the
$LJ::ESN_OVER_SQSpercentage-based routing splitLJ::Event::fire_job()deleted;process_fired_events()rewritten to drainDW::TaskQueuesynchronously (used by the ESN test suite)
LJ::ESN: removedschwartz_capabilities(),process_fired_events(),jobs_of_unique_matching_subs(), and all fourLJ::Worker::*packages (~280 lines)DW::TaskQueue::LocalDisk::receive()gains an optionalwait_secsparameter (default 10) soprocess_fired_eventscan do non-blocking receives instead of busy-waiting on empty queuesInfrastructure changes:
bin/worker/(the individual and mass variants ofesn-fired-event,esn-cluster-subs,esn-filter-subs,esn-process-sub, andprocess-esn)esn-legacycategory entries fromconfig/workers.jsonesn-legacyfromCategoryOrderin dwtool'sworkers.goNet: -682 lines, +39 lines.
All three ESN tests pass with the new drain implementation:
t/esn-end2end.t(12 assertions)t/esn-journalnewcomment.t(50 assertions)t/esn-duplicatesubscriptions.t(7 assertions)CODE TOUR: This is a behind-the-scenes cleanup. Dreamwidth's Event-Subscription-Notification (ESN)
system — the thing that sends you emails and inbox notifications when someone comments on your
entry, etc. — used to have two parallel paths for processing events: an older one based on
TheSchwartz job queue, and a newer one based on SQS. We've been running 100% on the new path for a
while now, so this removes the old one entirely along with the 10 worker scripts that powered it.
Nothing changes from a user perspective; notifications keep working exactly as before.
🤖 Generated with Claude Code