Skip to content

Updates script to write all preferences to store#11668

Merged
mekarpeles merged 2 commits intointernetarchive:masterfrom
jimchamp:update-preference-migration-script
Jan 7, 2026
Merged

Updates script to write all preferences to store#11668
mekarpeles merged 2 commits intointernetarchive:masterfrom
jimchamp:update-preference-migration-script

Conversation

@jimchamp
Copy link
Copy Markdown
Collaborator

@jimchamp jimchamp commented Jan 7, 2026

Advances #11009

Adds --legacy flag to the preference migration script. When --legacy is true, all remaining preferences in the thing table will be written to the store.

Technical

Testing

Screenshot

Stakeholders

Copilot AI review requested due to automatic review settings January 7, 2026 00:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a --legacy flag to the preference migration script that allows writing all remaining preferences from the thing table to the store table, rather than just a subset based on specific criteria.

  • Introduces new _fetch_legacy_preference_keys() function to query all preferences in thing table not yet in store
  • Adds --legacy command-line flag to switch between the two migration modes
  • Updates documentation to reflect the new functionality

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
scripts/migrations/write_prefs_to_store.py Adds new _fetch_legacy_preference_keys() function and --legacy argument to enable migration of all preferences
scripts/migrations/README.md Updates description to document the --legacy flag functionality

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +117 to +122
keys = []
try:
legacy_preference_entries = oldb.query(legacy_key_query)
keys = [entry.get('key', '') for entry in list(legacy_preference_entries)]
except DatabaseError as e:
print(f"An error occurred while fetching preference keys: {e}")
Copy link

Copilot AI Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new function lacks transaction handling that exists in the original _fetch_preference_keys() function. The original function creates a transaction with t = oldb.transaction() and performs a rollback in both the error case and the success case. This transaction pattern should be replicated here for consistency and to ensure proper database resource management.

Suggested change
keys = []
try:
legacy_preference_entries = oldb.query(legacy_key_query)
keys = [entry.get('key', '') for entry in list(legacy_preference_entries)]
except DatabaseError as e:
print(f"An error occurred while fetching preference keys: {e}")
keys = []
t = None
try:
t = oldb.transaction()
legacy_preference_entries = oldb.query(legacy_key_query)
keys = [entry.get('key', '') for entry in list(legacy_preference_entries)]
except DatabaseError as e:
if t is not None:
t.rollback()
print(f"An error occurred while fetching preference keys: {e}")
else:
if t is not None:
t.rollback()

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transaction in the other function was added to support the use of a temporary table. Rolling back that transaction cleans up the temporary table.

Also, I'm not exactly sure what benefit could be had by rolling back the single SELECT query that is executed in this function......

@jimchamp jimchamp mentioned this pull request Jan 7, 2026
13 tasks
@mekarpeles mekarpeles merged commit 55336f5 into internetarchive:master Jan 7, 2026
4 checks passed
mystic-06 pushed a commit to mystic-06/openlibrary that referenced this pull request Jan 11, 2026
…nce-migration-script

Updates script to write all preferences to `store`
@jimchamp jimchamp deleted the update-preference-migration-script branch February 4, 2026 00:46
lokesh pushed a commit to lokesh/openlibrary that referenced this pull request Feb 4, 2026
…nce-migration-script

Updates script to write all preferences to `store`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants