feat: Add Internet Archive Save Page Now integration

[ishank-dev]

Closes #11090

Add archive.py module for Save Page Now API integration in OL

Technical

• is_already_archived(url) - Checks Wayback Machine availability API
• archive_url(url, ...) - Submits URLs to Save Page Now API

Testing

Tests use sample URLs from this collab: https://colab.research.google.com/drive/1y7c4n34N6oahlMjImDfG1QzembHWUJW4#scrollTo=g4YZshpc_ICP

export IA_ACCESS_KEY="your_key"
export IA_SECRET_KEY="your_secret"
python scripts/tests/test_archive.py

Screenshot

Stakeholders

@mekarpeles

[mekarpeles]

We probably want to use a different key pair for this purpose

[mekarpeles]

Looks good to me overall

1. this code could live in core/ia.py as there's already Internet Archive specific code there
2. we should use different keys than xauth; staff can help with this.

squash on merge

[ishank-dev]

Sounds good.

1. I initially referenced openlibrary/conf/openlibrary.yml for the variables, but I need guidance on the correct source of truth.
2. Yes makes sense to move the code to ia.py adding this change in my next commit

[ishank-dev]

moved the changes, for testing
change log:

• Add save_page_now() in openlibrary/core/ia.py
• Extract get_spn_s3_keys() (TODO: finalize source of truth), preferring lending/config/env
• Remove openlibrary/core/archive.py
• Convert scripts/tests/test_archive.py to pytest with parametrized live URLs and skip without creds

For testing run

docker compose run --rm \
  -e IA_ACCESS_KEY=<ACCESS KEY>\
  -e IA_SECRET_KEY=<SECRET_KEY>\
  -v "$PWD/scripts/tests:/openlibrary/scripts/tests" \
  web bash -lc "pytest -q /openlibrary/scripts/tests/test_archive.py | cat"```

[ishank-dev]

@mekarpeles for these new changes to work with internetarchive python library
I had to create openlibrary/.secrets/ia.ini locally

[s3]
access = <my_access_key>
secret = <my_secret_key>

I need inputs and help in verifying if it will pick credentials in beta or prod cloud environment as per the docs, it reads ~/.config/ia.ini (or ~/.ia) for the running user