Migrate browse endpoint to FastAPI#12006

Sanket17052006 · 2026-03-04T06:44:40Z

Related to #11999
This PR migrates the browse endpoint from web.py to FastAPI.

Changes Made :-

openlibrary/fastapi/internal/api.py - Added the FastAPI browse endpoint.
openlibrary/plugins/openlibrary/api.py - Deprecated the web.py endpoint.

Testing

Test command -

./tests/integration/temporary/test_browse_migration.sh http://localhost:8080 http://localhost:18080

Stakeholders

@RayBB

RayBB

It's a good start, but you need to follow the best practices per the comment I left.

openlibrary/fastapi/internal/api.py

RayBB · 2026-03-04T23:34:34Z

Look in the models.py

…

On Wed, Mar 4, 2026, 3:14 PM Sanket Srivastava ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In openlibrary/fastapi/internal/api.py <#12006 (comment)> : > +async def browse(q: str = "", page: int = 1, limit: int = 100, subject: str = "", sorts: str = "") -> JSONResponse: + """Browse endpoint (migrated from openlibrary.plugins.openlibrary.api).""" + sorts_list = sorts.split(",") + Hey @RayBB <https://github.com/RayBB>, I have made the requested changes but I couldn't locate where the parse_comma_separated_list function is defined, guide me. — Reply to this email directly, view it on GitHub <#12006 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHA5AJHGIO3RUYQ7DWBF5L4PC2F7AVCNFSM6AAAAACWGJWBN6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTQOJSGU4DGMZWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Sanket17052006 · 2026-03-06T17:01:25Z

Hey @RayBB , I think the parse_comma_separated_list function is not defined in the models.py and also while taking a look at Parth's PR #12000 I saw that he defined the parse_comma_separated_list function locally and discussed about a existing identical function in fastapi/search.py and potential move. So for now, I kept conversion of the comma separated string to list[str] as it is.
I have made all the other requested changes.

RayBB

Sorry for the delay in feedback. It's a good start!
I understand this one is tricky because it can't be tested locally.
In this case, I think it's best to just return the exact same thing and not try to form it into a special response object.

RayBB · 2026-03-15T07:17:59Z

openlibrary/fastapi/internal/api.py

+    response_model_exclude_none=False,
+)
+async def browse(request: Annotated[BrowseRequest, Depends()], pagination: Annotated[Pagination, Depends()]) -> BrowseResponse:
+    """Browse endpoint (migrated from openlibrary.plugins.openlibrary.api)."""


Please add a short description of what it does

RayBB · 2026-03-15T07:25:54Z

openlibrary/fastapi/internal/api.py

+    if not url:
+        works = []
+    else:
+        works = await asyncio.to_thread(lending.get_available, url=url)


We don't want to put this on a thread. Generally we should avoid doing that. In this case it doesn't work on testing and it's hard to see the errors when it's on a thread.

RayBB · 2026-03-15T07:27:38Z

openlibrary/fastapi/internal/api.py

+    if isinstance(works, list):
+        for work in works:
+            if isinstance(work, str) and (work.lower() == "error" or not work):
+                continue
+
+            if isinstance(work, str):
+                processed_works.append(WorkModel(key=work))
+            elif hasattr(work, "dict"):
+                processed_works.append(WorkModel(**work.dict()))
+            elif isinstance(work, dict):
+                if work.get("key") == "error":
+                    continue
+                processed_works.append(WorkModel(**work))
+            elif hasattr(work, "key"):
+                processed_works.append(WorkModel(key=work.key))


Why do we need to do this? I don't see anything similar in the old endpoint.

for more information, see https://pre-commit.ci

Sanket17052006 · 2026-03-15T12:36:46Z

Thanks @RayBB for the review! I have removed the threading and those messy if-else checks. Do the changes look good?
I am up for any more changes you think are required.

openlibrary/fastapi/internal/api.py

RayBB

Great job on this on. I think it's exactly what we need.

I put it on testing and tested with this url and quite a few others and it works perfectly.

http://testing.openlibrary.org/browse.json?subject=openlibrary_staff_picks&sorts=lending___last_browse%20desc&limit=18

Sanket17052006 · 2026-03-23T18:37:37Z

Thanks @RayBB , for those detailed reviews and looking forward to any suitable tasks you have for me. :)

RayBB · 2026-03-24T04:37:45Z

@Sanket17052006 would you like to migrate the public_observations endpoint as well now?

Sanket17052006 force-pushed the migrate-browse-fastapi branch 2 times, most recently from 65bf9dd to 8b17920 Compare March 4, 2026 07:35

RayBB requested changes Mar 4, 2026

View reviewed changes

openlibrary/fastapi/internal/api.py Outdated Show resolved Hide resolved

RayBB self-assigned this Mar 5, 2026

github-project-automation bot added this to Ray's Project Mar 5, 2026

github-project-automation bot moved this to Waiting Review/Merge from Staff in Ray's Project Mar 5, 2026

github-actions bot added the Needs: Response Issues which require feedback from lead label Mar 7, 2026

RayBB added the On Testing label Mar 15, 2026

RayBB requested changes Mar 15, 2026

View reviewed changes

Sanket17052006 and others added 6 commits March 15, 2026 12:15

Migrate browse endpoint to FastAPI

8749ee1

[pre-commit.ci] auto fixes from pre-commit.com hooks

0141bd5

for more information, see https://pre-commit.ci

Update FastAPI route to browse.json

2a9ae13

Backup

7a0db2e

Use Pagination model, add Request, Response model

7220669

Add description, simplify response, and remove threading

47aa052

Sanket17052006 force-pushed the migrate-browse-fastapi branch from 9e50d64 to 47aa052 Compare March 15, 2026 12:17

Delete tests/integration/temporary/test_browse_migration.sh

38ac927

RayBB reviewed Mar 23, 2026

View reviewed changes

openlibrary/fastapi/internal/api.py Outdated Show resolved Hide resolved

Apply suggestion from @RayBB

ade063b

RayBB self-requested a review March 23, 2026 18:18

don't use request model for simple change

82c4c3a

RayBB approved these changes Mar 23, 2026

View reviewed changes

RayBB merged commit ba219c4 into internetarchive:master Mar 23, 2026
3 checks passed

github-project-automation bot moved this from Waiting Review/Merge from Staff to Done in Ray's Project Mar 23, 2026

RayBB mentioned this pull request Mar 23, 2026

Migrate internal endpoints to FastAPI #11999

Open

7 tasks

RayBB removed the On Testing label Mar 23, 2026

Uh oh!

Conversation

Sanket17052006 commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Stakeholders

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RayBB commented Mar 4, 2026 via email

Uh oh!

Sanket17052006 commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

RayBB Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

RayBB Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

RayBB Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

Sanket17052006 commented Mar 15, 2026

Uh oh!

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Sanket17052006 commented Mar 23, 2026

Uh oh!

RayBB commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sanket17052006 commented Mar 4, 2026 •

edited

Loading

Sanket17052006 commented Mar 6, 2026 •

edited

Loading