API

One neat use case for the HTTP ETag header is client-side HTTP caching for GET requests. Along with the ETag header, the caching workflow requires you to fiddle with other conditional HTTP headers like If-Match or If-None-Match. However, their interaction can feel a bit confusing at times. Every time I need to tackle this, I end up spending some time browsing through the relevant MDN docs on ETag, If-Match, and If-None-Match to jog my memory. At this point, I’ve done it enough times to justify spending the time to write this. ...

Ever since Rob Pike published the text on the functional options pattern, there’s been no shortage of blogs, talks, or comments on how it improves or obfuscates configuration ergonomics. While the necessity of such a pattern is quite evident in a language that lacks default arguments in functions, more often than not, it needlessly complicates things. The situation gets worse if you have to maintain a public API where multiple configurations are controlled in this manner. ...

Suppose, you have a function that takes an option struct and a message as input. Then it stylizes the message according to the option fields and prints it. What’s the most sensible API you can offer for users to configure your function? Observe: // app/src package src // Option struct type Style struct { Fg string // ANSI escape codes for foreground color Bg string // Background color } // Display the message according to Style func Display(s *Style, msg string) {} In the src package, the function Display takes a pointer to a Style instance and a msg string as parameters. Then it decorates the msg and prints it according to the style specified in the option struct. In the wild, I’ve seen 3 main ways to write APIs that let users configure options: ...

I love Go’s implicit interfaces. While convenient, they can also introduce subtle bugs unless you’re careful. Types expected to conform to certain interfaces can fluidly add or remove methods. The compiler will only complain if an identifier anticipates an interface, but is passed a type that doesn’t implement that interface. This can be problematic if you need to export types that are required to implement specific interfaces as part of their API contract. ...

Recently, I purchased a domain for this blog and migrated the content from rednafi.github.io to rednafi.com. This turned out to be a much bigger hassle than I originally thought it’d be, mostly because, despite setting redirection for almost all the URLs from the previous domain to the new one and submitting the new sitemap.xml to the Search Console, Google kept indexing the older domain. To make things worse, the search engine selected the previous domain as canonical, and no amount of manual requests were changing the status in the last 30 days. Strangely, I didn’t encounter this issue with Bing, as it reindexed the new site within a week after I submitted the sitemap file via their webmaster panel. ...

While working with GitHub webhooks, I discovered a common webhook security pattern a receiver can adopt to verify that the incoming webhooks are indeed arriving from GitHub; not from some miscreant trying to carry out a man-in-the-middle attack. After some amount of digging, I found that it’s quite a common practice that many other webhook services employ as well. Also, check out how Sentry handles webhook verification. Moreover, GitHub’s documentation demonstrates the pattern in Ruby. So I thought it’d be a good idea to translate that into Python in a more platform-agnostic manner. The core idea of the pattern goes as follows: ...

At my workplace, we have a large Django monolith that powers the main website and works as the primary REST API server at the same time. We use Django Rest Framework (DRF) to build and serve the API endpoints. This means, whenever there’s an error, based on the incoming request header - we’ve to return different formats of error responses to the website and API users. The default DRF configuration returns a JSON response when the system experiences an HTTP 400 (bad request) error. However, the server returns an HTML error page to the API users whenever HTTP 403 (forbidden), HTTP 404 (not found), or HTTP 500 (internal server error) occurs. This is suboptimal; JSON APIs should never return HTML text whenever something goes wrong. On the other hand, the website needs those error text to appear accordingly. ...

I was working on a DRF POST API endpoint where the consumer is expected to add a URL containing a PDF file and the system would then download the file and save it to an S3 bucket. While this sounds quite straightforward, there’s one big issue. Before I started working on it, the core logic looked like this: # src.py from __future__ import annoatations from urllib.request import urlopen import tempfile from shutil import copyfileobj def save_to_s3(src_url: str, dest_url: str) -> None: with tempfile.NamedTemporaryFile() as file: with urlopen(src_url) as response: # This stdlib function saves the content of the file # in 'file'. copyfileobj(response, file) # Logic to save file in s3. _save_to_s3(des_url) if __name__ == "__main__": save_to_s3( "https://citeseerx.ist.psu.edu/viewdoc/download?" "doi=10.1.1.92.4846&rep=rep1&type=pdf", "https://s3-url.com", ) In the above snippet, there’s no guardrail against how large the target file can be. You could bring the entire server down to its knees by posting a link to a ginormous file. The server would be busy downloading the file and keep consuming resources. ...

While working with microservices in Python, a common pattern that I see is - the usage of dynamically filled dictionaries as payloads of REST APIs or message queues. To understand what I mean by this, consider the following example: # src.py from __future__ import annotations import json from typing import Any import redis # Do a pip install. def get_payload() -> dict[str, Any]: """Get the 'zoo' payload containing animal names and attributes.""" payload = {"name": "awesome_zoo", "animals": []} names = ("wolf", "snake", "ostrich") attributes = ( {"family": "Canidae", "genus": "Canis", "is_mammal": True}, {"family": "Viperidae", "genus": "Boas", "is_mammal": False}, ) for name, attr in zip(names, attributes): payload["animals"].append( # type: ignore {"name": name, "attribute": attr}, ) return payload def save_to_cache(payload: dict[str, Any]) -> None: # You'll need to spin up a Redis db before instantiating # a connection here. r = redis.Redis() print("Saving to cache...") r.set(f"zoo:{payload['name']}", json.dumps(payload)) if __name__ == "__main__": payload = get_payload() save_to_cache(payload) Here, the get_payload function constructs a payload that gets stored in a Redis DB in the save_to_cache function. The get_payload function returns a dict that denotes a contrived payload containing the data of an imaginary zoo. To execute the above snippet, you’ll need to spin up a Redis database first. You can use Docker to do so. Install and configure Docker on your system and run: ...

Django Rest Framework exposes a neat hook to customize the response payload of your API when errors occur. I was going through Microsoft’s REST API guideline and wanted to make the error response of my APIs more uniform and somewhat similar to this example. I’ll use a modified version of the quickstart example in the DRF docs to show how to achieve that. Also, we’ll need a POST API to demonstrate the changes better. Here’s the same example with the added POST API. Place this code in the project’s urls.py file. ...

Updated on 2023-09-11: Fix broken URLs. Recently, I was working with Mapbox’s Route optimization API. It tries to solve the traveling salesman problem where you provide the API with coordinates of multiple places and it returns a duration-optimized route between those locations. This is a perfect usecase where Redis caching can come handy. Redis is a fast and lightweight in-memory database with additional persistence options; making it a perfect candidate for the task at hand. Here, caching can save you from making redundant API requests and also, it can dramatically improve the response time as well. ...