Vinayak Mehta

How to build a notebook-based data platform

Vinayak Mehta — Sun, 19 Jan 2025 15:00:00 +0530

This post is based on my JupyterCon 2020 talk. I decided to write it 5 years later, because I think that notebook-based data platforms are relevant, now more than ever as AI helps us do more things, to enable citizen data science and seamless collaboration in large orgs. It's a …

Cerfs-volants dans le ciel de Delhi

Vinayak Mehta — Sun, 12 Jan 2025 15:00:00 +0530

J'ai besoin de présenter un exposé sur un sujet peu connu de mon pays dans mon cours de français cette semaine. J'ai choisi comme sujet : le vol des cerfs-volants dans le ciel pour célébrer la fête de l'indépendance à Delhi, d'où je viens. J'ai essayé de faire des recherches mais …

IPychat — An AI extension for IPython to make it work like Cursor

Vinayak Mehta — Thu, 02 Jan 2025 00:00:00 +0530

Recently, I've been exploring geospatial data about Paris. Although I've seen others work with this type of data and know basic terms like shapefiles and polygons, this is my first time working with it myself. I've been looking up terms and going through docs to learn how various geospatial data …

Running goals on my TRMNL

Vinayak Mehta — Sun, 15 Dec 2024 00:00:00 +0530

Last month, I saw Ferruccio with a TRMNL and it picqued my interest because I'd been thinking about putting some things that I always wanted top of mind right in front of me. For example, my running goals that I track using the Strava widget on my phone.

After some …

Houseplant: Database Migrations for ClickHouse

Vinayak Mehta — Tue, 03 Dec 2024 00:00:00 +0530

Originally published on the June blog.

At June, we use ClickHouse as our analytics database to process billions of user events and generate real-time insights for our customers. While ClickHouse is incredibly powerful for analytical workloads, managing database schema changes has been a pain in our dev workflow.

Unlike traditional …

First sea swim

Vinayak Mehta — Sun, 31 Jul 2022 00:00:00 +0530

Cap d'Antibes

In May 2022, I swam in the sea for the first time in my life. Before that, I'd only swam in pools (except for that one time in a piscine naturelle in Nongriat), so this experience was both exhilarating and scary at the same time.

Exhilarating because I …

Releasing Camelot v0.10.0

Vinayak Mehta — Mon, 12 Jul 2021 05:00:00 +0530

I'm happy to announce that Camelot v0.10.0 is out!

tl;dr

You can now choose between two image conversion backends, or supply your own.
pip install camelot-py[base] instead of pip install camelot-py[cv]

Background

Camelot uses ghostscript to convert a PDF page into a PNG so that …

How to create a desktop entry for an AppImage

Vinayak Mehta — Tue, 09 Feb 2021 05:00:00 +0530

I've been using Fraidycat as an RSS reader for some time. The project puts out an AppImage as a release asset, which is a neat way for distributing software on Linux because it does not need root privileges to install things in system directories.

After making a Software.AppImage file …

Day 60 — NEVER GRADUATE!

Vinayak Mehta — Mon, 02 Nov 2020 23:59:00 +0530

Friday was the last day of my batch :(

I've been writing these posts a day or two after the actual day, but this one is slightly late because my laptop charger gave up on me while I was hanging out with Edith and Adam after the Halloween party on Friday …

Day 59 — The littlest Jupyter console in Rust!

Vinayak Mehta — Sun, 01 Nov 2020 23:55:00 +0530

Today I continued working on translating cutypr to Rust.

I was able to serialize and sign messages, and send them to the kernel, but the kernel kept saying that the message has an invalid signature :( I found it difficult to debug and check if the (message signature) bytes sent from …

Day 58 — I miss the Python standard library

Vinayak Mehta — Sun, 01 Nov 2020 23:51:00 +0530

Today I worked on translating cutypr to Rust.

I found that I could use pyo3 to start the IPython kernel from Rust! So I wrote a start_kernel() Python function to start the IPython kernel, returned some useful information (the key required to sign messages + ports for all kernel channels) from …

Day 57 — cutypr: an even littler Jupyter console

Vinayak Mehta — Wed, 28 Oct 2020 23:59:00 +0530

Today I read the first chapter of the zeromq guide and looked at the client-server and pub-sub examples! I love this paragraph from the preface (sounds so similar to Derek Sivers's CD Baby email!):

We took a normal TCP socket, injected it with a mix of radioactive isotopes stolen from …

Day 56 — The littlest Jupyter console

Vinayak Mehta — Tue, 27 Oct 2020 23:59:00 +0530

This week I'm planning to build a terminal frontend for Jupyter (in Rust!) so today I started looking into how Jupyter works!

I found this doc which shows how messaging works in Jupyter. When we start a jupyter_console (like IPython), it starts a Python kernel in the "backend".

The kernel …

Day 55 — Moar Rust reading!

Vinayak Mehta — Mon, 26 Oct 2020 23:55:00 +0530

Today I read up on structs, enums, and pattern matching in the Rust book. I also did some rustlings, and read a lot of Rust blog posts to figure out what I should work on in my last week at RC! I want to build a small project to get …

Day 54 — Rust ownership and Python garbage collection

Vinayak Mehta — Sun, 25 Oct 2020 23:55:00 +0530

Today I read the chapter on ownership in the Rust book. In this post, I'll try to summarize what I learned for future me!

Ownership in Rust

Ownership is Rust's central feature and the chapter explains how it allows Rust to make memory safety guarantees without needing a garbage collector …

Day 53 — A tool to list PE file dependencies

Vinayak Mehta — Fri, 23 Oct 2020 23:55:00 +0530

Today I read up on some numpy docs to try and find how they bundle the OpenBLAS DLL with the numpy wheel. I've updated the day 51 post with the things that I found.

I also packaged the find_dll_dependencies function from wheel_repair.py into a convenient CLI tool which can …

Day 52 — Bundling DLLs with Windows wheels (the DLL mangling way)

Vinayak Mehta — Thu, 22 Oct 2020 18:00:00 +0530

Yesterday, after reading about the DLL Search Order on Windows and DLL Hell, I got concerned about shipping DLLs as package_data, because they might clash with other DLLs with same names that are shipped by other Windows wheels.

This is the gist of the whole issue: If a DLL with …

Day 51 — Bundling DLLs with Windows wheels (the package_data way)

Vinayak Mehta — Wed, 21 Oct 2020 23:55:00 +0530

Last week, I was able to build pdftopng on Windows, but the extension worked only when all the DLLs it depended on were kept in the same directory as the PYD (which is basically a DLL).

At the end of that post, I had some questions around bundling DLLs with …

Day 50 — `pdftopng` works!

Vinayak Mehta — Tue, 20 Oct 2020 23:51:00 +0530

Today I created a test PR to replace ghostscript with pdftopng, and most tests passed! The ones that failed have coordinates hardcoded in them. I'll just need to add a small tolerance in that float comparison and everything should be good to go! Not quite, I need to build Windows …

Day 49 — JupyterCon!

Vinayak Mehta — Sat, 17 Oct 2020 23:59:00 +0530

For the past 4 days, I've been spending some time at JupyterCon! It's been great listening to talks and looking at all the things people are building with Jupyter! All of the chat has been happening on the JupyterCon Mattermost instance. Thank you to the organizers for all their work …

Day 48 — Curlyboi walks into the Cheese Shop

Vinayak Mehta — Sat, 17 Oct 2020 23:55:00 +0530

I made Linux and MacOS wheels for curlyboi and uploaded them to PyPI! You can now install curlyboi using:


  $ pip install curlyboi

Since I used pybind11 (which works with C++ code) to wrap curlyboi, I had to rename all the .c files to .cpp.

I ran into an error when …

Day 47 — What? Snek is evolving! Congratulations! Your snek evolved into Curlyboi!

Vinayak Mehta — Sat, 17 Oct 2020 23:51:00 +0530

Recently I found this snake game by Julia Evans and thought that it might be fun to go through it to learn more about large-ish C projects and ncurses! I also paired with Edith (who is writing her own text editor from scratch!) to walk through the code and understand …

Day 46 — Oh no! A bug :(

Vinayak Mehta — Thu, 15 Oct 2020 23:59:00 +0530

Today I read through some open issues on camelot, and found a bug for when you install it from conda-forge. I'd assumed that installing ghostscript from conda-forge installs all of its dependencies. It does, but looks like all the those depedencies are statically linked into one gs executable.

This would've …

Day 45b — How to (almost) build a C extension wheel on Windows (with external dependencies)

Vinayak Mehta — Wed, 14 Oct 2020 20:30:00 +0530

I looked into how to build C extension wheels on Windows over the weekend. Since there isn't a fastmac equivalent to get a Windows machine for debugging, I booted up Windows on my laptop after a really long time! I need to find a fastwin or winfast!

Installation

Visual Studio …

Day 45 — I have Linux and macOS wheels!

Vinayak Mehta — Tue, 13 Oct 2020 23:55:00 +0530

Today I continued my quest of packaging my Python C extension for multiple OSes. Yesterday while doing a packaging "test run" with the curses "hello world" program, I found that curses is not supported on Windows (it should work with WSL, but not with the "default" Windows terminal I guess …

Day 44 — Packaging Driven Development

Vinayak Mehta — Mon, 12 Oct 2020 23:55:00 +0530

Today I started looking into building cross-platform wheels for my Python C extension. I've never built binary extension wheels so I decided to start simple by building wheels for this snake game I want to write in C. Right now it's just a "hello world" ncurses program.

I started looking …

Day 43 — Mysterious PEPs and where to find them

Vinayak Mehta — Wed, 07 Oct 2020 23:59:00 +0530

I've been reading (and forgetting about) a lot of PEPs lately. When you read a PEP, it mentions how "this" thing was defined in "this" PEP, and how "that" thing was defined in "this other" PEP. So I thought that it would be cool to look at all the PEPs …

Day 42 — I have an answer to the ultimate question of how to convert a PDF to a PNG in Python!

Vinayak Mehta — Wed, 07 Oct 2020 23:55:00 +0530

Yesterday I paired with Ilia to walk through the pdftoppm code, and we identified some code that wasn't required to wrap pdftoppm. I also wrapped a "hello world" ncurses game with pybind11!

Today I started looking into pdftoppm again and commented some code (like command-line argument parsing and JPEG support …

Day 41 — Exploring pybind11 with a snek

Vinayak Mehta — Tue, 06 Oct 2020 23:55:00 +0530

Today I paired with Ilia to walk through the poppler codebase. This was of great help as I don't have a lot of experience navigating through large C/C++ codebases. I got to learn about how cmake and C++ preprocessor directives work!

We also walked through the pdftoppm code to …

Day 40 — Playing with poppler utils

Vinayak Mehta — Mon, 05 Oct 2020 23:55:00 +0530

Today I replaced some ghostscript code in camelot with a pdftoppm subprocess call just to see if the tests pass. They did! I think poppler could be the ghostscript alternative that I'm looking for!


  pdftoppm -r 300 -png -singlefile foo.pdf foo

That got me all excited to look into …

Day 39 — manylinux is awesome!

Vinayak Mehta — Mon, 05 Oct 2020 22:00:00 +0530

Today I watched these awesome talks on wheel building by Elana Hashman and Paul Kehrer. I also read PEPs 513, 571, 599, and 600 where I learned about manylinux, and how it solved a problem I used to face a lot! It's the problem of installing Python packages that contain …

Day 38 — What's inside a Python wheel?

Vinayak Mehta — Sun, 04 Oct 2020 23:55:00 +0530

Today I read PEP 427 (and other PEPs mentioned in there) to learn more about Python wheel format! There are two types of distribution formats in Python - a source distribution, and a built distribution. The wheel format belongs to the second category in which things are already built, ready to …

Day 37 — A rustup doc for Python!

Vinayak Mehta — Thu, 01 Oct 2020 23:59:00 +0530

When I was trying to learn Rust (I need to get back :( ) some weeks ago, I really liked how I was able to install every tool I needed to write Rust with just one curl command! I also loved the rustup doc command which opened the docs for my installed …

Day 36 — Moar Python C extension talks!

Vinayak Mehta — Thu, 01 Oct 2020 23:55:00 +0530

Today I spent most of my time watching some talks on writing Python C extensions. Every talk mentioned that the main reasons for writing an extension are:

Interfacing with C/C++ (I'm in this camp!)
Improving performance of Python code by rewriting it in C/C++

I need to try …

Day 35 — What's inside an ELF executable? (symver edition)

Vinayak Mehta — Tue, 29 Sep 2020 23:55:00 +0530

I saw "ELF" being mentioned a lot while reading about linkers yesterday so I thought about looking into it. Turns out it's a file format for executables. Just hearing the word "executable" used to give me the chills because I couldn't look at what's inside one. (One time I ran …

Day 34 — Linkers go brrrrr

Vinayak Mehta — Mon, 28 Sep 2020 22:00:00 +0530

Continuing from the day before yesterday, I started looking into linkers and loaders. Till now, I haven't had experience writing multiple C files that work together. And up until yesterday, my only other experience with .so files has been knowing that if I put opencv's .so file in my Python …

Day 33 — There are so many LD variables!

Vinayak Mehta — Thu, 24 Sep 2020 23:55:00 +0530

Using strace to spy on ghostscript's system calls made me wonder if there's a way to print the names of all function calls (along with their file paths) that a compiled executable makes after you invoke it on the command-line. This would be awesome to navigate large C codebases …

Day 32 — A Python C extension module!

Vinayak Mehta — Wed, 23 Sep 2020 23:55:00 +0530

Today I did some Yak shaving and made opep because opening/reading each PEP in its own browser tab (with all the extra information on the webpage) was getting tiring both for me and the browser.

I'd used the man command a lot yesterday to read some manual entries, so …

Day 31 — Spying on ghostscript

Vinayak Mehta — Tue, 22 Sep 2020 23:55:00 +0530

I recently (at the start of my batch) found out about strace from this zine by Julia Evans! Today I used it to look at things ghostscript does under the hood (system calls!) to do a PDF to PNG conversion. I also read through the manual page for strace which …

Day 30 — Upgraded to Ubuntu 20.04

Vinayak Mehta — Sun, 20 Sep 2020 23:59:00 +0530

Today I spent a lot of time upgrading my Ubuntu from 18.04 to 20.04, just to get a new version of gnome-shell which was needed by this cool window manager I wanted to try out. The whole process reminded me of 2014 when I was literally trying out …

Day 29 — EasyOCR dabblements

Vinayak Mehta — Sun, 20 Sep 2020 23:55:00 +0530

Today I paired with Andrew to try out EasyOCR since we thought that it could be useful in both chronicle (to extract text out of camera images) and camelot (to extract text out of image-based PDFs). Andrew suggested that we could use images from duetocovid19.com, another cool project that …

Day 28 — A divide-and-conquer strategy for recording videos

Vinayak Mehta — Wed, 16 Sep 2020 23:59:00 +0530

This is not something new, almost every video in the world is recorded this way! At first, I fought against this strategy by doing just one take, thinking that it would save time. I couldn't be more wrong!

Disclaimer: I'm a new speaker, and I'm not very good at speaking …

Day 27 — Finished my talk!

Vinayak Mehta — Wed, 16 Sep 2020 23:55:00 +0530

Today (plus yesterday, and the whole weekend!) I focused all my energy on writing (and trying to record) my talk for JupyterCon.

Now I just need to record it.

Day 26 — How are six weeks over already?!

Vinayak Mehta — Tue, 15 Sep 2020 21:00:00 +0530

Oh no, I'm almost at the midpoint of the batch! Half of my batch is almost over :( I wish I could be in batch forever, and work on learning and building new things every day!

I took some time to reflect on the past 5 weeks. These are the goals …

Day 25 — A thing I didn't know about Unix

Vinayak Mehta — Sun, 13 Sep 2020 23:59:00 +0530

Welcome to my blog. ICYMI, I'm using it to post about how I spend each day at RC, so that I can look back upon the things I learned, and the interactions I had, things that can otherwise be so ephemeral!

This post got some replies on Zulip, which means …

Day 24 — JupyterHub and Airflow on microk8s

Vinayak Mehta — Thu, 10 Sep 2020 23:59:00 +0530

Some days ago I set up minikube to put together a small demo (or at least some screenshots) for a talk. After a minikube start, it has been taking a long time to come up for me (maybe because of my smol laptop), and it also throws some errors before …

Day 23 — Why is it called a.out?

Vinayak Mehta — Wed, 09 Sep 2020 23:59:00 +0530

tl;dr

a.out is short for "assembler output", the filename of the output of Ken Thompson's PDP-7 assembler. It remains the default output filename for executables created by compilers like gcc, even though the created files actually are not in the original a.out format! [source]

Today I paired …

Day 22 — WASM + Python = ❤️

Vinayak Mehta — Tue, 08 Sep 2020 23:59:00 +0530

Today I attended the "Intro to WebAssembly" session by Ezzeri. He talked about some useful performance benchmarks that he's been working on.

I went on to play with wasmer-python which is a WebAssembly runtime for Python! It lets you execute WASM binaries with an API similar to how you'd execute …

Day 21 — Timezones are weird

Vinayak Mehta — Mon, 07 Sep 2020 23:59:00 +0530

For the past month, I've been working in Eastern Time, while being physically located in Indian Standard Time. Which means I wake up around 12-2pm IST and go to sleep around 5-6am IST. Timezones are weird.

Over the weekend I attended PyConline AU in Australian Central Standard Time (Curlyboi Standard …

Day 20 — Make Javascript run faster with this simple trick!

Vinayak Mehta — Sun, 06 Sep 2020 23:59:00 +0530

Today I paired with Ezzeri to look at some of his Exercism Rust track solutions. The discussion moved to WebAssembly and we went through some minimal examples he has been working on. He's also written a nice blog post which has some beginner level WebAssembly resources.

The whole exercise filled …

Day 19 — I madeth a Sphinx extension

Vinayak Mehta — Thu, 03 Sep 2020 23:59:00 +0530

Today I thought that adding a gallery to display what everyone is building with present would be great! I guess this was a distraction from the talk research I previously mentioned (which I need to finish ASAP!) :(

But I learned how to write a simple Sphinx extension, which is cool …

Day 18 — Mini kubey kube, where are you?

Vinayak Mehta — Wed, 02 Sep 2020 23:59:00 +0530

I decided to set up minikube to create a small demo for a talk that I have to submit this weekend. Right now I'm going through the Phase 4 that Hynek Schlawack talks about in his article on conference speaking.

I SWEAR TO GOD I’M NEVER SUBMITTING A TALK …

Day 17 — Pair programming is fun!

Vinayak Mehta — Tue, 01 Sep 2020 23:59:00 +0530

Today I paired with Taro to walk us through the present codebase. It was a nice learning experience to talk about (and explain) how everything is structured because I don't get to do that while working alone. I need to pair program more!

After the walkthrough, we came up with …

Day 16 — What's inside a Jupyter notebook?

Vinayak Mehta — Mon, 31 Aug 2020 23:59:00 +0530

Today I was reading up on some Jupyter things, and wondered how images, widgets, and maps are stored within a notebook. I love the literate programming model where your docs, code and outputs; everything is viewable with just one file!

A Jupyter notebook is basically a Python dictionary. It has …

Day 15 — Let's play some codio

Vinayak Mehta — Fri, 28 Aug 2020 23:59:00 +0530

I'd been procrastinating on the codio pull request after recording my PyConline AU talk on that branch earlier this week. I finally pushed some fixes and merged it today! This is the current version:

You can make a codio by writing some yaml (shown below), and use it inside your …

Day 14 — I rewrote itslit in Rust!

Vinayak Mehta — Thu, 27 Aug 2020 23:59:00 +0530

Today I read through ownership, references and borrowing in Rust. I can't write about what I learned in this short post as I spent all my time playing skribbl at game night (it was really fun and I would totally do it again!). I also did a lot of rustlings …

Day 13 — Hello, Rust!

Vinayak Mehta — Wed, 26 Aug 2020 23:59:00 +0530

Today I started reading The Rust Programming Language book. The first step was to set up Rust. In one curl command, it installed everything (a compiler, a package manager, a code formatter, and more) I would need to write Rust code! This was an awesome experience, and diametrically opposite to …

Day 12 — Bugs bugs bugs

Vinayak Mehta — Tue, 25 Aug 2020 23:59:00 +0530

Today I had two nice coffee chats and tidied up my learning outline. I will dive deep into Rust and Python packaging in the next few weeks.

I also added a comment on this asciimatics issue to possibly contribute codio as a "typewriter effect"! It would be awesome to see …

Day 11 — reveal-notes + tmux + keylogger = present-notes

Vinayak Mehta — Mon, 24 Aug 2020 23:59:00 +0530

Today I spent most of my time recording the talk video for PyConline AU. I wanted to use present and since it doesn't have a speaker notes view right now, I managed to hack together a way to make the reveal.js speaker notes work with present. It worked out …

Day 10 — present can now play code!

Vinayak Mehta — Fri, 21 Aug 2020 23:59:00 +0530

Today I worked on the "pre-recorded playable code block" idea from yesterday. During a talk, it's hard for me to remember and type commands on the terminal while also explaining what I'm doing. So I usually record a gif earlier and play that instead. And since gifs don't render nicely …

Day 9 — Slow day

Vinayak Mehta — Thu, 20 Aug 2020 23:59:00 +0530

Today I couldn't get a lot done, just ended up fixing some bugs on present. I had some enhancement ideas though.

What if you can run the Python interpreter inside your slide's code block? Or even a shell prompt? What about a RICE-like view where you can execute a …

Day 8 — I made a terminal-based presentation tool!

Vinayak Mehta — Wed, 19 Aug 2020 23:59:00 +0530

For the last two days, I've been working on present.

I've been wanting to build something like this ever since I watched the North Bay Python 2017 keynote by Brandon Rhodes. The markdown slide format is similar to reveal.js because I want present to work with my existing slides …

Day 7 — Python dataclasses are awesome!

Vinayak Mehta — Tue, 18 Aug 2020 23:59:00 +0530

Today I tried dataclasses for a project (first time!). And I'm sold because I don't have to define an __init__ function anymore, and they automatically generate a repr too. Also, type annotations!


  >>> from dataclasses import dataclass
  >>>
  >>> @dataclass
      class Point:
          x: int = 0
          y: int = 0
          z: int = 0
  >>>
  >>> p = Point …

Day 6 — Some PEP talk, or: What is a pyproject.toml?

Vinayak Mehta — Mon, 17 Aug 2020 23:59:00 +0530

Today I talked about PEPs 518 and 517 at RFCs We ❤️. I learned a lot from Brett Cannon's blog posts on the topic, and from reading the PEPs themselves. This post is a simpler version of everything.

First, let's briefly look at the history of Python packaging based on the …

Day 5 — HowToVer?!

Vinayak Mehta — Sat, 15 Aug 2020 23:59:00 +0530

After reading software versioning posts by Brett Cannon, Donald Stufft, and Bernát Gábor; I thought that I should try to summarize what I learned (for future me!).

Why do we need software versioning? We need it to snapshot each point in the software evolution process, and to follow a convention …

Day 4 — Hyrum's Law

Vinayak Mehta — Thu, 13 Aug 2020 23:59:00 +0530

Today I worked on some open issues and pull requests for camelot and excalibur. Last month, pdfminer.six (one of camelot's dependencies) broke backwards compatibility by renaming the PDFTextExtractionNotAllowed exception. camelot raises it while getting the page layout if the page is not extractable. I'd added it back in …

Day 3 — Made some feta, and itslit!

Vinayak Mehta — Wed, 12 Aug 2020 23:59:00 +0530

Today I used a countdown timer to timebox my coding sessions which really helped with focus. I did this during Hack and Tell where everybody says what they're going to work on, hack on that for 2 hours, and finally tell what they worked on at the end of the …

Day 2 — Some bash and then the memex

Vinayak Mehta — Tue, 11 Aug 2020 23:59:00 +0530

For the past one month, I've been trying to adjust to Eastern Time based on the assumption that I might miss out on RC activities if I'm not up at the same time as everyone else. As a result, I've been sleeping at 6am Indian Standard Time. After spending two …

Day 1 — Virtual RC!

Vinayak Mehta — Mon, 10 Aug 2020 23:59:00 +0530

Today was my first day at RC! I've been hanging out on Zulip since last week though, mostly participating in the #games stream where we're doing a 30-day video game music challenge. I feel great nostalgia every day when I post a track from a video game I used to …

Day 0 — What I want to do at Recurse Center

Vinayak Mehta — Sun, 09 Aug 2020 23:59:00 +0530

I've been accepted to the Fall 1 batch at the Recurse Center (yay!), which starts tomorrow. RC is an educational retreat for programmers, based in New York City. It is heavily influenced by unschooling, which means that it is self-directed and everyone is free to explore and learn what they're …

The Hitchhiker's Guide to CLIs in Python

Vinayak Mehta — Mon, 04 May 2020 08:00:00 +0530

This article is based on my PyCon 2020 talk of the same title.

In the beginning, was the command line. Well, not quite the command line we know today.

We had typewriters. We were communicating using Morse code. One fine day, someone had an idea to connect a typewriter to …

The Hitchhiker's Guide to CLIs in Python — Part 4: User Experience

Vinayak Mehta — Mon, 04 May 2020 07:45:00 +0530

Now that we know how to write CLIs in Python, let's talk about the CLI user experience. As mentioned earlier, we're operating in a constrained design space. In contrast to GUIs, which offer a lot of visual cues and guidance to the user.

There are some principles that can help …

The Hitchhiker's Guide to CLIs in Python — Part 3: Writing and packaging a CLI using Click

Vinayak Mehta — Mon, 04 May 2020 07:30:00 +0530

Let's look at some common CLI use cases and how we can use click to implement them.

This time we'll use an example CLI called smol-git.


  $ smol-git --help
  Usage: smol-git [OPTIONS] COMMAND [ARGS]...

    smol-git - the stupid content tracker

  Options:
    --version  Show the version and exit.
    --help     Show this message and …

The Hitchhiker's Guide to CLIs in Python — Part 2: Python packages for writing CLIs

Vinayak Mehta — Mon, 04 May 2020 07:15:00 +0530

Let's see how we can write a command-line interface using Python. There are several options to do this, in the standard library and on PyPI.

We'll use an example CLI called smol-pip, and see how we can implement it using these different options.


  $ smol-pip install --upgrade package_name

smol-pip has one …

The Hitchhiker's Guide to CLIs in Python — Part 1: Anatomy of a Terminal and CLI

Vinayak Mehta — Mon, 04 May 2020 07:00:00 +0530

tty (short for teletype) is a Unix command that prints the name of the terminal connected to standard input. It is also the prefix in names for virtual terminals on Unix-based operating systems.

The fundamental type of application that runs on a terminal is the shell. The …

How To Fall In Love With Running

Vinayak Mehta — Tue, 15 Jan 2019 20:00:00 +0530

I ran 1094 km in 2018. For the first time in my life, I've run almost every week for an entire year. I'll run my first marathon this weekend!

Running is the easiest thing you can do for your fitness. It needs no equipment, just a pair of shoes (or …

An Open-Source Tool to Extract Tables from PDFs into CSVs

Vinayak Mehta — Mon, 26 Nov 2018 06:00:00 +0530

I have also published this post on Hacker Noon.

Borrowing the first three paragraphs from my previous blog post since they perfectly explain why extracting tables from PDFs is hard.

The PDF (Portable Document Format) was born out of The Camelot Project to create “a universal way to communicate documents …

Last Year Tonight at NH7 Weekender Meghalaya 2017

Vinayak Mehta — Wed, 31 Oct 2018 20:00:00 +0530

Last year tonight, I was in Meghalaya for NH7 Weekender with my friends (also co-workers at the time), Deepu and Rishabh (aka Jain). We'd been aching to visit Meghalaya after watching Ethereal's "Meghalaya Alive!" video. And after hearing about Weekender happening there (with Steve Vai headlining it) there was no …

Announcing Excalibur, a Web Interface to Extract Tabular Data from PDFs

Vinayak Mehta — Sun, 21 Oct 2018 03:30:00 +0530

Last week, Camelot trended at #1 on Hacker News, Github and #5 on Product Hunt. Thank you for the love! There's still a lot to do to make it more awesome. You can follow the roadmap on its Github wiki. You can also check out my previous blog post on …

Announcing Camelot, a Python Library to Extract Tabular Data from PDFs

Vinayak Mehta — Wed, 03 Oct 2018 03:30:00 +0530

I originally wrote this post for the SocialCops engineering blog, and then published it on Hacker Noon.

The PDF (Portable Document Format) was born out of The Camelot Project to create “a universal way to communicate documents across a wide variety of machine configurations, operating systems and communication networks”. Basically …

Airflow, Meta Data Engineering, and a Data Platform for the World’s Largest Democracy

Vinayak Mehta — Sat, 25 Aug 2018 00:00:00 +0530

I originally wrote this post for the SocialCops engineering blog, and then published it on Hacker Noon.

In our last post on Apache Airflow, we mentioned how it has taken the data engineering ecosystem by storm. We also talked about how we’ve been using it to move data across …

How to Create a Workflow in Apache Airflow to Track Disease Outbreaks in India

Vinayak Mehta — Mon, 18 Jun 2018 00:00:00 +0530

I originally wrote this post for the SocialCops engineering blog, and then published it on Hacker Noon.

What is the first thing that comes to your mind upon hearing the word ‘Airflow’? Data engineering, right? For good reason, I suppose. You are likely to find Airflow mentioned in every other …

Starry Night at Inchhapuri

Vinayak Mehta — Wed, 16 Sep 2015 00:00:00 +0530

I'm a Star Trek fan. Space has always fascinated me. So when I saw a friend reading this, I just had to ask him to lend it to me. Nice book BTW.

And after watching Cosmos: A Spacetime Odyssey, I've been looking for ways to dive into astronomy. I started …

Summarize it!

Vinayak Mehta — Fri, 26 Jun 2015 00:00:00 +0530

End-term exams got over on 9th. I've been looking for an internship ever since (if you're reading this and looking for a machine learning intern, please contact me). Also, I've been going to hackathons! Went to AngelHack Delhi this weekend. I had formed the team a week back. Yask: my …

21 revolutions around a G-type main-sequence star

Vinayak Mehta — Tue, 31 Mar 2015 00:00:00 +0530

You would've guessed from the title. It's my birthday. I've completed 21 revolutions around a yellow dwarf on this bounded spaceship we call Earth. I wanted to talk about something deep in this post, another time maybe. Let's listen to Coldplay instead. :)

Some handy git commands

Vinayak Mehta — Sun, 29 Mar 2015 20:30:00 +0530

Man, I haven't blogged in a long time! I don't have any particular excuse for that :| So I'm gonna try and write a random post in order to start again, and try to blog frequently.

These are some git commands I found useful in the past two months of contributing …

BVCOE Programming Club

Vinayak Mehta — Sun, 31 Aug 2014 14:30:00 +0530

I think BVCOE really needs a programming club where people can get to know about upcoming programming contests and participate in discussions.

So, I made it here on Facebook. I think the group should have people just from BVCOE and yeah, no spamming allowed.

End Terms over!

Vinayak Mehta — Thu, 12 Jun 2014 22:30:00 +0530

Ah! This post is due for a long time now. So my end term exams just ended yesterday, whereas my friends at other colleges got rid of them by the end of April. Y U DO THIS, GUGSEEPOO?

But no respite for me, as my college has this compulsory summer …

Vi IMproved

Vinayak Mehta — Thu, 06 Mar 2014 20:30:00 +0530

Vim is a text editor written by Bram Moolenaar and first released publicly in 1991. Based on vi editor common to Unix-like systems, it is designed for use both from a CLI and as a standalone application in a GUI. However, I prefer the command line interface. It is my …

Now Running Arch Linux

Vinayak Mehta — Fri, 28 Feb 2014 16:30:00 +0530

After using Ubuntu for a year, I decided to install Arch Linux on my laptop. This post by Jai Luthra got me all excited about Arch. It took me a whole weekend to do it and I've been using it for a month now. I initially thought of using awesome-wm …

First!

Vinayak Mehta — Wed, 12 Feb 2014 02:40:00 +0530

The customary first post. I will write frequently from now on, I guess. Move along.