Training a Custom Object Detector in Half a Day with YOLOv8

The most efficient way to build a powerful KTP detector.

As part of the continuous effort to improve our Indonesian Identity Card (KTP) OCR service, I wanted to find a replacement algorithm for the ID Card Detector. I’ve covered this in a previous post, where the card detector is used to crop out the relevant part of the image so that the OCR doesn’t get confused with non-relevant text that might be present in the scene.

However, there were a couple of deployment-related issues with this:

1. For the implementation, we relied on TensorFlow, while the OCRs that we were using (EasyOCR and DocTR) were both PyTorch. This introduced an additional 1 GB in terms of dependencies, which made the building times frustratingly long.

2. Getting the two libraries to play nice was not a trivial affair, and we’ve hit production issues that made things quite hard to debug.

3. The Card Detector still depended on a really old version of TensorFlow (version 1.15).

I dug around in GitHub but couldn’t find anyone who trained an object detector, and, in PyTorch. No dice when trawling around Hugging Face, either. Hence, a self-experiment ensued.

In the post, I’ll show you how I trained a KTP Detector from scratch using YOLOv8. I’ll happily assume that you don’t have much Deep Learning knowledge because you don’t need to. Still, you do need to know some basic Machine Learning, but I’ll explain things as we go along.

But before we go any further, just to prove that the card detector works:

Step 1: Gather the Data

The first inclination I had was to search for an existing dataset. Thankfully, there’s the MIDV-500. From the site’s description:

MIDV-500 consists of 500 video clips for 50 different identity document types including 17 ID cards, 14 passports, 13 driving licences and 6 other identity documents of different countries with ground truth which allows to perform research in…