-
Low poly cat
What it does
It classifies 3D models into various predefined categories. To do so, the 3D model is used to generate one or more grayscale images with some rotations and translations. Afterwards, a deep learning model takes as input these grayscale images and returns an aggregated classification score.
How I built it
The project has been structured into various phases:
- Data gathering. 3D models have been downloaded using open source software for crawling thingiverse's website.
- Preprocessing. The 3D model is stored using the .tsl format. This format has to be converted into some N-array structure to be able to train a deep learning model. In consequence, we decided to generate one or more grayscale images from the 3D model by applying some rotation and translation.
- Model Training/Validation. The resulting grayscale images are then used to train a deep learning model. We implemented three models, Xception, DenseNet and MobileNetV2. All of them have been pretrained using the ImageNet dataset and the weights of the convolutionals layers are transferred to our models to pretrain our weights.
Project Structure: Crawler. Crawls thingiverse and downloads the .tls files and their categories Preprocessing Read .stl file (numpy-stl). Generate one or more rotations and translations of the 3D model (numpy-stl). Save result into rgb image Convert rgb image into grayscale image (Pillow) Datasets Small: 2 categories Medium: 5 categories Large: 11 categories Models Xception DenseNet MobileNetV2
The tools used are: thingiverse_downloader, MeshLab, Python, Numpy-STL, TensorFlow, Keras
Challenges I ran into
The extraction process has been exhausting and time-consuming. The time needed to generate the grayscale images far exceeded our initial schedule.
Accomplishments that I'm proud of
- Develop a complete suite of tools to automatize the training of images from a 3D model download site.
Log in or sign up for Devpost to join the conversation.