-
-
an example doodle converted into its relevant object in real time.
-
interface of colorbook
-
interface of colorbook
-
support of gallery and save ,delete options
Inspiration
We wanted to bridge the gap between human creativity and machine learning by making hand-drawn doodles recognizable by AI. Seeing the rise of AI-powered tools, we aimed to develop a fun, interactive system for sketch recognition accessible via web.
What it does
Our app allows users to draw freehand sketches which are instantly recognized and classified into objects by cutting-edge AI models. It makes doodling meaningful by providing real-time feedback on what the sketch represents.
How we built it
We used React and Vite for a fast and responsive frontend UI. The AI backend leverages Hugging Face's Inference API to run large vision transformer models. Images drawn by the user are sent to the API, which returns prediction labels displayed live.
Challenges we ran into
Finding a high-accuracy doodle recognition model deployable via API was difficult.
Integrating real-time drawing input with backend AI predictions required smooth syncing.
Optimizing performance to reduce latency while maintaining user interaction flow.
Accomplishments that we're proud of
Successfully integrated advanced vision transformer models with real-time sketch recognition.
Developed a user-friendly interface that encourages creativity and learning.
Achieved decent accuracy on freehand doodles using publicly available AI models
What we learned
We gained hands-on experience with AI model inference APIs, React frontend design, and cloud integration. We explored model limitations and realized the need for custom fine-tuning for niche tasks like doodle recognition.
What's next for colorbook:
Improve model accuracy through fine-tuning on specialized doodle datasets.
Add features for sketch-to-image generation using generative AI.
Expand to mobile platforms and support multi-touch sketch inputs.
Build a larger dataset by crowdsourcing user doodles for continuous learning.
