DigiText | Devpost

About Page.
Generate Page.

Inspiration

Despite the popularity of handwritten notes, both from students and instructors, it is impossible to deny the advantages of digital notes such as their ease of sharing with other academic peers, compatibility with accessibility-focused tools like screen readers, and standardized readability across a wide range of devices. However, digital notetaking through a keyboard requires users to memorize several APIs to maximize their effectiveness, and digital tablets may not be available to the majority of students. While it is much faster than handwriting, the skill floor required can be quite challenging for a majority of users. So, what if there was a tool that allows both students and instructors to utilize the benefits of digital notes while maintaining their habit of handwritten notes? Regardless of whether students want to save their handwritten notes across multiple devices in an easy-to-read format, or whether instructors want to easily upload their math equations for their students in a clear format, this tool aims to make digital notes more accessible for everyone, regardless of its source.

What it does

DigiText is capable of converting pictures of handwritten notes and recorded audio from lectures into Markdown + KaTex syntax for use within popular digital note-taking apps such as Joplin and Obsidian. Markdown is a versatile and easy-to-use syntax for text-formatting, while KaTex is a modified version of LaTex used for web-based renderings of mathematical notations. The combination of these two syntaxes combines ease of formatting with the capability of representing mathematical expressions, hence being extremely useful to STEM students. By incorporating the Gemini API, the app allows for any form of handwritten notes and speech to be converted into this versatile notetaking format, increasing accessibility towards these powerful digital notetaking tools.

How we built it

Using React, Vite, and Gemini, the team developed a frontend application that is capable of processing images and audio recordings into a Markdown + KaTex format. Once the Gemini output processes the initial file, the built-in rendering system will automatically display a preview of the notes to confirm the accuracy of translation. With a high rate of accuracy, only minor modifications are needed at most to complete the transition from physical to digital. The raw Markdown + KaTex interpretation can be directly copied and pasted into popular digital note-taking applications such as Joplin and Obsidian.

Challenges we ran into

Several iterations were required to improve the prompt and parameters of the Gemini model until it produced a deterministic output that fits with the intended vision of the hybrid syntax output. In addition, rendering the syntax requires using the Gemini model's output as another input for a custom rendering system.

Accomplishments that we're proud of

Our completed rendering system and file processing capabilities for multiple file formats are major achievements for the scope of this project, extending far beyond our original vision to transform this app into a versatile tool for a wider audience.

What we learned

We learned about frontend development, file processing, API calls, prompting AI models, AI model parameters, parsing, and incorporating custom React blocks.

What's next for DigiText

The team plans to integrate these capabilities as integrated plugins for Joplin and/or Obsidian directly. Given the growing popularity of these digital note-taking apps with a plethora of features for all fields, such a plugin can fit naturally within the everyday workflow of everyone.