teXnology | Devpost

Inspiration

For any STEM major, writing with LaTeX is a must. There's no other tool as powerful for formatting documents, but with the steep learning curve and complex commands, it's incredibly frustrating to learn. We wanted a tool that would allow people to use LaTeX without necessarily knowing LaTeX, making it even more accessible by using speech to text. This gives people more time to research as opposed to formatting their research on a hard-to-use software. This is where teXnology comes in.

What it does

teXnology allows users to turn spoken equations into LaTeX, greatly streamlining the process of typing scientific documents. Instead of having to constantly scan the docs and looking for the proper syntax, you can read a sentence to teXnology which does the hard work for you. It gives you a preview of the actual LaTeX code and renders it side by side to show you what it would actually look like. It also gives users the option to render the LaTeX in Overleaf, letting them maintain their workflow.

How we built it

We built the frontend using React and Node, using a built in React package for the speech to text. The text was converted to LaTeX using Groq's chat completion API, which we wrote in JS and set up an API for using Express.

Challenges we ran into

There were two major challenges we faced: speech to text and PDF rendering. For speech to text we had two options. The first was using a React component, which simplified the code and showed a live transcript, but was somewhat inaccurate. The second was using Groq's whisper Open AI API, which was very accurate but couldn't generate a live transcript without constant API calls, and also needed an audio file to be sent to backend which was a huge headache. We eventually opted to use the React component, but something we definitely want to touch up in the future. We also wanted to have a PDF rendered in the browser that would show a more accurate depiction of the final LaTeX doc. After spending hours reading through documentation changing the entire backend to Javascript (our first draft was in Python), we opted on just the Overleaf API. We realized that our solution was more focused on generating the LaTeX, and it made more sense to use an existing tool for PDF rendering rather than writing our own.

Accomplishments that we're proud of

Since we're all first-time hackers, we're proud of making a fully functional full-stack application in the 36 hours. We had very minimal experience with web dev and learned a lot of it on the fly, so we're proud to have been able to tie it all together and make a website we're genuinely proud of. We're also very happy that we got the text to LaTeX model to work with relatively good accuracy, thanks in large to Groq. This is a tool we intend to expand on greatly in the future, so we're very happy to have established such a strong foundation.

What we learned

We learned A LOT about full stack development. We had to work on tasks we've never done before (like backend for me), so we really rounded ourselves out We also learned a lot about prompt engineering and using an LLM for a more niche task, and were really impressed by the capabilities of Groq. Testing our speech to LaTeX forced us to learn the intricacies of LaTeX so our users don't have to. We learned how to integrate and implement API implementations, specifically speech to text, and also how to set up our own API endpoints for using the Groq model.

What's next for teXnology

Our goal for teXnology is to hone in on what makes it unique--the speech to LaTeX. Rather than making our own LaTeX rendering, we want to integrate these features into an extension so users can simultaneously use *teXnology with apps like Overleaf, using existing tools and allowing them to maintain their workflow. That said, we still want to keep the teXnology website if users prefer that, but also add a built in PDF viewer that renders the LaTeX. We're also looking into fine tuning the model for higher accuracy with more complex equations.