LABuddy | Devpost

Inspiration

As many research scientists and engineers know well, an hour in the library can save a week in the lab. As the stakes of the experiment increase, whether assaying expensive proteins or integrating sensors within animal models, the preparation and knowledge required for rigorous and ethical data collection skyrocket. In the wet-lab, scientists are pressured to memorize their protocols and lose the theory behind the design of the experiment, leading to improper data collection and uninformed decisions in tight situations. When the time-consuming pre-research is not done properly, these errors can propagate throughout a discipline, leading to improperly-drawn conclusions and even retroactively falsified data.

While designing and implementing a solution, we found inspiration in the common lab struggles faced by researchers. When their protocols are not specific enough, researchers don’t have the luxury of pausing their time-sensitive experiments to clarify the issue. When simultaneously keeping track of eight experimental groups, researchers don’t have live-time access to individual theory-backed predictions. When trying to keep notes or collect data, researchers seldom have the ability to put down their pipettes or glassware for a pencil, resulting in loss of vital thoughts. How can one tool mitigate these issues? You’ll see soon.

What it does

LABuddy is built to enhance research productivity. It is an AI-driven augmented reality system to refine a researcher’s thought process and integrate it with experimental protocols to ensure proper data collection and informed decisions in the lab. Feature include: Protocol analysis based on models trained on the researcher’s literature review Dual view of lab environment and virtual experimental protocol platform Basic image retrieval from the internet Voice-driven note-taking capacity linked to Samsung Notes

LABuddy is also a versatile tool for research education in a wet lab setting. In the research setting, understanding the experiment is perhaps more important than performing them. Reviewing concepts while performing experiments is vital in formulating critical analysis and connecting the theory to the data. Using the capabilities described earlier, undergraduate students can reinforce unfamiliar portions of the protocol and develop the ability to ask relevant questions to the trained model. Additionally, uploading references, protocols, lab guides, and safety documents to LABuddy encourages students to review material prior to entering the lab environment, where many factors are out of their control. Once in the lab, students can access theory and concepts on demand - all while performing their experiments.

How we built it

We utilized a variety of different technologies such as Android Studio, CloudFlare, SerpApi, and AI models. During the span of the hackathon we explored multiple different APIs such as DynamicPDF API and an API to write to a spreadsheet. These APIs are all designed to augment a person’s experience when experimenting by reducing the time needed to find documentation. Android Studio was used as the interface to communicate between the user and the Rokid glasses using audio commands. We chose Android Studio since there was limited support for development platforms. Additionally, we used CloudFlare to host the server that would perform the back-end services such sending back a text response or an image depending on the user’s query. The text response generated by the CloudFlare hosted AI model was simple to develop with, performant, and yielded fast response times compared to hosting an AI model locally. SerpApi was used to perform searches on the internet to hopefully yield the optimal image that the user wished for. This project is an amalgamation of different technologies spanning multiple different platforms that led to a product that is very responsive in a time-critical environment.

Challenges we ran into

We encountered multiple issues when developing this project. For instance, we encountered timeliness issues when developing the back-end services, especially when trying to host an AI model on a laptop locally. Although the responses were of good quality, the AI model generated results oftentimes took minutes to complete a simple request by the user. We addressed this issue by scouring our available resources and investigating any means of hosting an AI model on the cloud. We eventually found CloudFlare and were able to quickly develop a solution that significantly reduced the model response time to a few seconds.

Accomplishments that we're proud of

Over the span of 2 days, we’ve developed an app that offers a comprehensive set of features that aim to shift time-intensive manual labor to a capable AI model (equipped with the context and knowledge of the research papers) to reduce the mental and physical burden imposed on a human. In the process of doing so, we learned about new technologies. In particular, we learned and leveraged the unique qualities of the Rokid glasses’ microphones and display capabilities when developing with it in Android Studio due to the limited documentation regarding this product. Additionally, this was the first hackathon where our team leveraged CloudFlare and were extremely happy with the extremely responsive interface the cloud-based server could provide for our back-end services.

What we learned

While developing this project we developed using various software tools and platforms such as CloudFlare, SerpApi, and AI models (deepseek and llama). Additionally, we advanced our skills regarding Android Studio in order to create a more cohesive interface between the limited support for the Rokid glasses. In particular, we learned how to create a service worker on CloudFlare to support all of our server-side needs and were able to host AI models conveniently and be able to experiment with multiple models. In general, given the wide background of our team, we were able to learn a lot about web-development and cloud-computing in a short amount of time!

What's next for LABuddy

The functionality of LABuddy can be enhanced by several important additions: Developing a context specific model based on the classification of the PDF file input (i.e. varied responses when drawing from protocols, primary research articles, and safety documentation). This will improve model response quality, and has already been implemented in a Python prototype, but due to hosting issues, we have not been able to deploy this model. Providing capability to report references for the output information so that the researcher can gauge the relevance of the output. Providing more intuitive and simple voice commands to allow the user to log testing data in a spreadsheet through voice commands.

Critically, receiving further stakeholder input will allow LABuddy to reach its full potential. Ultimately, this product is designed for researchers from a variety of disciplines, so it will be important to individualize the product to combat the niche struggles, such as standardizing reagent tracking and managing large simulation datasets.