Gemiscript.ai

It’s just a google doc. 📄

Or is it? 🤔

What if it makes everything hands-free, and is the future of AR? 🚀👓

Emphasis on our usage of google projects, and our combination of them Used google, In google, combined google, for google

Inspiration

• Google Workspace
• Notebook LLM
• Google Docs
• Gemini
• Notion 🌐

Problem

• User interaction with Gemini AI through speech in Google Docs is difficult, and must be changed for everyone, and especially for AR. 🗣️👥
• Google Docs tools do not have as much of an emphasis on inclusivity. For example, users with dyslexia may comprehend with better fonts and grammar changes, and making that interaction much easier is a necessity. 🧠📚

What it does

• "Copilot for google docs" but with a speech emphasis
• Determines command versus normal text by classification, then takes action. ✅
• Continuous grammar changes on time interval, font, and size action taking, command AI editing. 🕒📝
• Allows speech processing with AI! 🗣️💻
• Allows more user-friendly interaction for text, which is necessary for the inclusivity of these products. For example, users with dyslexia can more easily change the text to fonts that are easier to read, and have automatic grammar corrections. 👓🔤
• In the future of AR, people will speak to text, and Gemiscript allows for easy text formatting, grammar fixes, and AI text enhancement. 🗣️➡️📝

How we built it

• Google app scripts - Script to deploy web-app for python -> document, scripts for continuous hands-free time-based grammar editing, size and font changes. 📜🔧
• Python 🐍 --Scripts to determine command versus normal text, and take action on the classification.
• Gemini API - to recognize commands versus normal text, directly in docs. 🤖📊
• Google Docs API - interacting with google doc!

Challenges we ran into

• The text wouldn’t correct itself unless user interacted - so we introduced a time interval. ⏲️🔄
• The text wouldn’t replace after commands - so we developed code specific to remove commands. 🛠️❌
• Gemini 1.5 Pro Rate Limit! 😡😡😡

Accomplishments that we're proud of

• We deployed the app script for the doc! ✅🚀
• Resolving our 3 main challenges!! 🎉👍
• Getting working 🏁

What we learned

• AppScripts! 🤯
• Working with python 🐍 and Google Docs 📄

What's next for Gemiscript.ai

• More action taking (e.g., send email directly from the doc) 📧➡️
• Automatic diffusion models 🥶🥶
• More research on inclusivity and making it better for a diverse group of users (e.g., what fonts do people with ADHD prefer that help them understand, what format of data to be bolded, automatically highlighted directly in docs) 🔬👥

In all, we hope you liked our project, and we had a lot of fun working on it! 🎉🙌

Built With

• gemini-api
• google-doc-api
• google-scripts
• python