Inspiration
When I had to upload files to websites, I was often frustrated by the need to use an external tool, software, or website just to modify, compress, or convert my files to the right format. To solve this problem, I decided to create a browser extension capable of editing and converting audio and video files directly before upload injecting them straight into the page’s upload form. Now, I no longer need any third-party software to prepare my files for upload.
What it does
EasyUpload is a conversational assistant that leverages powerful tools such as FFmpeg and ImageMagick to edit, convert, and optimize audio and video files before seamlessly reinjecting them into the web page’s upload form. Everything runs locally no files are ever sent to external servers. The assistant uses Chrome’s built-in AI model and local LLM for all processing.
How I built it
Built with TypeScript, React, Zustand (state management), CRXJS, MiniSearch, ImageMagick, FFmpeg, ChatUI, and react-filerobot-image-editor.
Challenges I ran into
The tools used for processing images and videos such as ImageMagick and FFmpeg are command-line based. To make them work, the assistant must generate valid command lines. However, since the local model is relatively small, it doesn’t have all the necessary command syntax in memory and often hallucinates when asked to produce them directly.
After running multiple experiments with different approaches, I eventually chose MiniSearch as the most reliable solution. It allows the model to search a local database of command examples and retrieve the right one based on the user’s request, ensuring accuracy without hallucinations.
Another major challenge was extracting upload requirements from web pages — things like file format, maximum size, resolution, or duration. I tested many methods to identify the most relevant information and pass it to the model effectively. To ensure everything worked as intended, I set up automated tests using Puppeteer to validate both the prompts and the extraction methods across several websites.
Accomplishments that I'am proud of
Finding a way to generate valid ImageMagick or FFmpeg commands to handle the tasks requested by the user. I’m really happy that I made it work.
What I learned
I learned how to use Chrome’s built-in AI models and how to enable communication between the different parts of the extension, such as the service worker, content scripts, offscreen document and side panel. Now it’s time to do a bit of cleanup and standardize the communication system, since it’s the end of the challenge 😄
What's next for Easy upload
The upload requirement extraction and tool execution systems are the two core building blocks needed to create a fully autonomous extension. Capable of analyzing whether a user’s uploaded file meets a website’s requirements and, if not, automatically modifying it to comply. The final goal of a fully autonomous system is now achievable.
Log in or sign up for Devpost to join the conversation.