Ctrl+Magic

The clipboard of the 21st century.

Comment

We've been using clipboards the same way since the 1980s. It's about time we changed.

Welcome to the clipboard of the 21st century.

GitHub Website

Our video is longer than 5 minutes for extra context, but feel free to just watch the first minutes for the general idea!

About The Project

Ctrl+Magic is the smart version of a clipboard. It understands what you copy and helps you reuse it intelligently, privately, and efficiently.

The Problem

Copying and pasting is one of the most common actions on a computer, but it’s so primitive. We've always switching windows to copy-paste stuff. Just think about the last time you filled out a form. You've copied your email address dozens of times in the past... and still your device doesn't remember it!

So we asked ourselves:

What if my clipboard could predict what I wanna paste?

Our Solution

Ctrl+Magic runs silently in the Windows system tray (though we also have a primitive UI for debugging) and enhances your clipboard with AI-powered intelligence.

Core Features:

  • Smart Clipboard Capture - Intercepts Ctrl+C events for text and files at the OS level and sends content to Snowflake Cortex AI for multimodal understanding. (Falls back on Gemini API for full safety in case of failures).

  • AI Processing - Classifies content, understanding the context of where your text should be pasted in the future (code, email, etc). Extracts structured meaning using Snowflake Cortex AI.

  • Intelligent Paste - When we do Ctrl+M(agic), the context of your screen is analyzed with the relevant local clipboard history and Snowflake memories to quickly identify what text you want to paste. We have a proactive prefetcher that actually searches Snowflake for relevant memories in the background so they are ready immediately upon Ctrl+M and we have as little latency as possible.

  • Secure Cloud Storage - Stores clipboard history on the cloud in Snowflake for quick and reliable access.

  • Vultr Cleanup Service - A background Python service built on Vultr VX1 VMs analyzes clipboard history using Snowflake Cortex LLM. It detects duplicate memories, passwords, API keys, financial data, and more. Automatically removes or flags risky entries based on confidence thresholds and past knowledge stored in agent memory database. Logs actions in database and also logs non identifiable actions in agent memory database to query during later calls and improve decision making. We are using our own version of the A-MEM agent memory (which is based on atomic "memories") from this paper that came out a few months ago: https://arxiv.org/abs/2502.12110

  • Audit Logging & Compliance - For extra safety, we have all cleanup actions logged in CLEANUP_AUDIT_LOG. This also supports a dry-run mode for safe testing

Built With

Sponsors are in bold

AI Layer:

  • Snowflake Cortex AI_PARSE_DOCUMENT
  • Snowflake Cortex Search and Vector Embeddings
  • Snowflake Cortex AI_COMPLETE Structured Outputs (LLM)
  • Gemini API (fallback)

Backend Storage:

  • Snowflake DB
  • Snowflake Hybrid Tables

Online Monitoring Dashboard:

  • Snowflake Dashboards
  • Snowflake Notebooks

Windows App:

  • Windows Forms (hidden UI)
  • .NET C#

Vultr Background Cleanup Service:

  • Python service hosted on a Vultr VX1 VM
  • Uses Snowflake Cortex AI
  • A-MEM (https://arxiv.org/abs/2502.12110) custom implementation which communicates with Snowflake for agentic data management
  • Tested to successfully remove sensitive and unnecessary info with <200 ms latency

Website:

  • Next,js site hosted on a Vultr VX1 VM with a custom firewall group
  • .tech domain name
  • PM2 and NGINX for serving

Architecture Overview

architecture diagram

Challenges we ran into

  • Capturing modern GPU-rendered apps (like Chrome): Required moving from legacy window capture methods to modern Windows Graphics Capture APIs.
  • Privacy & security design: We had to ensure sensitive clipboard data could be detected, audited, and safely removed automatically.
  • AI reliability: Designing confidence-based thresholds so the system doesn’t over-delete useful data or miss risky content.
  • OS-level integration: Implementing global keyboard hooks without impacting system performance or user experience.
  • Speed: Right now, we need a VLM to process CTRL+M pastes, and this takes time to process. We could've used a local small VLM, but we wanted to use the APIs to be considered for the tracks, so that's why we didn't go that route. We experimented with Hybrid Tables in Snowflake, but the bottleneck was not the storage; we implemented memory pre-fetching, and this gave us a huge speedup - though it's not instant (future work!).

Accomplishments that we’re proud of

  • Built a fully working OS-level clipboard AI assistant which integrated two AI systems: Snowflake Cortex for understanding + privacy analysis with Cortex and Vultr
  • Implemented automated database provisioning and deployment, management, dashboarding and monitoring (all in Snowflake).
  • It just works! This is the most important thing. Usually when we're done with a hackathon we just throw our project out cause it's just a demo, but this is something we actually wanted to have and use! It can handle, more or less, anything you throw at it, so once we finished recording the video, we kept using the software!

What we learned

  • It's super hard to build Windows-native tools that interact directly with OS events, system permissions are a pain and device specificities make everything even harder.
  • That it's hard to come up with some hotkeys that are not used, everything's taken :/
  • How to combine multimodal AI with structured databases, and how to make it actually be efficient.
  • That Snowflake is much more than a database. It's a whole AI engine, you can build anything with it!
  • That when you build something you wanna use yourself, you have a much clearer vision and things just work out.

Built With

Share this project:

Updates