Scandium | Devpost

Inspiration

The modern world is rampant with cybersecurity threats. Often, the attack vector are simply misconfigured git repositories that leak secret keys and .env files. In just 2023 alone, GitGuardian detected over 12 million leaked secrets on GitHub, 90% of which remained valid 5 days after being leaked. Furthermore, We believe that this stems from the fact that most scanners are reactive, notifying the developer after the secret has already been published to the internet.

What it does

Instead, Scandium takes a proactive approach, identifying security issues far before they ever get pushed to a repository. We use local LLMs to instantly detect leaked secrets and notify the developer via our Visual Studio Code extension in a similar fashion to linters. Furthermore, users are able to chat with the bot that detected the vulnerability for further clarification and guidance.

How we built it

Our project is split into two parts. The client extension, and the local server. First, the client sends a list of its directories, along with current file to the server. The server then connects to Ollama to allow us to run small local LLMs on the user’s machine. Scandium will automatically determine which files to scan, along with ignoring any files in .gitignore as it will not be added to the git repository. This selective scanning process significantly improves the performance, while ensuring sufficient coverage to maintain security. After scanning, the server returns vulnerabilities, which are directly highlighted within the client editor.

Challenges we ran into

One major challenge we faced early on was in the accuracy of the LLMs. Since the LLM must be run locally for security, we aimed for 7B models, which are somewhat inconsistent. To solve this, we performed extensive prompt engineering, and tested various models. Eventually, we settled on a complex prompt with numerous examples, and qwen2.5-coder , a 7B Alibaba model that specialized in programming.

Accomplishments that we're proud of

In the end, we are proud of the fact that we successfully integrated different components together and developed a VS Code extension that scans files and provides user real-time with feedback on any potential vulnerabilities to limit any possibilities of leaking any user’s privacy information online. We are especially proud of the fact that we were able to develop a full VS Code extension within 24 hours, despite having no previous experience with it.

What we learned

Throughout this Hackathon, not only were we able to explore a large range of technologies and frameworks, we also had the opportunity to develop numerous fascinating features for Scandium. Through the process of developing Scandium, we gained experience in running local LLM, script engineering and customizing VS Code extensions.

What's next for Scandium

In the future, we would like to expand our product beyond VS Code to become compatible with other editors/IDEs such as Sublime Text and IntelliJ via Language Server Protocols. Furthermore, we would also like to integrate with Git to require the user to confirm their uploads if there are unresolved secret leaks.