Trace | Devpost

Home page
Analysis page
Investigation page
Example report

Inspiration

Humans need help analyzing data logs. Doing this is tedious work, which is why automation and AI for completing these kinds of tasks is really useful in saving time and money for professionals.

How we built it

We did a comparative analysis of different models and different architectures. Below are some of the results that we obtained:

Model	View	PR-AUC	95% CI	Recall@50	Notes
Deep AE ensemble (×7)	CTX	0.918	[0.79, 1.00]	20/22	GPU, best recall (21/22 at op point)
GMM (k=4, diag)	CTX	0.909	[0.77, 1.00]	20/22	trains in <1s, no GPU, recommended
Ensemble	CTX	0.907	[0.77, 1.00]	20/22	no better than GMM alone
RuleNovelty (no ML)	CTX	0.867	[0.71, 1.00]	19/22	interpretable, best ROC (0.990)
ECOD	CTX	0.862	[0.70, 0.99]	19/22
KMeans	CTX	0.836	—	20/22
GPT-5 (blind LLM)	—	0.324	—	10/22	recall 0.77, precision 0.07, $4.72/run
Claude Opus 5 (blind LLM)	—	≤0.11*	—	6/22	recall 0.77, precision 0.03, 614 FPs; *partial run (154/192 windows, credit exhausted)
KMeans	RAW	0.646	—	16/22	same algorithm, naive features

From this analysis, we chose to use GMM and Deep AE as the model architectures that are best suited for this task. To further enhance the performance of the GMM model, we also added OpenAI API (GPT-5) capabilities for further triage assistance to reduce the number of false positives that the GMM model could potentially provide.

Challenges we ran into

Frontend design/implementation
Connecting the backend and frontend together
Sleep...

Accomplishments that we're proud of

We were able to build a fully functional and ready-to-go product that provides real insights (reports) and that is 100% scalable (can be also used for other data types/structures as well).

What we learned

AI is a cool tool to implement and use for cyber security logging, monitoring and analysis and is useful in helping detecting anomalies and driving decisions and impacts for humans to make.

What's next for Trace

What's next is to continually improve the frontend and obtain feedback from professionals who could potentially use this tool for their own logs and hook this up with existing services and platforms.

Additionally, maybe we could import some datasets with labels and train supervised learning models (depending on data availability).

Furthermore, we could also scale the backend further, caching past investigations/results as well as having users accounts and so on.

Lastly, for the unsupervised learning models, more data and more logs are necessary to make further model improvements.