Presentation Link: https://docs.google.com/presentation/d/1C6qIFLcgZu_esx1usscCtgeC327TC7wj645ZY2eVYQU/edit?usp=sharing
Inspiration
Financial markets are complex, and existing visualization tools often fail to capture the nuanced relationships between stocks over time. We wanted to build a system that leverages AI to uncover hidden patterns in financial data, enabling smarter investment decisions. Inspired by the success of foundation models in other fields, we sought to apply them to financial data analysis.
What it does
Jack Street transforms financial data visualization with AI-driven temporal embeddings, clustering stocks into insightful price baskets that reveal macroeconomic trends and statistical arbitrages.
How we built it
- Natural Language Processing on Graphs: Visualize financial news using NLP-driven graph representations.
- Pretraining a Foundation Model: We pretrained a attention-based transformer model on historical financial data to generate latent temporal embeddings that capture essential stock movement patterns.
- Visualization with PCA: We applied PCA to visualize embeddings, showing key financial attributes such as average percentage increase, total percentage increase, and correlation with specific stocks.
- Stock Clustering with K-Means: We applied K-Means clustering on latent representations to form interpretable clusters based on stock tickers. This allows us to create price baskets that track different macroeconomic sectors.
- Statistical Arbitrage Detection: We developed a system to detect stocks with historically similar trends but recent deviations, enabling investors to identify potential arbitrage opportunities.
- Graph-Based Navigation in React: Our front-end presents these insights using a hierarchical graph interface, resembling a Linux directory system, allowing users to explore macroeconomic trends at different levels of granularity.
Challenges we ran into
Some of the challenges we faced were ensuring that our temporal embeddings were high quality, providing meaningful names to clusters, and creating a graph UI to represent the data analysis.
Accomplishments that we're proud of
- Successfully pretrained a foundation model to generate high-quality temporal embeddings that preserve essential stock relationships.
- Developed an intuitive, interactive graph-based UI for financial market exploration.
- Created a novel method for identifying statistical arbitrage opportunities using historical data.
- Built a scalable and interpretable clustering method for organizing stocks into price baskets.
What we learned
- How to fine-tune foundation models for financial data analysis.
- The importance of feature engineering in ensuring embeddings capture real-world stock behavior.
- Best practices for visualizing high-dimensional data in a user-friendly format.
- How to integrate AI-driven insights into a functional, real-world financial application.
What's next for Jack Street
- Real-Time Data Integration: Expanding the model to process live financial data and detect opportunities in real time.
- User Customization: Allowing users to define their own price baskets and track custom financial trends.
- Advanced Arbitrage Strategies: Enhancing our statistical arbitrage models with reinforcement learning to optimize trading strategies.
- Expanding to Other Asset Classes: Applying our approach to commodities, forex, and cryptocurrency markets.
Log in or sign up for Devpost to join the conversation.