OptiZframe

Inspiration

My project, OptiZframe, was inspired by the need to optimize mainframe resMyce utilization, particularly in relation to MSU (Million Service Units) costs. Mainframe optimization is often done manually and reactively, making it difficult to continuously manage and optimize workloads. I aimed to leverage machine learning to automate the detection of expensive jobs, predict cost spikes, and suggest optimizations to reduce these costs.

What It Does

OptiZframe uses machine learning to analyze mainframe job data (in CSV format) and provide actionable insights. It can:

• Detect high-cost jobs and peak MSU usage times.
• Forecast future MSU usage trends to predict potential cost surges.
• Suggest optimizations such as rescheduling jobs to off-peak times.
• Provide data-driven insights summarizing job patterns and suggesting cost-saving strategies.
• Estimate potential cost savings after applying optimizations.

By uploading job data, users can instantly see job consumption patterns and receive actionable recommendations to optimize their mainframe workload.

How I Built It

I used the following technologies to bring OptiZframe to life:

• Frontend: Streamlit for a fast, interactive dashboard.
• Backend: FastAPI to handle the processing of uploaded data and running the machine learning models.
• Machine Learning & Forecasting:
  • Scikit-learn for building models to detect cost hotspots and predict MSU usage trends.
  • Facebook Prophet for forecasting future MSU usage based on historical data.
• Data Processing: Pandas for reading and cleaning the job data (CSV format).
• Visualization: Plotly to display MSU trends, job patterns, and forecasting graphs.

The project is deployed on Streamlit Cloud, which allows users to easily upload their data and get instant insights.

Challenges I Ran Into

• Data Quality: Mainframe job data can be inconsistent, with missing values or outliers that needed to be cleaned before using machine learning models.
• Model Accuracy: While scikit-learn models performed decently, ensuring the accuracy of cost predictions and optimizing recommendations was challenging with limited data points.
• Real-Time Forecasting: Facebook Prophet did Ill with the forecasting, but handling dynamic, real-time job data and updating forecasts continuously was a bit tricky.
• Time Constraints: I didn’t have time to implement a fully automated What-If Simulator feature, though I did develop a solid core MVP.

Accomplishments That I'm Proud Of

• Predicting Cost Hotspots: Successfully applying machine learning techniques to detect expensive jobs and time windows based on historical MSU usage.
• Accurate MSU Forecasting: I used Facebook Prophet to generate meaningful MSU forecasts that allow users to plan for upcoming peaks.
• Streamlined User Interface: With Streamlit, I built an interactive interface that allows users to easily upload data and view results without any coding required.
• Impact Potential: I’re excited about the possibility of applying machine learning to save significant costs for enterprises and help them automate optimization efforts.

What I Learned

• Machine Learning for Optimization: I learned how to apply machine learning techniques like classification and forecasting to real-world operational challenges.
• Time Series Forecasting: Working with Facebook Prophet gave us practical experience in handling time series data and making predictions based on historical trends.
• Data Preprocessing: I learned how important it is to clean and preprocess the data before applying machine learning models, especially when working with real-world datasets.

What’s Next for OptiZframe

• Improved Forecasting: I plan to improve the forecasting model by incorporating additional factors, like job dependencies or external data, to enhance prediction accuracy.
• Dynamic Job Rescheduler: I will add functionality that automatically reschedules jobs based on forecasted MSU usage to optimize costs in real-time.
• Scalability: The tool needs to handle larger, real-world datasets, which requires more advanced data processing capabilities.
• More Optimization Strategies: I’re looking to incorporate techniques like parallel processing and job grouping to optimize job execution further.
• What-If Simulator: A tool that lets users experiment with different job schedules and instantly see potential savings.

📜 Conclusion

OptiZframe is a machine learning-powered tool designed to help enterprises reduce their mainframe costs by automating job optimization. By leveraging data-driven insights and predictive forecasting, the tool empowers users to make smarter scheduling decisions and optimize their workload without manual intervention. As I continue to develop the project, I’m excited about the potential to save organizations significant costs while improving operational efficiency.

Built With

• python