From the course: Advanced Data Processing: Batch, Real-Time, and Cloud Architectures for AI

Characteristics of batch AI

From the course: Advanced Data Processing: Batch, Real-Time, and Cloud Architectures for AI

Start my 1-month free trial Buy for my team

Characteristics of batch AI

“

- [Instructor] In this chapter, we will explore Batch AI. We will discuss how to architect machine learning pipelines for Batch processing. Model training is inherently an iterative process. Even for inference, Batch systems are easier to build and manage than real-time systems. So it's recommended to build Batch architectures by default, unless the use case demands real-time architectures. What are the key characteristics of Batch AI? In Batch AI processing is done through jobs. These jobs can be scheduled to run at fixed intervals, or they can be invoked when required. Data is processed in bulk as batches for both feature engineering and model training. Big data batch-processing techniques are typically used. Model training is done offline. It can be done in a development environment. Data that is processed can be used again and again for model training. Data is pulled from data sources through queries. Final results are pushed to data syncs. The pull and push mode for data is possible since both are done on demand when the job runs. Because it's offline, data scientists have better control of the process. They can do model validation until they are satisfied and promote models when required. What are some key focus areas while architecting Batch AI? Jobs need to be scheduled in Batch processing. Multiple jobs need to be orchestrated together, so the output of one job can flow as input to another job. This requires job orchestration software. Large-scale feature engineering needs to use big data processing patterns and tools to scale to expected data volumes. Model training costs can quickly get out of control. Managing costs for training and inference requires careful budgeting, resource provisioning, and monitoring. To speed up processing and reduce latency, the architecture should provide for maximum parallelism and minimize bottlenecks in the data flow. Batch AI should be repeatable, so if a given step fails, it can be fixed and the processing can resume from there on. This requires a design that lends itself to roll back and reprocessing.

Contents