Photo by Pinterest

When deploying a new service or database to the cloud, the budget-conscious developer’s default instinct is to scroll straight to the bottom of the virtual machine catalog and select the cheapest option available.

On Amazon Web Services (AWS), this might be a t3.micro. On Microsoft Azure, a Standard_B1s.

On paper, these “burstable” instances look like an incredible deal: they cost mere pennies a day, offering plenty of RAM and basic processing power for dev environments or microservices.

Then, you run a simple database migration, compile some assets, or experience a minor traffic spike. Suddenly, your application slows to a crawl, API responses timeout, and SSH connections freeze.

You’ve hit the Compute Cliff.

In this guide, we will dive deep into the underlying mechanics of cloud compute tiers. We will map out the mathematical models behind burstable CPU credits, explain how decoupling storage from compute guarantees durability, and detail how to use resource governance hierarchies to prevent silent billing leaks.

1. The Mathematics of Burstable Compute

Burstable instances (like the AWS T3/T4g family and Azure B-series) are designed for workloads that sit idle most of the time but need sudden, short-lived bursts of CPU power.

Instead of paying for dedicated, continuous CPU access, you operate on a CPU Credit Economy.

The Mathematical Model

Each burstable VM is assigned a baseline CPU performance and a credit accumulation rate based on its instance size.

Architectural Takeaway: Use burstable instances only for asynchronous tasks (like message queues, background workers, or staging servers) where a sudden slowdown won’t break the user experience. For critical web APIs or primary databases, always select “General Purpose” (e.g., AWS M-series, Azure D-series) or “Compute Optimized” (e.g., AWS C-series, Azure F-series) instances with dedicated, continuous CPU cores.

2. Storage Decoupling: Ephemeral vs. Persistent Block Storage

When you spin up a cloud virtual machine, the operating system is installed on a virtual hard drive. In a modern cloud, this drive is almost never physically inside the physical blade hosting your VM. Instead, it is network-attached block storage.

Persistent Block Storage

Services like AWS Elastic Block Store (EBS) and Azure Managed Disks decouple storage from compute entirely. This architectural separation provides massive advantages for system durability and disaster recovery:

┌─────────────────────┐               ┌───────────────────────┐
│     COMPUTE VM      │               │ PERSISTENT STORAGE    │
│  (CPU & RAM State)  │               │   (EBS / Managed Disk)│
│  Can be destroyed,  │               │                       │
│  resized, or moved. │               │   State is preserved  │
└──────────┬──────────┘               │   synchronously.      │
           │                          └──────────┬────────────┘
           │      Network Attachment             │
           └─────────────────────────────────────┘

If the underlying hardware node hosting your VM fails, the cloud hypervisor can automatically provision a new virtual machine on a healthy host, detach your persistent block volume from the old node, and attach it to the new one. Your data remains perfectly intact and up-to-date.

Ephemeral Storage (Instance Store)

Conversely, some instances offer local NVMe drives attached directly to the physical host. This is known as ephemeral storage or instance store.

• Pros: Extremely low latency and astronomical read/write IOPS (Input/Output Operations Per Second).
• Cons: If the virtual machine is stopped, deallocated, or fails, all data on the ephemeral drive is lost forever.

Architectural Takeaway: Never store database engines or stateful application configurations on ephemeral drives. Use persistent, network-attached block volumes (like Azure standard LRS/SSD or AWS gp3) for your persistent data, and reserve ephemeral storage strictly for swap files, temp databases, caching layers, and high-speed data processing buffers.

3. Resource Governance and the “Orphaned Resource” Tax

When teams build and deploy systems incrementally, they often spin up VMs to test architectures, destroy them when finished, and move on.

However, because compute and storage are decoupled, deleting a virtual machine does not automatically delete its associated resources.

If you delete an Azure VM or an AWS EC2 instance without checking your dependency trees, you will leave behind:

• Unattached block storage volumes (charging you per-gigabyte/month).
• Unused Public IP addresses (both AWS and Azure tax you hourly for holding a static public IP without associating it to an active VM).
• Orphaned Network Interfaces (NICs) and security group artifacts.

This is where Resource Governance Hierarchies save budgets.

[MANAGEMENT GROUP]
              │
              ▼
         [SUBSCRIPTION]
              │
              ▼
       [RESOURCE GROUP]  <─── Virtual Machine, Disk, Public IP, NIC (Tied Lifecycle)

By organizing your applications within logical containers like Azure Resource Groups or AWS Resource Groups, you can tie the lifecycle of related resources together.

When you tear down a project, you don’t delete the VM — you simply delete the entire Resource Group. The cloud provider then triggers a cascading deletion, systematically wiping out the VM, its network cards, its public IPs, and its persistent disks in one clean sweep, ensuring zero orphaned resources on your next monthly invoice.

Summary Checklist: Designing Resilient Compute Architectures

Before launching your next cloud system into production, ensure your design meets these architectural standards:

• [ ] Evaluate CPU Profiles: Do not use burstable instances for production databases. Ensure critical APIs run on general-purpose or compute-optimized tiers.
• [ ] Decouple Stateful Data: Store all critical data on persistent block storage volumes (e.g., Azure LRS, AWS gp3). Never write persistent records directly to ephemeral/instance drives.
• [ ] Monitor Your Credit Banks: If you use burstable VMs, set up alarms in AWS CloudWatch or Azure Monitor to alert your team if your credit balance ($C(t)$) falls below $20\%$.
• [ ] Enforce Group Governance: Organize all resources inside logical containment boundaries (Resource Groups) to manage lifecycles and avoid “zombie resources” billing leaks.

DOWNLOAD RESOURCES: https://docs.google.com/document/d/1nkFv9rXbpyPT0anDC9mCn2mPSG6hWuvMk53Lz60WpK4/edit?usp=sharing

How does your team optimize its compute spending? Have you ever run into the burstable performance throttling cliff? Share your operational stories in the comments below!

The ultimate multi-cloud guide to subnets, firewall statefulness, and how to prevent silent network blocks in AWS and Azure.

Photo by: Pinterest

When teams begin migrating infrastructure to public clouds like Amazon Web Services (AWS) or Microsoft Azure, they often treat cloud networking like their old on-premises server closets.

They write a few firewall rules, spin up a virtual machine, and expect traffic to flow smoothly.

Then, the troubleshooting tickets start piling up:

• “Our application server can’t download package updates.”
• “We added an inbound HTTP rule, but our health checks are still failing.”
• “We ran out of IP addresses in our subnet within three weeks.”

In the cloud, networking is entirely software-defined, and security is integrated directly into the fabric of the hypervisor. To build resilient, secure infrastructure, you need to master three fundamental concepts: CIDR IP arithmetic, Firewall Statefulness, and the Stateful vs. Stateless boundary.

In this guide, we’ll demystify these core concepts, compare AWS and Azure security structures, and map out the mathematics of modern cloud subnetting.

1. Cloud Subnetting Math and the “Minus 5” Tax

In traditional networking, IP allocation is calculated using standard Classless Inter-Domain Routing (CIDR) notation. An IPv4 address consists of $32$ bits. When you declare a CIDR block with a prefix length of $n$ (for example, /24), you are reserving the first $n$ bits for the network identifier, leaving $32 - n$ bits for host allocation.

Mathematically, the total number of IP addresses available in a given subnet is:

Total IPs} = 2^{32 — n}

For a standard /24 subnet:

Total IPs = 2^{32–24} = 2⁸ = 256 addresses

For a smaller /28 subnet:

Total IPs = 2^{32–28} = 2⁴ = 16 addresses

The Cloud Tax: 5 Reserved IPs

In on-premises networks, you only lose $2$ IP addresses per subnet: the network address (all host bits set to $0$) and the broadcast address (all host bits set to $1$).

However, both AWS VPC Subnets and Azure Virtual Network Subnets reserve five IP addresses per subnet.

For any cloud subnet of size $n$, the actual number of usable host addresses is:

Usable Cloud IPs = 2^{32 — n} — 5

If you provision a /28 subnet expecting $14$ usable addresses for a cluster of microservices, you will find yourself with only $11$ usable IPs:

Usable IPs = 2^{32–28} — 5 = 16–5 = 11

The cloud providers claim these five addresses for critical infrastructure operations:

1. Host .0: Network Address.
2. Host .1: Default Gateway (VPC/VNet router).
3. Host .2: DNS Server (e.g., AWS Route 53 resolver or Azure DNS).
4. Host .3: Future capability / internal system mapping.
5. Host .255: Network Broadcast Address (even though the cloud does not support traditional IP broadcasting).

Architectural Takeaway: Never size your subnets too tightly. If you run out of IP addresses in a subnet, you cannot resize it. You must destroy the subnet and recreate it from scratch, which often causes major deployment disruptions.

2. Stateful Firewalls: Under the Hood of Security Groups

The primary line of defense for a cloud virtual machine is the Security Group (known as an EC2 Security Group in AWS, and a Network Security Group or NSG in Azure). Security Groups are stateful, host-level distributed firewalls evaluated at the hypervisor level (specifically, at the virtual network interface or NIC).

The term stateful is the key to their efficiency.

[Inbound Traffic] ──> Passes Firewall Rule Check ──> Establishes Active Connection State
                                                               │
                                                               ▼
[Outbound Response] <───────────────────────────── Dynamically Allowed Back Out (No Rule Needed)

When a packet arrives at your virtual machine, the security group evaluates its inbound rules. If a rule matches (e.g., allowing inbound HTTP traffic on TCP Port 80), the firewall permits the packet to pass and immediately creates an entry in an internal connection tracking table.

When the operating system sends a response back to the client, the firewall checks the connection tracking table. Seeing an active, established connection, it dynamically allows the outbound response packet to leave, completely bypassing all outbound rules.

AWS Security Groups vs. Azure Network Security Groups (NSGs)

While they serve the same purpose, AWS and Azure design Security Groups with fundamentally different logic:

TABLE REF: Elastic Network Interfaces (ENIs), Virtual Network Interface (NIC)

For detailed implementation steps on creating these rules, you can consult the AWS Security Groups User Guide or the Azure NSGs Overview.

3. Stateful (Security Groups) vs. Stateless (NACLs)

Many security architectures enforce defense-in-depth by placing another firewall layer at the subnet boundary. In AWS, this is known as a Network Access Control List (NACL).

Unlike Security Groups, NACLs are stateless.

Stateful (Security Groups)             Stateless (NACLs)
 ┌───────────────────────────┐      ┌───────────────────────────┐
 │ Evaluates INBOUND only.   │      │ Must evaluate both        │
 │ Response is automatically │      │ INBOUND and OUTBOUND      │
 │ permitted back out.       │      │ rules separately.         │
 └───────────────────────────┘      └───────────────────────────┘

Because a stateless firewall does not maintain a connection tracking table, it views every incoming and outgoing packet as an entirely independent event.

The Ephemeral Port Trap

This stateless nature is where many DevOps teams run into trouble. Imagine you want to allow your internal virtual machines to fetch software updates from the public internet.

In your Security Group, you simply add an outbound rule allowing TCP Port 443 (HTTPS) to 0.0.0.0/0. The stateful group automatically permits the incoming response.

If you try to restrict this at the NACL level, you must configure two distinct rules:

1. An Outbound Rule allowing TCP Port 443 to the internet.
2. An Inbound Rule allowing traffic back in from the internet.

But what port does the incoming response use? It does not connect back to your server on port 443. Instead, your server initiated the connection using a randomized client port, known as an ephemeral port.

According to the Internet Assigned Numbers Authority (IANA), the default ephemeral port range is $49152$ to $65535$. However, different operating systems and NAT gateways utilize ranges from $1024$ to $65535$.

If your stateless NACL does not explicitly have an inbound rule allowing return traffic on these high-numbered ephemeral ports, your outbound connection will fail silently.

For step-by-step guidance on setting up stateless parameters, read the AWS NACLs User Guide and check the AWS Ephemeral Ports Guide.

Summary: Best Practices for Cloud Networking Security

To protect your cloud architecture while maintaining seamless connectivity, keep these four guidelines in mind:

1. Calculate Subnet Growth (2^{32-n} — 5): Account for the 5 reserved cloud IPs and build your CIDR ranges to accommodate horizontal scaling and green-blue deployment configurations.
2. Leverage Security Group Referencing: Instead of hardcoding public or private IP addresses inside your security groups, reference the ID of other security groups. For example, configure your database’s security group to only accept inbound traffic on port 3306 if the source is your application-sg ID.
3. Avoid SSH/RDP Exposure: Never expose administrative ports like SSH (port 22) or RDP (port 3389) directly to 0.0.0.0/0. Use modern, session-managed services like AWS Systems Manager Session Manager or Azure Bastion.
4. Use NACLs Sparingly: Keep your NACL rules simple. Use them for broad “blocking” actions (like denying known malicious IP ranges at the subnet boundary) and rely on stateful Security Groups for granular micro-segmentation.

DOWNLOAD THE RESOURCES: https://docs.google.com/document/d/1bLc2_gGPQRJdBFTGynRBDPsIT8oPhJVQiP837XQUCTA/edit?usp=sharing

How does your team manage network boundaries? Do you use a combination of Security Groups and NACLs, or do you rely on centralized firewalls? Let’s discuss in the comments below!

Photo by: Pinterest

Imagine it’s 2015. You’ve just finished writing a revolutionary machine learning utility. You want to share it with your team, so you do what every Python developer did back then: you create a setup.py script, drop a few dependencies into a requirements.txt file, and run python setup.py install.

It works on your machine. But when your teammate pulls the code, their environment breaks. Pip complains about missing dependencies, a library compilation fails midway through, and your local development packages get inexplicably overwritten.

For decades, Python packaging was notoriously described as “the wild west.” The mechanism was brittle, imperatively designed, and plagued by security vulnerabilities.

Thankfully, those days are over.

Today, Python packaging has evolved into a robust, standardized, and fully declarative ecosystem overseen by the Python Packaging Authority (PyPA). If you are still shipping code with an imperative setup.py file, it’s time to upgrade. In this guide, we’ll demystify modern Python packaging, look under the hood of PEP 517 and 518, explain why the src/ layout is your best shield against test-suite bugs, and build a production-ready package configuration from scratch.

1. The Imperative Nightmare: Why setup.py Had to Die

To appreciate the modern standard, we must first understand the flaws of the legacy system. Historically, creating a Python package required writing a Python script called setup.py that invoked setuptools.setup().

This meant that compiling or installing a package required running arbitrary Python code.

This imperative design introduced three massive architectural flaws:

1. The Bootstrap Paradox (The Chicken-and-Egg Problem): If your setup.py script needed a helper library (like numpy or cython) to compile your code during installation, how could you run the script to install the package if those tools weren't already installed? There was no standard way to declare "pre-installation" dependencies.
2. Security Risk: Because installing a package executed code locally, malicious packages could run arbitrary exploits on a developer’s machine or target CI/CD runners during the simple pip install phase.
3. Implicit Tool Lock-in: setup.py tightly coupled the packaging ecosystem to setuptools. If a developer wanted to build packages using alternative, modern backends like Poetry, Flit, or Hatch, they were completely out of luck.

2. The Great Standardization: PEP 517, 518, and pyproject.toml

Between 2016 and 2018, the Python community introduced two game-changing enhancements: PEP 518 and PEP 517. Together, they decoupled the packaging process into two clean, separate layers: Build Frontends and Build Backends. You can read the authoritative guidelines in the official Python Packaging User Guide.

┌────────────────────────────────────────────────────────┐
│                     BUILD FRONTEND                     │
│                  (e.g., pip, build)                    │
└──────────────────────────┬─────────────────────────────┘
                           │  Communicates via PEP 517
                           ▼
┌────────────────────────────────────────────────────────┐
│                     BUILD BACKEND                      │
│         (e.g., setuptools.build_meta, poetry)          │
└────────────────────────────────────────────────────────┘

PEP 518: Declaring the Tools [build-system]

PEP 518 established a single, unified configuration file for all Python tools: pyproject.toml. It introduced the [build-system] table to explicitly define the tooling dependencies before the build begins:

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

• requires: Tells the frontend (like pip) exactly what packages it needs to download into an isolated, temporary virtual environment to compile your package.
• build-backend: Explicitly points to the engine responsible for generating your distribution files.

PEP 517: Decoupling the Engine

PEP 517 defines a standard API that build backends must implement. This means a frontend tool like pip doesn’t care whether you use setuptools, poetry, hatch, or flit. As long as the backend implements the standard hooks (like build_wheel and build_sdist), your project compiles flawlessly.

3. The src/ Layout Shield: Preventing Silent Test Failures

If you look at modern enterprise Python codebases, you will notice that the core package directory isn’t located in the repository’s root. Instead, it is nested inside a /src directory:

my_project/
├── pyproject.toml
├── src/
│   └── fraud_detection/      <-- Your actual package code lives here
│       ├── __init__.py
│       └── predict.py
└── tests/
    └── test_predict.py

This is called the src/ layout, and it is highly recommended by PyPA over the traditional flat layout (where fraud_detection/ sits directly in the root directory alongside pyproject.toml).

Why is the Flat Layout a Trap?

When you run a testing framework like pytest in a flat-layout repository, the current working directory is added to Python's import path (sys.path).

If your test suite imports your library with import fraud_detection, Python will import the raw source files directly from your local directory, not the compiled package you built.

This introduces two severe vulnerabilities:

1. False Positives: Your tests might pass perfectly on your machine, but fail in production because a crucial asset file or compiled extension was left out of the built package (.whl).
2. Missing Dependency Detection: Since your local files are imported directly, your tests won’t catch whether you forgot to list a dependency in pyproject.toml.

By nesting your code under src/, Python cannot find your package unless it has been explicitly installed into your active virtual environment. Your test runner is forced to test the actual built artifact, ensuring what you ship matches what you tested. For a deeper discussion on this concept, refer to the Pytest Documentation on Import Mechanics.

4. Building a Production-Ready pyproject.toml Blueprint

Let’s put this theory into practice. Below is a highly robust, production-grade template for an enterprise machine learning package (e.g., fraud_detection). This configuration follows the modern pyproject.toml specification and integrates metadata, strict Python restrictions, dependency specifications, and URL references.

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "fraud_detection"
version = "0.1.0"
description = "Enterprise-grade real-time fraud prediction engine."
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
    { name = "xFusionCorp DevOps", email = "devops@xfusioncorp.com" }
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "Intended Audience :: Science/Research",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Scientific/Engineering :: Artificial Intelligence"
]
dependencies = [
    "scikit-learn>=1.2.0",
    "pandas>=2.0.0",
    "numpy>=1.24.0"
]

[project.urls]
"Homepage" = "[https://github.com/xfusioncorp/fraud-detection](https://github.com/xfusioncorp/fraud-detection)"
"Bug Tracker" = "[https://github.com/xfusioncorp/fraud-detection/issues](https://github.com/xfusioncorp/fraud-detection/issues)"
"Documentation" = "[https://xfusioncorp.github.io/fraud-detection/](https://xfusioncorp.github.io/fraud-detection/)"

Breaking Down the Configuration:

• requires-python: Strictly guards your runtime environment. Trying to install this package on Python 3.9 or older will fail gracefully, avoiding obscure runtime syntax errors.
• classifiers: Standardized tags used by PyPI (Python Package Index) to help developers search for stable, compatible, and open-source software. You can browse the complete list of Trove Classifiers on PyPI.
• dependencies: Declares semantic version pins (>=) ensuring your package installs compatible ecosystem versions while preventing breaks from outdated APIs. See the specifications for dependency specifiers for more formatting patterns.

For an exhaustive guide on configuration fields, see the PyPA Guide on Writing your pyproject.toml.

5. From Code to Artifact: Compiling the Wheel

Once your configuration is written, building your package is incredibly simple. Install the official PyPA build frontend and compile the source according to the PyPA packaging tutorial:

# Install the build frontend
python3 -m pip install --upgrade build

# Compile your package from the project root
python3 -m build

This commands reads your pyproject.toml file, provisions an isolated build environment, processes your files under src/, and outputs your artifacts into a new /dist folder.

Inside dist/, you will find two files:

1. fraud_detection-0.1.0.tar.gz (Source Distribution): A compressed archive of your raw source code, ideal for fallback compilations on highly custom architectures.
2. fraud_detection-0.1.0-py3-none-any.whl (The Wheel): A pre-built binary distribution. Installing this is lightning fast. Pip simply unzips it directly into the system's runtime folder—no compilers, no setup script runs, and no overhead. Refer to PEP 427 for the complete specification of the wheel binary format.

How to Inspect Your Wheel

To prove that a wheel is just a glorified ZIP file, try peaking inside it using standard command-line tools:

unzip -l dist/fraud_detection-0.1.0-py3-none-any.whl

You’ll see a clean, predictable file output showing your internal library structure alongside a .dist-info directory containing your parsed metadata and install records.

Summary Cheat Sheet: Modern Python Packaging

To ensure your code meets modern software engineering standards, verify your project structures align with this quick checklist:

• [ ] No setup.py: Use pyproject.toml for all package metadata and dependency definitions.
• [ ] Explicit Build System: Ensure your [build-system] table is configured with a valid PEP 517 backend.
• [ ] Use the src/ Layout: Protect your codebase against silent testing errors by nesting code inside src/.
• [ ] Define Python Guards: Explicitly declare requires-python limits to prevent user deployment failures on unsupported systems.
• [ ] Clean Compilation: Build artifacts using python -m build and distribute the compiled .whl files.

By upgrading your packaging architecture to modern standards, you eliminate local machine inconsistencies, secure your distribution pipeline, and future-proof your application for professional, cloud-native deployments.

Documentations to refer more:

• PyPA Authority Link: Added a link to the Python Packaging Authority (PyPA) in the introduction.
• Standardization & Spec Overview: Linked the main Python Packaging User Guide under the PEP 517/518 flow diagram.
• PEP References: Added direct links to PEP 517 and PEP 518.
• Source Layout Import Mechanics: Added an import mechanics link to the official Pytest Documentation.
• Specification Details: Added links to the pyproject.toml specification, PyPI’s Trove Classifiers, and Dependency Specifiers.
• Detailed Setup Guide: Linked the official Writing your pyproject.toml Guide and PyPA’s Packaging Projects Tutorial.
• Wheel Spec (PEP 427): Integrated a link to the binary distribution specification PEP 427.

Did you find this guide helpful? Have you successfully migrated your legacy setups to pyproject.toml? Let me know in the comments below, and don't forget to share this article with your DevOps and Python engineering teams!

How I fixed a crashing backend container when my CI/CD pipeline said everything was fine.

We’ve all been there. You push your code, watch the GitHub Actions workflow turn green, and feel that sense of satisfaction. You go to check your live site… and it’s down.

This happened to me today while deploying my Mobile Distribution System backend. Here is a breakdown of what went wrong, how I found it, and how I fixed it.

The Problem

After a fresh deployment, my API was unreachable. I SSH’d into my Ubuntu VPS to check the status of my containers.

docker ps

The output was not what I wanted to see: CONTAINER ID ... STATUS: Restarting (1) 14 seconds ago ... mobile-dist-backend

The container was stuck in a boot loop. It would start, crash, and try again.

The Investigation

When a Docker container crashes immediately, it usually means the application inside can’t start. In Node.js apps, this is often due to missing environment variables or database connection errors.

I checked my file system and realized I had made a mess. I had two versions of my project folder:

1. One in my home directory (~/mds-server)
2. One in the deployment directory (/opt/mobile-dist-app)

My Docker container was configured to look for an .env file in /opt/, but I had been editing the .env file in ~/. As a result, the container was launching with an empty or incorrect configuration. It had no MONGODB_URI, so the Mongoose connection failed, and the app crashed.

The Solution

1. Clean House First, I removed the incorrect directory to prevent future confusion.

rm -rf ~/mds-server

2. Update the Source of Truth I navigated to the correct directory and updated the .env file with the correct MongoDB credentials.

cd /opt/mobile-dist-app
nano .env

3. Hard Reset the Container I manually stopped and removed the glitchy container, then ran the run command manually to ensure it picked up the new environment file.

docker stop mobile-dist-backend
docker rm mobile-dist-backend

docker run -d \
  --name mobile-dist-backend \
  --restart unless-stopped \
  -p 5000:5000 \
  --env-file /opt/mobile-dist-app/.env \
  ghcr.io/innovior-developers/mds-server:latest

The Result

A quick check with docker ps confirmed the fix: Up 33 seconds (healthy)

Conclusion

Automation is great, but it relies on your server environment being set up correctly. If your Docker container keeps restarting, 9 times out of 10, it’s a configuration file in the wrong place.

Happy coding!

#Docker #DevOps #Programming #Tutorial

Are you facing these challenges in your Flutter development journey?

• Your code is becoming increasingly messy and hard to maintain
• Business logic is tightly coupled with UI code
• Testing feels like an uphill battle
• Scaling your application seems impossible
• State management is giving you headaches

If you answered yes to any of these, BLoC (Business Logic Component) Architecture is the solution you’ve been looking for. In this comprehensive guide, I’ll walk you through everything you need to know about BLoC architecture with practical examples and real-world applications.

What is BLoC Architecture?

BLoC is an architectural pattern introduced by Google that emphasizes the separation of business logic from the UI layer using streams and reactive programming.

Understanding BLoC with a Simple Analogy

Think of a restaurant:

1. Customer (UI) — You (the user) place an order with the waiter
2. Waiter (BLoC) — Takes your order and passes it to the kitchen
3. Kitchen (Data Layer) — Prepares your food
4. Waiter (BLoC) — Delivers the finished meal back to you
5. Customer (UI) — Enjoys the meal

This is exactly how BLoC works:

• UI is the customer — handles user interactions
• BLoC is the waiter — receives events and emits states
• Repository/Data Layer is the kitchen — fetches and processes data

Core Components of BLoC Architecture

1. Events (Requests)

Events represent user interactions or system triggers. They’re the “commands” sent to the BLoC.

// User presses a button
// Requests data to be loaded
// Submits a form

abstract class AuthEvent {}
class LoginButtonPressed extends AuthEvent {
  final String email;
  final String password;
  
  LoginButtonPressed({required this.email, required this.password});
}
class LogoutButtonPressed extends AuthEvent {}
class CheckAuthStatus extends AuthEvent {}

Events are simple Dart classes that carry the necessary data for the BLoC to process the request.

2. States (Representations of UI)

States represent the current condition of your UI at any given moment.

abstract class AuthState {}

class AuthInitial extends AuthState {}
class AuthLoading extends AuthState {}
class AuthAuthenticated extends AuthState {
  final User user;
  
  AuthAuthenticated({required this.user});
}
class AuthUnauthenticated extends AuthState {}
class AuthError extends AuthState {
  final String message;
  
  AuthError({required this.message});
}

Each state tells the UI exactly how it should render itself. Loading state shows a spinner, error state displays an error message, and so on.

3. BLoC (The Brain)

The BLoC is the component that receives events, processes business logic, and emits states.

class AuthBloc extends Bloc<AuthEvent, AuthState> {
  final AuthRepository authRepository;
  
  AuthBloc({required this.authRepository}) : super(AuthInitial()) {
    
    // Handle login event
    on<LoginButtonPressed>((event, emit) async {
      // Emit loading state
      emit(AuthLoading());
      
      try {
        // Fetch data from repository
        final user = await authRepository.login(
          email: event.email,
          password: event.password,
        );
        
        // Emit authenticated state on success
        emit(AuthAuthenticated(user: user));
      } catch (e) {
        // Emit error state on failure
        emit(AuthError(message: e.toString()));
      }
    });
    
    // Handle logout event
    on<LogoutButtonPressed>((event, emit) async {
      await authRepository.logout();
      emit(AuthUnauthenticated());
    });
  }
}

The BLoC acts as the “brain” of your feature, making decisions about what to do with incoming events and determining what state to emit.

Integrating BLoC with Flutter UI

BlocProvider — Adding BLoC to the Widget Tree

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return BlocProvider(
      // Create BLoC instance
      create: (context) => AuthBloc(
        authRepository: AuthRepository(),
      ),
      child: MaterialApp(
        home: LoginPage(),
      ),
    );
  }
}

BlocBuilder — Updating UI Based on State Changes

class LoginPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Login')),
      body: BlocBuilder<AuthBloc, AuthState>(
        builder: (context, state) {
          // Show loading indicator during authentication
          if (state is AuthLoading) {
            return Center(child: CircularProgressIndicator());
          }
          
          // Show error message
          if (state is AuthError) {
            return Center(
              child: Column(
                mainAxisAlignment: MainAxisAlignment.center,
                children: [
                  Text('Error: ${state.message}'),
                  ElevatedButton(
                    onPressed: () {
                      // Retry logic
                    },
                    child: Text('Retry'),
                  ),
                ],
              ),
            );
          }
          
          // Navigate to home on successful authentication
          if (state is AuthAuthenticated) {
            return HomePage(user: state.user);
          }
          
          // Default: show login form
          return LoginForm();
        },
      ),
    );
  }
}

BlocListener — Handling Side Effects

class LoginForm extends StatelessWidget {
  final emailController = TextEditingController();
  final passwordController = TextEditingController();
  
  @override
  Widget build(BuildContext context) {
    return BlocListener<AuthBloc, AuthState>(
      listener: (context, state) {
        // Navigate on success
        if (state is AuthAuthenticated) {
          Navigator.pushReplacement(
            context,
            MaterialPageRoute(builder: (_) => HomePage()),
          );
        }
        
        // Show snackbar on error
        if (state is AuthError) {
          ScaffoldMessenger.of(context).showSnackBar(
            SnackBar(content: Text(state.message)),
          );
        }
      },
      child: Padding(
        padding: EdgeInsets.all(16),
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            TextField(
              controller: emailController,
              decoration: InputDecoration(labelText: 'Email'),
            ),
            SizedBox(height: 16),
            TextField(
              controller: passwordController,
              decoration: InputDecoration(labelText: 'Password'),
              obscureText: true,
            ),
            SizedBox(height: 24),
            ElevatedButton(
              onPressed: () {
                // Dispatch event to BLoC
                context.read<AuthBloc>().add(
                  LoginButtonPressed(
                    email: emailController.text,
                    password: passwordController.text,
                  ),
                );
              },
              child: Text('Login'),
            ),
          ],
        ),
      ),
    );
  }
}

Key Differences:

• BlocBuilder rebuilds the UI when state changes
• BlocListener executes one-time actions (navigation, dialogs, snackbars)
• BlocConsumer combines both functionalities

Real-World Example: Building a Todo App with BLoC

Let’s build a complete todo application using BLoC architecture to see how everything comes together.

Step 1: Define the Model

class Todo {
  final String id;
  final String title;
  final bool isCompleted;
  
  Todo({
    required this.id,
    required this.title,
    this.isCompleted = false,
  });
  
  Todo copyWith({String? title, bool? isCompleted}) {
    return Todo(
      id: id,
      title: title ?? this.title,
      isCompleted: isCompleted ?? this.isCompleted,
    );
  }
}

Step 2: Define Events

abstract class TodoEvent {}

class LoadTodos extends TodoEvent {}
class AddTodo extends TodoEvent {
  final String title;
  AddTodo(this.title);
}
class UpdateTodo extends TodoEvent {
  final Todo todo;
  UpdateTodo(this.todo);
}
class DeleteTodo extends TodoEvent {
  final String id;
  DeleteTodo(this.id);
}
class ToggleTodoComplete extends TodoEvent {
  final String id;
  ToggleTodoComplete(this.id);
}

Step 3: Define States

abstract class TodoState {}

class TodoInitial extends TodoState {}
class TodoLoading extends TodoState {}
class TodoLoaded extends TodoState {
  final List<Todo> todos;
  
  TodoLoaded(this.todos);
}
class TodoError extends TodoState {
  final String message;
  
  TodoError(this.message);
}

Step 4: Create the Repository

class TodoRepository {
  // Simulate API calls or database operations
  
  Future<List<Todo>> getTodos() async {
    // Simulate network delay
    await Future.delayed(Duration(seconds: 1));
    return [
      Todo(id: '1', title: 'Learn Flutter'),
      Todo(id: '2', title: 'Master BLoC pattern'),
    ];
  }
  
  Future<void> addTodo(String title) async {
    await Future.delayed(Duration(milliseconds: 500));
    // API call to add todo
  }
  
  Future<void> updateTodo(Todo todo) async {
    await Future.delayed(Duration(milliseconds: 500));
    // API call to update todo
  }
  
  Future<void> deleteTodo(String id) async {
    await Future.delayed(Duration(milliseconds: 500));
    // API call to delete todo
  }
}

Step 5: Implement the BLoC

class TodoBloc extends Bloc<TodoEvent, TodoState> {
  final TodoRepository repository;
  List<Todo> _todos = [];
  
  TodoBloc({required this.repository}) : super(TodoInitial()) {
    
    // Load Todos
    on<LoadTodos>((event, emit) async {
      emit(TodoLoading());
      try {
        _todos = await repository.getTodos();
        emit(TodoLoaded(List.from(_todos)));
      } catch (e) {
        emit(TodoError(e.toString()));
      }
    });
    
    // Add Todo
    on<AddTodo>((event, emit) async {
      try {
        await repository.addTodo(event.title);
        _todos.add(Todo(
          id: DateTime.now().toString(),
          title: event.title,
        ));
        emit(TodoLoaded(List.from(_todos)));
      } catch (e) {
        emit(TodoError(e.toString()));
      }
    });
    
    // Toggle Complete
    on<ToggleTodoComplete>((event, emit) async {
      try {
        final index = _todos.indexWhere((todo) => todo.id == event.id);
        if (index != -1) {
          _todos[index] = _todos[index].copyWith(
            isCompleted: !_todos[index].isCompleted,
          );
          await repository.updateTodo(_todos[index]);
          emit(TodoLoaded(List.from(_todos)));
        }
      } catch (e) {
        emit(TodoError(e.toString()));
      }
    });
    
    // Delete Todo
    on<DeleteTodo>((event, emit) async {
      try {
        await repository.deleteTodo(event.id);
        _todos.removeWhere((todo) => todo.id == event.id);
        emit(TodoLoaded(List.from(_todos)));
      } catch (e) {
        emit(TodoError(e.toString()));
      }
    });
  }
}

Step 6: Build the UI

class TodoPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return BlocProvider(
      create: (context) => TodoBloc(
        repository: TodoRepository(),
      )..add(LoadTodos()), // Load todos when page opens
      child: Scaffold(
        appBar: AppBar(title: Text('Todo App with BLoC')),
        body: BlocBuilder<TodoBloc, TodoState>(
          builder: (context, state) {
            if (state is TodoLoading) {
              return Center(child: CircularProgressIndicator());
            }
            
            if (state is TodoError) {
              return Center(
                child: Column(
                  mainAxisAlignment: MainAxisAlignment.center,
                  children: [
                    Text('Error: ${state.message}'),
                    ElevatedButton(
                      onPressed: () {
                        context.read<TodoBloc>().add(LoadTodos());
                      },
                      child: Text('Retry'),
                    ),
                  ],
                ),
              );
            }
            
            if (state is TodoLoaded) {
              if (state.todos.isEmpty) {
                return Center(
                  child: Text('No todos yet. Add one!'),
                );
              }
              
              return ListView.builder(
                itemCount: state.todos.length,
                itemBuilder: (context, index) {
                  final todo = state.todos[index];
                  return ListTile(
                    leading: Checkbox(
                      value: todo.isCompleted,
                      onChanged: (_) {
                        context.read<TodoBloc>().add(
                          ToggleTodoComplete(todo.id),
                        );
                      },
                    ),
                    title: Text(
                      todo.title,
                      style: TextStyle(
                        decoration: todo.isCompleted
                            ? TextDecoration.lineThrough
                            : null,
                      ),
                    ),
                    trailing: IconButton(
                      icon: Icon(Icons.delete),
                      onPressed: () {
                        context.read<TodoBloc>().add(
                          DeleteTodo(todo.id),
                        );
                      },
                    ),
                  );
                },
              );
            }
            
            return Center(child: Text('Something went wrong'));
          },
        ),
        floatingActionButton: FloatingActionButton(
          onPressed: () => _showAddTodoDialog(context),
          child: Icon(Icons.add),
        ),
      ),
    );
  }
  
  void _showAddTodoDialog(BuildContext context) {
    final controller = TextEditingController();
    
    showDialog(
      context: context,
      builder: (dialogContext) => AlertDialog(
        title: Text('New Todo'),
        content: TextField(
          controller: controller,
          decoration: InputDecoration(hintText: 'Enter todo title'),
        ),
        actions: [
          TextButton(
            onPressed: () => Navigator.pop(dialogContext),
            child: Text('Cancel'),
          ),
          TextButton(
            onPressed: () {
              if (controller.text.isNotEmpty) {
                context.read<TodoBloc>().add(AddTodo(controller.text));
                Navigator.pop(dialogContext);
              }
            },
            child: Text('Add'),
          ),
        ],
      ),
    );
  }
}

Advantages of BLoC Architecture

1. Separation of Concerns

Business logic is completely isolated from UI code, making both easier to maintain and modify independently.

2. Exceptional Testability

BLoCs are plain Dart classes that can be unit tested without any widget testing infrastructure.

void main() {
  group('TodoBloc', () {
    late TodoBloc todoBloc;
    late TodoRepository repository;
    
    setUp(() {
      repository = MockTodoRepository();
      todoBloc = TodoBloc(repository: repository);
    });
    
    test('Initial state should be TodoInitial', () {
      expect(todoBloc.state, isA<TodoInitial>());
    });
    
    test('LoadTodos should emit TodoLoading then TodoLoaded', () async {
      // Arrange
      final todos = [Todo(id: '1', title: 'Test')];
      when(() => repository.getTodos()).thenAnswer((_) async => todos);
      
      // Assert
      expectLater(
        todoBloc.stream,
        emitsInOrder([
          isA<TodoLoading>(),
          isA<TodoLoaded>(),
        ]),
      );
      
      // Act
      todoBloc.add(LoadTodos());
    });
  });
}

3. Reusability

A single BLoC can be shared across multiple widgets and even different platforms (web, mobile, desktop).

4. Scalability

As your app grows, BLoC architecture scales gracefully. Adding new features doesn’t impact existing code.

5. Predictable State Management

Unidirectional data flow makes it easy to understand how data moves through your app and simplifies debugging.

6. Time Travel Debugging

You can track all state changes, making it easy to trace how you arrived at the current state.

Disadvantages to Consider

1. Boilerplate Code

You need to create multiple files for events, states, and BLoC classes. This can feel like overkill for simple apps.

2. Learning Curve

Beginners may find BLoC concepts challenging initially. Understanding streams and reactive programming is essential.

3. Development Time

Initial setup takes longer compared to simpler state management solutions like Provider.

BLoC vs Other State Management Solutions

BLoC vs Provider

Provider:

• Simpler and more straightforward
• Less boilerplate code
• Good for small to medium apps

BLoC:

• More structured and scalable
• Better for large applications
• Easier to test
• Better for team collaboration

BLoC vs Riverpod

Riverpod:

• Modern and flexible
• Compile-time safety
• Less boilerplate than BLoC

BLoC:

• Well-established and mature
• Larger community
• Better documentation
• More opinionated structure

BLoC vs GetX

GetX:

• All-in-one solution (routing, state management, dependency injection)
• Less code required
• Faster development

BLoC:

• Follows the single responsibility principle
• Better separation of concerns
• More testable
• Industry standard

Best Practices

1. Use Equatable for State Comparison

import 'package:equatable/equatable.dart';

class TodoLoaded extends Equatable {
  final List<Todo> todos;
  
  const TodoLoaded(this.todos);
  
  @override
  List<Object> get props => [todos];
}

Equatable helps BLoC accurately compare states to determine if a rebuild is necessary.

2. One BLoC Per Feature

Create one BLoC for each feature:

• AuthBloc for authentication
• ProfileBloc for user profile
• TodoBloc for todo management

3. Use the Repository Pattern

Abstract your data sources behind repositories. BLoCs shouldn’t know where data comes from.

class TodoRepository {
  final TodoApiClient apiClient;
  final TodoLocalDatabase database;
  
  TodoRepository({
    required this.apiClient,
    required this.database,
  });
  
  Future<List<Todo>> getTodos() async {
    try {
      // Try to get from API
      return await apiClient.fetchTodos();
    } catch (e) {
      // Fallback to local database
      return await database.getTodos();
    }
  }
}

4. Handle Errors Properly

Always use try-catch blocks in event handlers.

on<LoadTodos>((event, emit) async {
  emit(TodoLoading());
  try {
    final todos = await repository.getTodos();
    emit(TodoLoaded(todos));
  } on NetworkException catch (e) {
    emit(TodoError('Network error: ${e.message}'));
  } on CacheException catch (e) {
    emit(TodoError('Cache error: ${e.message}'));
  } catch (e) {
    emit(TodoError('Unknown error: ${e.toString()}'));
  }
});

5. Use BlocObserver for Debugging

class SimpleBlocObserver extends BlocObserver {
  @override
  void onEvent(Bloc bloc, Object? event) {
    super.onEvent(bloc, event);
    print('Event: $event');
  }
  
  @override
  void onTransition(Bloc bloc, Transition transition) {
    super.onTransition(bloc, transition);
    print('Transition: $transition');
  }
  
  @override
  void onError(BlocBase bloc, Object error, StackTrace stackTrace) {
    print('Error: $error');
    super.onError(bloc, error, stackTrace);
  }
}

void main() {
  Bloc.observer = SimpleBlocObserver();
  runApp(MyApp());
}

6. Keep States Immutable

States should always be immutable. Create new instances instead of modifying existing ones.

// ❌ Wrong
class TodoState {
  List<Todo> todos;
  TodoState(this.todos);
}

// ✅ Correct
class TodoState {
  final List<Todo> todos;
  const TodoState(this.todos);
}

7. Close BLoCs Properly

Always close BLoCs when done to prevent memory leaks.

@override
void dispose() {
  _todoBloc.close();
  super.dispose();
}

Recommended Project Structure

lib/
├── data/
│   ├── models/
│   │   └── todo.dart
│   ├── repositories/
│   │   └── todo_repository.dart
│   └── data_sources/
│       ├── todo_api_client.dart
│       └── todo_local_database.dart
├── logic/
│   └── bloc/
│       ├── auth/
│       │   ├── auth_bloc.dart
│       │   ├── auth_event.dart
│       │   └── auth_state.dart
│       └── todo/
│           ├── todo_bloc.dart
│           ├── todo_event.dart
│           └── todo_state.dart
├── presentation/
│   ├── pages/
│   │   ├── home_page.dart
│   │   ├── login_page.dart
│   │   └── todo_page.dart
│   └── widgets/
│       ├── todo_item.dart
│       └── todo_list.dart
└── main.dart

Structure Explanation:

• data/ - Data layer (models, repositories, data sources)
• logic/ - Business logic layer (BLoCs)
• presentation/ - UI layer (pages, widgets)

Advanced Topics

1. BLoC to BLoC Communication

When one BLoC needs data from another:

class ProfileBloc extends Bloc<ProfileEvent, ProfileState> {
  final AuthBloc authBloc;
  late StreamSubscription authSubscription;
  
  ProfileBloc({required this.authBloc}) : super(ProfileInitial()) {
    // Listen to AuthBloc state changes
    authSubscription = authBloc.stream.listen((authState) {
      if (authState is AuthAuthenticated) {
        add(LoadProfile(authState.user.id));
      }
    });
  }
  
  @override
  Future<void> close() {
    authSubscription.cancel();
    return super.close();
  }
}

2. Hydrated BLoC — State Persistence

Restore state when the app is reopened:

class TodoBloc extends HydratedBloc<TodoEvent, TodoState> {
  TodoBloc() : super(TodoInitial());
  
  @override
  TodoState? fromJson(Map<String, dynamic> json) {
    try {
      final todos = (json['todos'] as List)
          .map((e) => Todo.fromJson(e))
          .toList();
      return TodoLoaded(todos);
    } catch (_) {
      return null;
    }
  }
  
  @override
  Map<String, dynamic>? toJson(TodoState state) {
    if (state is TodoLoaded) {
      return {
        'todos': state.todos.map((e) => e.toJson()).toList(),
      };
    }
    return null;
  }
}

3. Debouncing Events

Prevent excessive API calls for features like search:

class SearchBloc extends Bloc<SearchEvent, SearchState> {
  SearchBloc() : super(SearchInitial()) {
    on<SearchTermChanged>(
      (event, emit) async {
        emit(SearchLoading());
        try {
          final results = await repository.search(event.term);
          emit(SearchLoaded(results));
        } catch (e) {
          emit(SearchError(e.toString()));
        }
      },
      transformer: debounce(Duration(milliseconds: 500)),
    );
  }
}

EventTransformer<E> debounce<E>(Duration duration) {
  return (events, mapper) => events.debounceTime(duration).flatMap(mapper);
}

4. Throttling Events

Limit event processing rate (useful for infinite scroll):

on<LoadMoreTodos>(
  (event, emit) async {
    // Load more logic
  },
  transformer: throttle(Duration(seconds: 1)),
);

EventTransformer<E> throttle<E>(Duration duration) {
  return (events, mapper) => events.throttleTime(duration).flatMap(mapper);
}

When Should You Use BLoC?

Use BLoC when:

• Building medium to large-scale applications
• Working in a team environment
• Testability is a priority
• You need a clear separation of concerns
• The app will scale and grow over time
• You’re building for multiple platforms

Consider alternatives when:

• Building a simple prototype or MVP
• Working solo on a small project
• Quick development is more important than structure
• Your team is unfamiliar with reactive programming

Getting Started with BLoC

1. Add Dependencies

dependencies:
  flutter_bloc: ^8.1.3
  equatable: ^2.0.5
dev_dependencies:
  bloc_test: ^9.1.4
  mocktail: ^1.0.0

2. Install VS Code Extension

Install the “Bloc” extension by Felix Angelov for automatic code generation.

3. Generate BLoC Files

Use the command palette (Cmd/Ctrl + Shift + P) and select “Bloc: New Bloc” to generate boilerplate code automatically.

4. Learn from Official Resources

• Official Documentation: https://bloclibrary.dev
• GitHub Repository: https://github.com/felangel/bloc
• Video Tutorials: Flutter Official Channel

Conclusion

BLoC architecture is a powerful pattern that brings structure, testability, and scalability to Flutter applications. While it comes with a learning curve and requires more boilerplate code, the benefits far outweigh the costs for medium to large applications.

Key takeaways:

• BLoC separates business logic from UI completely
• Events represent user actions, States represent UI conditions
• The pattern makes testing straightforward and effective
• It scales beautifully as your application grows
• The repository pattern keeps data sources abstracted

Start with a small feature, implement it with BLoC, and gradually expand. Once you understand the pattern, you’ll find it natural and highly productive.

Happy coding! 🚀

Resources

• Official BLoC Documentation
• Flutter BLoC Package
• Bloc Test Package
• Felix Angelov’s GitHub
• Flutter Architecture Samples

If you found this guide helpful, please give it a clap and share it with fellow Flutter developers!

If you’re a cloud engineer, DevOps practitioner, or aspiring to enter this field, here’s what you need to know about the trends reshaping the industry and the skills that will make you invaluable.

1. Platform Engineering Is Eating DevOps

Remember when everyone wanted to hire DevOps engineers? The job market is shifting. By 2026, roughly 80% of large software organizations will adopt internal developer platforms (IDPs), fundamentally changing what employers seek in cloud talent.

What’s changing: Companies are moving away from bespoke DevOps pipelines toward standardized, self-service platforms. Instead of each team maintaining their own deployment scripts, organizations are building centralized platform engineering layers that let developers deploy and manage environments independently.

The GitOps revolution: This shift centers on GitOps — using Git as the single source of truth for code, infrastructure, configurations, and deployments. The numbers tell the story: 64% of organizations have already implemented GitOps, with most reporting higher reliability and faster rollbacks. Tools like Argo CD and Flux automatically sync your cluster state with your Git repository, making deployments consistent and recoverable.

Why it matters: Platform engineers now earn approximately 20% more than traditional DevOps engineers. The role combines infrastructure expertise with product thinking — you’re not just building systems, you’re building platforms that other engineers use.

Skills to develop:

• GitOps workflows with Argo CD or Flux CD
• Kubernetes as a platform foundation
• Internal developer platform (IDP) design
• Self-service infrastructure patterns

2. Infrastructure as Code Grows Up (Welcome to IaC 2.0)

Writing Terraform scripts is table stakes. The industry is moving toward “Infrastructure as APIs” — what analysts call IaC 2.0.

The new paradigm: Instead of writing monolithic infrastructure scripts for every project, modern teams treat infrastructure as programmable, reusable services. Define a VPC, database, or Kubernetes cluster once, then let applications request them like APIs.

Tools like Crossplane turn Kubernetes into a control plane for all your cloud resources, while Pulumi lets you write infrastructure in real programming languages (Python, TypeScript, Go) with full testing and CI/CD integration. This isn’t just about convenience — it’s about treating infrastructure with the same engineering rigor as application code.

The evolution includes:

• Policy-as-code enforcement (Open Policy Agent, Sentinel)
• Automated testing of infrastructure changes
• Full-featured programming languages instead of limited DSLs
• Modular, versioned infrastructure libraries

Why it matters: Teams avoid repeatedly rewriting templates and gain on-demand, multi-cloud infrastructure that scales safely. Infrastructure becomes as dynamic and testable as your application code.

Skills to develop:

• Pulumi or AWS CDK for code-based IaC
• Crossplane for Kubernetes-native infrastructure
• Policy-as-code with OPA
• Infrastructure testing frameworks

3. Observability Meets AI: The AIOps Explosion

Logs and dashboards aren’t enough anymore. Modern cloud systems demand full observability — correlating metrics, logs, and traces end-to-end to truly “see inside” distributed services.

The observability stack: Teams are standardizing on tools like Prometheus for metrics, Grafana for visualization, and OpenTelemetry for distributed tracing. The payoff is significant: mature observability can cut mean time to recovery by approximately 40%.

But here’s where it gets interesting: AI is transforming operations. AIOps adoption jumped from 42% in 2024 to 54% in 2025, and the trend is accelerating.

What AI brings to ops:

• Automatic anomaly detection across millions of metrics
• Predictive failure analysis before outages occur
• AI-assisted root cause analysis
• Automated remediation for common issues
• Dynamic resource scaling based on predicted load

Some teams are already using AI to auto-fix security vulnerabilities or scale infrastructure proactively. Tools like Datadog Watchdog, Dynatrace, and cloud-native AIOps features are moving teams from reactive firefighting to proactive operations.

Why it matters: AI-enabled observability has been linked to halving outage costs and drastically improving uptime. In 2026’s complex distributed systems, human-only monitoring simply won’t scale.

Skills to develop:

• Prometheus, Grafana, and OpenTelemetry
• Distributed tracing patterns
• AI-powered monitoring platforms (Datadog, Dynatrace, New Relic)
• Incident management with AI-assisted workflows

4. AI Infrastructure: The New Cloud Frontier

If you’re not thinking about AI infrastructure, you’re already behind. Global AI infrastructure spending will exceed $2 trillion by 2026, and over half of enterprises will deploy AI agents by 2027.

The MLOps challenge: Deploying AI models in production requires an entirely new skillset — provisioning GPU clusters, serving models with low latency, monitoring model drift, optimizing inference costs. It’s cloud engineering meets machine learning operations.

What this looks like in practice:

• Containerizing ML models for production deployment
• Scheduling GPU workloads on Kubernetes
• Managing model serving with tools like NVIDIA Triton or KServe
• Building real-time inference pipelines
• Operating vector databases and feature stores

Kubernetes itself is adapting to AI workloads, with GPU scheduling and specialized operators becoming standard. Cloud platforms are racing to offer managed ML services (AWS SageMaker, Azure ML, Google Vertex), but enterprises still need engineers who understand both cloud and ML to orchestrate it all.

Why it matters: MLOps job openings grew approximately 10-fold over five years. As businesses embed AI in products — from chatbots to recommendation engines to autonomous systems — they desperately need engineers who can bridge cloud infrastructure and machine learning.

Skills to develop:

• Kubernetes with GPU node management
• Model serving frameworks (Triton, KServe, Kubeflow)
• ML orchestration tools like Ray
• Understanding of model performance and retraining pipelines
• Vector databases and ML data infrastructure

5. Event-Driven and Serverless: The New Default

Synchronous request-response architectures are giving way to event-driven systems. By 2026, asynchronous communication will be the default for high-scale applications.

The pattern: Applications break into decoupled services that react to events — user actions, data changes, system triggers — through message queues and event buses. Platforms like Apache Kafka, AWS EventBridge, and Google Pub/Sub enable services to publish and subscribe to event streams.

Serverless functions amplify this: AWS Lambda, Google Cloud Functions, and Azure Functions automatically execute code in response to events, with sub-10-minute deployment times and pay-per-use pricing. Serverless usage jumped 25% in 2024 alone.

Real-world applications:

• CI/CD pipelines triggered by Git events
• Real-time data processing streams
• On-demand AI inference triggered by data events
• Automated incident response workflows

Teams using event-driven orchestration report cutting mean time to fix from hours to minutes.

Why it matters: Event-driven and serverless architectures deliver better performance, reliability, and cost efficiency. For high-volume microservices, IoT pipelines, or systems with variable load, this pattern is becoming essential.

Skills to develop:

• Event streaming platforms (Kafka, RabbitMQ)
• Serverless function development and optimization
• Event-driven architecture patterns
• Workflow orchestration (Argo Workflows, Tekton)

Regional Perspectives: Where the Opportunities Are

Cloud adoption isn’t uniform globally, and understanding regional trends can guide your career strategy.

North America remains the largest market, with leading innovation in platform engineering and AI projects. IDP/GitOps and AIOps skills are highly sought in tech hubs like Silicon Valley, Seattle, and New York.

Europe shows strong adoption with heightened focus on security and compliance. Companies invest heavily in internal cloud platforms to balance agility with strict data regulations.

Asia-Pacific is experiencing explosive growth and leads in AI adoption. Nearly 90% of APAC enterprises now run multi-cloud workloads, and organizations there often embrace cutting-edge trends (serverless, service mesh, WebAssembly) faster than Western counterparts.

Emerging markets in Latin America, Middle East, and Africa are accelerating cloud adoption from a smaller base, driven by government cloud initiatives and digital transformation programs.

The Bottom Line: What to Learn Now

The most in-demand cloud roles for 2026 include:

Platform/DevOps Engineers with platform engineering focus — designing IDPs and GitOps workflows. These roles now rival classic DevOps jobs in frequency.

Cloud SREs specializing in observability and AIOps — managing complex distributed systems with AI-augmented tools.

Machine Learning/AI Ops Engineers — bridging cloud infrastructure and ML model deployment at scale.

Cloud Software Engineers skilled in serverless and event-driven design — building the next generation of scalable applications.

All these roles command premium salaries, with platform engineers earning significantly more than traditional DevOps practitioners.

Start Today

The cloud industry rewards those who stay ahead of the curve. Basic containerization and simple CI/CD are now baseline expectations. The engineers who thrive in 2026 will be those who master platform thinking, embrace AI-powered operations, and understand how to build self-service infrastructure at scale.

The trends are clear. The tools are available. The question is: are you ready to level up?

What cloud skills are you focusing on for 2026? Share your thoughts in the comments below.

Deploying a full-stack application is easy when you stick to one platform. But what happens when you want to optimize costs and performance by splitting your architecture?

Recently, I migrated my application — a comprehensive mobile distribution system — from a simple development setup to a professional production architecture. I decided to host my Frontend on Render (for easy CD/CI) and my Backend on a Linux VPS (for full control and performance).

It sounds great on paper, but I hit several roadblocks involving DNS configuration, SSL termination, and the dreaded CORS errors.

Here is the story of my migration, the embarrassing mistakes I made, and how I finally fixed them.

🏗️ The Goal Architecture

Instead of a monolithic deployment, I wanted a “Headless” approach:

1. Frontend (React): Hosted on Render. It serves the UI and connects to the backend API.
2. Backend (Node.js/Docker): Hosted on a Linux VPS. It handles the database, business logic, and API requests.
3. Domain: One custom domain (awesome-startup.com) to rule them all.

The target URLs:

• Frontend: https://awesome-startup.com
• Backend: https://api.awesome-startup.com

🚫 The “Oops” Moments (My Mistakes)

If you are new to DevOps, you might recognize some of these.

Mistake 1: The “Forwarding” Trap

I bought my domain on GoDaddy and saw a feature called “Forwarding”. I thought, “Great! I’ll just forward awesome-startup.com to my Render URL.”

Why it failed: Forwarding just redirects the user. The browser address bar changed from my custom domain to myapp.onrender.com. It looked unprofessional and messed up my cookies.

Lesson: Never use Forwarding for a production site. Always use DNS Records.

Mistake 2: The “Split Brain” DNS

I wanted my domain to work for both the frontend and backend, so I pointed the root record (@) to Render’s IP… and also to my VPS IP.

Why it failed: DNS doesn’t work like a load balancer. The internet didn’t know which server to send traffic to. Sometimes the site loaded (Render), sometimes it showed a 404 (VPS).

Lesson: The root domain (@) can only point to one server. You must use subdomains (like api.) for the second server.

Mistake 3: The Double CORS Nightmare

This was the hardest to debug. I configured Nginx on my VPS to add Access-Control-Allow-Origin headers. But I also had my Node.js app (using the cors package) adding the same headers.

The Result: The browser received two sets of headers:

Access-Control-Allow-Origin: https://awesome-startup.com, https://awesome-startup.com

Chrome immediately blocked the request because duplicate headers are invalid.

✅ The Solution: A Clean, Secure Setup

Here is the step-by-step configuration that finally worked.

Step 1: DNS Configuration (The Roadmap)

I cleaned up my DNS records to strictly separate traffic.

Step 2: Nginx Reverse Proxy (The Traffic Cop)

On the VPS, I used Nginx to listen for the api subdomain and forward traffic to my Docker container running on port 5000.

Crucial detail: I removed all manual CORS headers from Nginx and let the application handle it.

server {
server_name api.awesome-startup.com;

location / {
proxy_pass [http://127.0.0.1:5000](http://127.0.0.1:5000);

# Standard Proxy Headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

# NOTE: No CORS headers here!
}
}

Then, I used Certbot to automatically generate SSL certificates:

sudo certbot — nginx -d api.awesome-startup.com

Step 3: Connecting the App (.env)

Finally, I had to tell my backend who to trust. Inside my Docker environment variables, I set the CORS origin to my new frontend domain.

NODE_ENV=production
CORS_ORIGIN=[https://awesome-startup.com](https://awesome-startup.com)

🚀 The Result

• Security: Both Frontend and Backend are fully encrypted (HTTPS).
• Performance: The frontend is served via Render’s CDN, while the backend runs on dedicated compute power.
• Professionalism: Users never see an IP address or a duckdns URL. They only see awesome-startup.com.

Moving from localhost to production is rarely a straight line. It involves understanding how the internet actually routes traffic (DNS), how servers talk to each other (Nginx/Proxy), and how browsers protect users (CORS).

Hopefully, sharing my “oops” moments helps you avoid them in your next deployment!

#DevOps #WebDevelopment #React #NodeJS #Docker #Nginx #Deployment #FullStack

Challenge 1: Automated Website Backup System

The Problem

The xFusionCorp Industries production support team needed to create an automated backup solution for their static website running on App Server 1. The requirements were comprehensive:

• Create a bash script for automated backups
• Generate zip archives of the /var/www/html/beta directory
• Store backups both locally and on a remote backup server
• Ensure password-less file transfers
• Make the solution executable by the appropriate server user

The Solution Approach

Script Architecture

The team developed a comprehensive bash script (beta_backup.sh) that addressed all requirements:

#!/bin/bash
# Website Backup Script for xFusionCorp Industries

# Variables for configuration
SOURCE_DIR="/var/www/html/beta"
ARCHIVE_NAME="xfusioncorp_beta.zip"
LOCAL_BACKUP_DIR="/backup"
BACKUP_SERVER="stbkp01.stratos.xfusioncorp.com"
BACKUP_SERVER_USER="clint"
REMOTE_BACKUP_DIR="/backup"
LOG_FILE="/var/log/beta_backup.log"
# Logging function for audit trails
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

Key Implementation Features

1. Error Handling and Validation

• Source directory existence checks
• Backup directory creation with proper permissions
• Archive creation validation
• File size reporting for verification

2. Security Implementation

• SSH key-based authentication for password-less transfers
• Proper file permissions (600 for private keys)
• User-specific ownership settings

3. Logging and Monitoring

• Comprehensive logging to /var/log/beta_backup.log
• Timestamped entries for audit trails
• Success/failure status reporting

Deployment Process

The implementation followed a structured approach:

1. Environment Setup

# Create scripts directory

sudo mkdir -p /scripts

# Set proper ownership

sudo chown tony:tony /scripts/beta_backup.sh

sudo chmod +x /scripts/beta_backup.sh

1. SSH Key Configuration

# Generate SSH key pair

ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -N ""

# Copy public key to backup server

ssh-copy-id clint@stbkp01.stratos.xfusioncorp.com

1. Testing and Validation

• Local backup verification
• Remote backup confirmation
• Log file analysis

Results and Benefits

The solution achieved:

• 100% automation of the backup process
• Zero password prompts during execution
• Comprehensive logging for troubleshooting
• Scalable architecture for multiple websites
• Audit-ready documentation of all backup operations

Challenge 2: Cloud Infrastructure Migration with Terraform

The Problem

The Nautilus DevOps team faced a complex AWS cloud migration project. Rather than attempting a massive single migration, they adopted an incremental approach, starting with fundamental infrastructure components like SSH key pair management.

Requirements:

• Create an RSA key pair named “datacenter-kp”
• Save the private key to /home/bob/datacenter-kp.pem
• Use Terraform for infrastructure as code
• Ensure proper security configurations

The Solution Journey

Initial Challenges

The team encountered several technical hurdles:

1. Provider Configuration Conflicts

• Duplicate required providers blocks
• Missing TLS and local provider dependencies

2. Resource Type Confusion

• Initially created only local TLS keys
• Validation system expected AWS key pair resources

Final Working Solution

# Generate RSA private key
resource "tls_private_key" "datacenter_key" {
  algorithm = "RSA"
  rsa_bits  = 2048
}

# Create AWS Key Pair using the generated public key
resource "aws_key_pair" "datacenter_kp" {
  key_name   = "datacenter-kp"
  public_key = tls_private_key.datacenter_key.public_key_openssh
}

# Save the private key to file
resource "local_file" "private_key" {
  content  = tls_private_key.datacenter_key.private_key_pem
  filename = "/home/bob/datacenter-kp.pem"
  file_permission = "0600"
}

Key Learning Points

1. Infrastructure as Code Best Practices

• Clear separation of concerns between providers
• Proper resource dependencies
• Security-first approach with file permissions

2. Validation Requirements Understanding

• AWS resources vs. local resources
• Proper naming conventions
• Type specifications (“rsa” type requirement)

3. Provider Configuration Management

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "5.91.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 4.0"
    }
    local = {
      source  = "hashicorp/local"
      version = "~> 2.4"
    }
  }
}

Challenge 3: Java Application Deployment with Tomcat

The Problem

The application development team needed to deploy a Java-based application (ROOT.war) on App Server 2 using Tomcat, with specific requirements:

• Custom port configuration (5001)
• Proper application deployment
• Direct base URL access

The Solution Implementation

Installation and Configuration Process

1. Environment Preparation

# Java installation verification
java -version

# Tomcat download and extraction
wget https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.75/bin/apache-tomcat-9.0.75.tar.gz
sudo tar -xzf apache-tomcat-9.0.75.tar.gz
sudo mv apache-tomcat-9.0.75 /opt/tomcat

2. Security Configuration

# Create dedicated tomcat user
sudo useradd -r -m -U -d /opt/tomcat -s /bin/false tomcat

# Set proper ownership and permissions
sudo chown -R tomcat:tomcat /opt/tomcat/
sudo chmod +x /opt/tomcat/bin/*.sh

3. Service Configuration Created a systemd service for proper process management:

[Unit]
Description=Apache Tomcat Web Application Container
After=network.target

[Service]
Type=forking
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh
User=tomcat
Group=tomcat

4. Application Deployment

# Deploy ROOT.war application
sudo cp /tmp/ROOT.war /opt/tomcat/webapps/
sudo chown tomcat:tomcat /opt/tomcat/webapps/ROOT.war

# Start and enable service
sudo systemctl enable tomcat
sudo systemctl start tomcat

Key Takeaways and Best Practices

1. Automation First Approach

• Comprehensive scripting eliminates human error
• Logging and monitoring provide visibility
• Error handling ensures reliability

2. Security by Design

• SSH key authentication over password-based access
• Proper file permissions (600 for private keys)
• Dedicated service users with minimal privileges

3. Infrastructure as Code Benefits

• Version control for infrastructure changes
• Reproducible deployments across environments
• Validation and testing before production

4. Incremental Migration Strategy

• Break complex tasks into manageable components
• Test each component independently
• Build confidence through small successes

5. Troubleshooting Methodology

• Systematic approach to problem identification
• Logging analysis for root cause determination
• Iterative testing until resolution

Conclusion

These real-world scenarios demonstrate the complexity and variety of challenges faced in modern DevOps environments. The solutions showcase the importance of:

• Thorough planning and requirement analysis
• Security-first implementation approaches
• Comprehensive testing and validation
• Documentation and logging for maintenance

By following these patterns and practices, DevOps teams can build robust, scalable, and maintainable infrastructure solutions that support business growth and operational excellence.

The key to success lies in understanding that each challenge is an opportunity to improve processes, enhance automation, and build more resilient systems. Whether it’s backup automation, infrastructure provisioning, or application deployment, the principles of careful planning, security consciousness, and systematic implementation remain constant across all scenarios.

The Challenge

Our security team identified a significant vulnerability:

• Apache running on port 6200 across multiple app servers
• No firewall protection — port accessible from anywhere
• Need selective access — only the Load Balancer (LBR) should reach the app servers
• Persistence required — rules must survive system reboots

Infrastructure Overview

The environment consisted of:

• 3 Application Servers: stapp01, stapp02, stapp03 (CentOS Stream 9)
• 1 Load Balancer: stlb01 (172.16.238.14)
• 1 Jump Host: For administrative access
• Apache running on port 6200 on all app servers

Step-by-Step Implementation

Phase 1: Installing iptables

First, I needed to install iptables and its services on all app servers. Modern CentOS systems often use firewalld, but for this specific requirement, iptables provided the granular control we needed.

# SSH to each app server
ssh tony@stapp01    # Password: Ir0nM@n
ssh steve@stapp02   # Password: Am3ric@
ssh banner@stapp03  # Password: BigGr33n

# Install iptables and iptables-services
sudo yum install -y iptables iptables-services

# Enable and start the service
sudo systemctl enable iptables
sudo systemctl start iptables

Key Learning: Always install iptables-services package - it provides the systemd service files needed for persistence and automatic startup.

Phase 2: Designing the Security Rules

The security policy required:

1. Allow SSH (port 22) from anywhere for administrative access
2. Allow port 6200 only from LBR (172.16.238.14)
3. Block port 6200 from all other sources
4. Maintain existing connectivity for established connections

Here’s the rule configuration I implemented:

# Clear existing rules and set permissive defaults
sudo iptables -F
sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

# Allow loopback traffic (essential for system processes)
sudo iptables -A INPUT -i lo -j ACCEPT

# Allow established and related connections
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from anywhere
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# CRITICAL: Allow port 6200 ONLY from LBR host
sudo iptables -A INPUT -p tcp --dport 6200 -s 172.16.238.14 -j ACCEPT

# Block port 6200 from all other sources
sudo iptables -A INPUT -p tcp --dport 6200 -j DROP

# Save rules for persistence
sudo service iptables save

Phase 3: The Critical Issue I Encountered

During the initial setup on stapp01, I encountered an unexpected problem:

sudo systemctl status httpd
● httpd.service - The Apache HTTP Server
   Active: failed (Result: exit-code)
   
Aug 16 18:23:43 stapp01 httpd[510]: (98)Address already in use: 
AH00072: make_sock: could not bind to address 0.0.0.0:3000

The Problem: Port 3000 (not 6200!) was already in use by sendmail, preventing Apache from starting.

The Solution:

# Identify the conflicting process
sudo netstat -tlnp | grep :3000
tcp 0 0 127.0.0.1:3000 0.0.0.0:* LISTEN 449/sendmail: accep

# Stop the conflicting service
sudo systemctl stop sendmail
sudo systemctl disable sendmail

# Start Apache successfully
sudo systemctl start httpd
sudo systemctl enable httpd

Lesson Learned: Always check for port conflicts before assuming firewall issues. Use netstat -tlnp to identify processes using specific ports.

Phase 4: Rule Order Matters

The order of iptables rules is crucial. My rules were structured as:

1. Allow loopback (essential system traffic)
2. Allow established connections (don’t break existing sessions)
3. Allow SSH (maintain administrative access)
4. Allow specific port from specific source (targeted access)
5. Block specific port from everywhere else (security enforcement)

This order ensures that legitimate traffic flows through while blocking unauthorized access.

Phase 5: Testing and Verification

Testing revealed the effectiveness of the configuration:

From Jump Host (unauthorized source):

telnet stapp01 6200
Trying 172.16.238.10...
telnet: connect to address 172.16.238.10: Connection timed out ✓

From LBR Host (authorized source):

telnet stapp01 6200
Trying 172.16.238.10...
Connected to stapp01. ✓

The HTTP responses from the app servers confirmed Apache was running and accessible only from the designated source.

Key Troubleshooting Tips

1. Port Conflict Resolution

# Always check what's using a port
sudo netstat -tlnp | grep :PORT_NUMBER
# or if lsof is available
sudo lsof -i :PORT_NUMBER

2. Rule Verification

# View rules with line numbers
sudo iptables -L -n --line-numbers
# Check specific chains
sudo iptables -L INPUT -n

3. Service Status Checks

# Verify iptables service
sudo systemctl status iptables
# Check rule persistence
sudo cat /etc/sysconfig/iptables

Best Practices Learned

1. Always Plan for SSH Access

Never block SSH without an alternative access method. Always include:

sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

2. Test Before Implementing

Use temporary rules first, then make them persistent:

# Test first (temporary)
sudo iptables -A INPUT -p tcp --dport 6200 -s 172.16.238.14 -j ACCEPT
# Only save when confirmed working
sudo service iptables save

3. Document Your Rules

Always document the purpose of each rule:

# Allow HTTP from LBR only - Security requirement SR-2025-001
sudo iptables -A INPUT -p tcp --dport 6200 -s 172.16.238.14 -j ACCEPT

4. Monitor After Implementation

Set up monitoring to detect any issues:

# Check dropped connections
sudo iptables -L -v -n | grep DROP

Results and Impact

The implementation successfully achieved all security objectives:

• ✅ Port 6200 secured: Only accessible from authorized LBR host
• ✅ Zero service disruption: All legitimate traffic maintained
• ✅ Persistent configuration: Rules survive system reboots
• ✅ Administrative access preserved: SSH remains accessible

Performance Considerations

The iptables rules added minimal overhead:

• Rule processing: O(n) complexity, but with only 5–6 rules per server
• Memory usage: Negligible impact on system resources
• Network latency: No measurable increase in response times

Conclusion

Implementing selective firewall access using iptables proved to be an effective security measure. The key to success was:

1. Thorough planning of rule structure and order
2. Identifying and resolving port conflicts proactively
3. Comprehensive testing from both authorized and unauthorized sources
4. Proper persistence configuration to ensure rules survive reboots

This implementation demonstrates that with careful planning and execution, significant security improvements can be achieved without impacting legitimate system functionality. The selective access control now ensures that our Apache servers are protected while maintaining the necessary connectivity for our load balancer infrastructure.

Next Steps

Future enhancements could include:

• Logging dropped connections for security monitoring
• Rate limiting to prevent brute force attacks
• Integration with monitoring systems for alert generation
• Automated backup of iptables configurations

The foundation is now in place for a more secure and manageable infrastructure that meets both security and operational requirements.

Want to stay ahead in the fast-paced world of machine learning? LinkedIn is a goldmine of knowledge, and following the right people can be a game-changer for your learning journey and career path. Based on our recent research, we’ve compiled a list of top machine learning influencers you should definitely have in your feed. Forget just names; we’re diving into what makes them influential and why you should care.

Deep Research Link: https://docs.google.com/document/d/1hQdwpCllghWI04eAbhdvNhXVCDcmc0xGWzjcvWuM4FE/edit?usp=sharing

The Foundational Thinkers: Shaping the Future of AI

These are the pioneers who laid the groundwork for modern AI:

• Andrew Ng 🧑‍🏫 (linkedin.com/in/andrewng): The master educator! Follow him for accessible explanations of complex AI concepts, practical applications, and insights into the latest advancements. His work democratizing AI education is unparalleled.
• Yann LeCun 🧠 (linkedin.com/in/yannlecun): A true visionary and one of the “Godfathers of AI.” Expect cutting-edge research, theoretical discussions, and his perspective on the societal impact of AI.
• Fei-Fei Li: A champion for human-centered AI and the creator of ImageNet. Her insights blend technical expertise with a focus on ethical and responsible AI development.
• Demis Hassabis: The mind behind Google DeepMind. Follow him to see AI’s potential in tackling complex challenges, from healthcare to achieving artificial general intelligence.

Why follow them? These individuals provide a deep understanding of the fundamental principles and the long-term vision of AI. They’re essential for grasping the “why” behind the “how.”

The Builders: Turning Theory into Reality

These are the hands-on experts who are building and deploying ML systems:

• Andrej Karpathy: Known for his practical approach and ability to explain complex topics simply. His content, including his popular YouTube tutorials, focuses on applied AI and MLOps.
• Soumith Chintala (linkedin.com/in/soumithchintala): Co-creator of PyTorch, a cornerstone of modern deep learning. Following him gives you a direct line to the evolution of crucial ML tools.
• Mike Tamir (linkedin.com/in/miketamir): Bridges the gap between academia and industry, offering insights into real-world ML applications.
• Peter Skomoroch (linkedin.com/in/peteskomoroch): A veteran in building data-driven products, his profile is a must-follow for those interested in creating impactful ML solutions at scale.

Why follow them? Learn the nuts and bolts of machine learning engineering and gain practical knowledge you can apply directly to your projects.

The Strategists and Communicators: Making AI Understandable

These leaders bridge the gap between complex tech and real-world impact:

• Allie K. Miller (linkedin.com/in/alliekmiller): The go-to voice for AI in business. Follow her for insights on AI startups, investment trends, and how businesses are leveraging AI for growth.
• Bernard Marr (linkedin.com/in/bernardmarr): A futurist who helps organizations understand and implement AI strategies for better performance.
• Cassie Kozyrkov: The pioneer of “Decision Intelligence.” Her posts focus on making data-driven decisions and understanding the practical application of AI in business.
• Kirk Borne (linkedin.com/in/kirkdborne): A data science veteran who shares valuable articles and insights across the data science and AI landscape.

Why follow them? Understand the business implications of AI and learn how to effectively communicate its value.

The Community and Ethics Advocates: Shaping a Responsible Future

These influencers focus on making AI accessible and ethical:

• Steve Nouri: A strong advocate for diversity and ethics in AI. Follow him for discussions on building responsible and inclusive AI systems.
• Sebastian Raschka: A leading AI educator who shares tutorials and practical insights into machine learning and deep learning.
• Kate Strachnyi (linkedin.com/in/katestrachnyi): Focuses on data visualization, storytelling, and building a strong data science community.

Why follow them? Learn about the crucial ethical considerations in AI and connect with a supportive community of learners.

Building Your Path

“Following the path” in machine learning isn’t about blindly copying someone’s journey. It’s about learning from diverse experts, understanding different facets of the field, and carving your own niche. Use this list as a starting point to curate your LinkedIn feed and actively engage with their content. Ask questions, participate in discussions, and most importantly, keep learning! Your AI journey starts now.

Ready to level up? Start connecting with these influential figures today!