Juggling useState booleans for complex UI creates “impossible states” and bugs. Finite state machines enforce exactly-one-state-at-a-time semantics, eliminating this problem. Use useReduce for simple state machines, or XState for complex orchestration with guards, timers, and hierarchies.

You’ve been there. I’ve been there. We’ve all been there. Staring at a React component that started as “just a simple form” and now looks like it needs its own architectural diagram. Your component has a dozen useState hooks at the top, all fighting each other like tabs in your browser:

const [isLoading, setIsLoading] = useState(false);
const [isStreaming, setIsStreaming] = useState(false);
const [isComplete, setIsComplete] = useState(false);
const [error, setError] = useState<Error | null>(null);
const [data, setData] = useState<string[] | null>(null);
const [isRetrying, setIsRetrying] = useState(false);
const [showConfetti, setShowConfetti] = useState(false); // Because why not?

You tell yourself it’s fine. “It’s manageable,” you whisper, as you write another useEffect to synchronize three of these boolean flags, creating a side effect that will haunt you in your dreams. This isn’t just messy; it’s a breeding ground for what I call “impossible states.”

What happens when isLoading and isComplete are both true? Does your UI show a loading spinner and the final result? What if error has a value but isLoading is also true? Is it loading or is it an error? Your UI becomes a quantum superposition of confusion, and you’re the unfortunate physicist tasked with observing it into a non-buggy state.

The AI Broke My Booleans

Now, let’s throw a modern AI-powered feature into the mix. Your simple data fetch is now a generative UI that streams responses from a large language model. The state diagram in your head, which used to be a linear flow, now has branching paths, error recovery, cancellation states, and retry logic.

You’re not just fetching data anymore. You are:

1. 🙂 Idle: Waiting for a user prompt.

2. 🔄 Submitting: Sending the prompt to the backend.

3. 🤔 Waiting for stream: The server has acknowledged the request, but the first token hasn’t arrived.

4. 𝌗 Streaming: Receiving the response, token by token.

5. ✅ Success: The stream has finished, and the full response is displayed.

6. ⛔️ Error: Something, somewhere, went horribly wrong. Maybe the AI is having an existential crisis. Maybe you forgot an API key.

How do you model this with useState?

const [isSubmitting, setIsSubmitting] = useState(false);
const [isStreaming, setIsStreaming] = useState(false);
const [isError, setIsError] = useState(false);
// … and so on, and so on.

You are now manually choreographing a ballet of booleans. setIsSubmitting(true), then setIsSubmitting(false) and setIsStreaming(true) in the same function. It’s fragile. It’s imperative. It’s a bug waiting to happen. You’ve created a system where isSubmitting and isStreaming can be trueat the same time, an impossible state that your UI has no idea how to render.

This is the moment of truth for many engineers: the realization that managing state with a loose collection of booleans can be fundamentally tricky for complex systems.

State Machines 101: What They Are and Why They Matter

A finite state machine (FSM) is a computational model that can be in exactly one of a finite number of states at any given time. Think of it like a flowchart with strict rules:

1. States: A defined set of conditions your system can be in (e.g., idle, loading, success, error)

2. Transitions: Allowed movements between states, triggered by events (e.g., SUBMIT event moves from idleto loading)

3. Guards: Conditions that must be met for a transition to occur (e.g., can’t submit if the form is invalid)

4. Actions: Side effects that occur during transitions (e.g., clear error message when retrying)

The key insight: you can only be in one state at a time. No overlapping, no ambiguity.

Imagine a traffic light:

const trafficLightMachine = {
initial: 'red',
states: {
red: { on: { TIMER: 'green' } },
yellow: { on: { TIMER: 'red' } },
green: { on: { TIMER: 'yellow' } }
  }
};

The light can’t be both red and green. The only valid transitions are defined. This is the power of explicit state modeling.

State Machines + The Actor Model: A Powerful Blend

You might have heard of the Actor Model — a concurrent computation model where “actors” are independent entities that communicate via messages. Here’s the key insight: these aren’t competing concepts, they’re complementary.

Modern state machine frameworks like XState blend both paradigms:

- State Machines: Model deterministic state transitions within each actor (what states can it be in, what transitions are valid)

- Actor Model: Enable multiple state machines to run independently and communicate via events (spawning children, sending messages between machines)

Think of it this way: each actor is a state machine. The state machine defines its internal behavior, while the actor model defines how multiple machines coordinate.

Building State Machines in React

Now that we understand what state machines are, let’s see how to actually implement them in React. We’ll start simple and progressively handle more complexity.

AI Streaming UX

Before we dive into solutions, let’s be clear about what we’re building. What seems like a simple “type a prompt, get a response” feature actually has **challenges** that break simple state management:

idle → connecting → streaming → complete
↓ ↓
error ← ─ ─ ─ ┘
↓
(retry logic)

Your useReducermachine, which happily handled loadingand success, suddenly feels inadequate. Here are the actual production scenarios you need to handle:

Solution 1: Start with useReducer

For simple to moderate state machine needs, React’s built-in `useReducer` is your friend. Instead of scattered `useState` calls, you define a single state type and explicit transitions:

type State =
| { status: 'idle' }
| { status: 'loading' }
| { status: 'success', data: string[] }
| { status: 'error', message: string };
type Action =
| { type: 'SUBMIT' }
| { type: 'SUCCESS', data: string[] }
| { type: 'ERROR', message: string }
| { type: 'RESET' };
function reducer(state: State, action: Action): State {
switch (state.status) {
case 'idle':
return action.type === 'SUBMIT' ? { status: 'loading' } : state;
case 'loading':
if (action.type === 'SUCCESS') return { status: 'success', data: action.data };
if (action.type === 'ERROR') return { status: 'error', message: action.message };
return state;
case 'success':
case 'error':
return action.type === 'RESET' ? { status: 'idle' } : state;
}
}

This is already a massive improvement:

- Impossible states are structurally impossible

- State transitions are explicit and centralized

- TypeScript ensures you handle all cases

- The reducer documents your component’s behavior

When to use useReducer:

- Simple async flows (fetch → loading → success/error)

- Form wizards with sequential steps

- UI state that doesn’t need timers or complex side effects

But what about our complex AI streaming scenario?

Where `useReducer` Falls Short

Our AI streaming component has requirements that `useReducer` struggles with:

- Race condition prevention: Can’t start new stream while another is active (need to cancel first)

- Guards: Can’t retry if max attempts reached; can’t start if prompt is empty

- Entry/exit actions: Clear streamed text when starting, preserve it on error for display

- Complex conditional logic: Different error recovery paths (network vs auth vs stream)

You could model this with useReducer + useEffect, but now you’re manually managing:

- Abort controllers and cleanup in useEffect for race conditions

- Manual state checking before every transition

- Synchronization between reducer state and async operations

- Your reducer becomes a tangled mess of nested `switch` statements with validation

You’re back to the same complexity problem, just in a different form.

Solution #2: XState for Complex Orchestration

This is the point where you’re not just managing state; you’re orchestrating a complex user experience. And for that, you need a tool designed for orchestration.

Enter XState. XState isn’t just a state management library; it’s a framework for building statecharts. A statechart extends finite state machines with hierarchical (nested) states, parallel states, guarded transitions, and entry/exit actions — critical features for modeling complex async flows like AI streaming.

Key capabilities relevant to our real-world AI challenges:

- Guards (Conditional Transitions): Prevent race conditions (block `START` while streaming, block `RETRY` if max attempts reached).

- Deterministic Transitions: Given current state + event, next state is always predictable (no `isLoading && isRetrying` ambiguity).

- Entry/Exit Actions: Reset or preserve context cleanly when states change (clear text on start, preserve on error).

- Unified Context: One source of truth for tokens, retries, errors, partial responses — atomic updates, no desync.

- Explicit Error Paths: Network failures, stream errors, and cancellations have defined recovery paths.

Here’s a sample state machine for AI streaming:

import { createMachine, assign } from 'xstate';
const aiStreamingMachine = createMachine({
id: 'aiStreaming',
initial: 'idle',
context: {
  prompt: '',
  streamedText: '',
  tokens: 0,
  retryCount: 0,
  error: null,
  abortController: null
 },
states: {
  idle: {
    on: {
      START: {
        target: 'connecting',
        guard: ({ context }) => context.prompt.trim().length > 0,
        actions: assign({
          abortController: () => new AbortController(),
          error: null
        })
      },
     UPDATE_PROMPT: {
      actions: assign({ prompt: ({ event }) => event.value })
    }
  }
},
connecting: {
  invoke: {
    src: 'connectToStream',
    onDone: { target: 'streaming' },
    onError: {
    target: 'error',
    actions: assign({
      error: ({ event }) => event.error.message
    })
  }
},
  on: {
    CANCEL: {
      target: 'cancelled',
      actions: 'abortConnection' // cleanup action
  }
 }
},
streaming: {
    entry: assign({ streamedText: '', tokens: 0 }), // clear on entry
    on: {
      CHUNK_RECEIVED: {
        // PARALLEL CONTEXT UPDATES: atomic updates, no desync
        actions: assign({
        streamedText: ({ context, event }) => context.streamedText + event.chunk,
        tokens: ({ context }) => context.tokens + 1
    })
  },
  STREAM_COMPLETE: { target: 'complete' },
  STREAM_ERROR: {
  target: 'error',
  // PRESERVE PARTIAL DATA: keep what we received
  actions: assign({
  error: ({ event }) => event.message
    // Note: streamedText NOT cleared, preserved for user
  })
},
 CANCEL: {
    target: 'cancelled',
    actions: 'abortStream'
  }
} 
},
complete: {
  on: {
    REGENERATE: {
    target: 'connecting',
    actions: assign({ retryCount: 0 }) // reset retry count
  },
RESET: {
  target: 'idle',
  actions: assign({
  prompt: '',
  streamedText: '',
  tokens: 0,
  retryCount: 0
....

Compare this to managing the same logic with scattered `useState` hooks and conditional `useEffect` cleanup. The machine is living documentation of your app’s behavior.

The Right Tool for the Job

- useState: Simple, independent values (form inputs, toggles)

- useReducer: Local state machines with moderate complexity

- Redux/Zustand/Jotai: Shared application state across components

- XState: Complex state orchestration with guards, timers, hierarchies, and visual debugging

For our AI streaming component, the complexity is in **state transitions and orchestration**, not in sharing data globally.

Try the Interactive Demo

Performance Considerations

Before you refactor your entire codebase to use state machines, let’s address the elephant in the room: **performance**.

The Overhead Question

State machines add a layer of abstraction, and abstraction has cost. Let’s be honest about the tradeoffs:

When Performance Actually Matters

For most applications, XState’s overhead is imperceptible. But there are edge cases:

❌ Avoid XState for:

- High-frequency updates (60fps animations, mouse tracking, canvas interactions)

- Hundreds of simultaneously active machines in a single view

- Ultra-lightweight components (simple toggles, accordions)

✅ XState Shines for:

- Complex async orchestration (like our AI streaming example)

- Infrequent but critical state transitions (checkout flows, multi-step forms)

- Features where correctness > raw speed (payment processing, data submission)

Optimization Strategies

If you adopt XState and need to squeeze out performance:

The Real Cost: Maintenance vs. Performance

Here’s the uncomfortable truth: premature optimization kills more projects than slow code.

A state machine that’s 2ms slower but prevents 10 hours of debugging impossible states is a massive win. The question isn’t “Is XState slower than useState?” It’s “What’s the cost of shipping buggy state management?”

The Bottom Line

Don’t choose state machines for performance. Choose them for correctness, maintainability, and developer experience. If you later discover a performance bottleneck, you can optimize selectively or replace specific hot paths.

But start with the right abstraction. Premature optimization is the root of all evil, but so is choosing useState for a problem that demands state machine semantics.

Originally published at https://fatmaali.dev on October 19, 2025.

Originally published at https://www.fatmaali.dev

Github Copilot doesn’t need to be smarter. It needs to be plugged in. That’s what free, open-source MCP servers do: they turn Copilot from “autocomplete with swagger” into a teammate that can actually fetch live docs, query your DB, automate your browser, and remember your coding quirks.

I’ve tested dozens of MCP servers. Most just duplicate what Copilot already does well (reading your open files, writing code). The 5 servers below are the best so far:

• Chroma MCP: Gives Copilot long-term memory of your decisions and architecture
• Context7 MCP: Keeps external documentation always fresh and accurate
• Task Master MCP: Structured AI-driven task planning & execution from PRDs
• browser-use MCP: Automates repetitive browser tasks and data extraction
• Knowledge Graph Memory: Structured graph of entities, relations & lessons-persistent contextual + error memory beyond raw vectors

Let’s see how each one transforms your daily workflow.

1. Chroma

The Problem: Six months ago, your team made a crucial architectural decision about payment retry logic. The reasoning was solid, discussed in depth, but now it’s scattered across PR comments, Slack threads, and meeting notes.

How Chroma MCP Solves It: Chroma creates a searchable semantic database of your team’s knowledge-README files, Architecture Decision Records (ADRs), design docs, and important discussions.

Install it from https://github.com/chroma-core/chroma-mcp

2. Context7

The Problem: You bookmark the Prisma documentation, but three weeks later, the API you’re using has been deprecated. The blog post you saved about React best practices is from 2022. Your knowledge goes stale fast.

How Context7 MCP Solves It: Context7 automatically fetches the latest version of any external documentation and keeps it synchronized. No more outdated information.

Install it from https://github.com/upstash/context7

3. TaskMaster

The Problem: Turning a Product Requirements Doc (PRD) into well‑scoped, dependency‑aware implementation tasks takes time. Work drifts from the original intent, priorities become unclear, and developers constantly ask, “What’s the next actionable thing?”

How Task Master Solves It: It ingests a PRD and generates a structured tasks.json (tasks, subtasks, dependencies, priority, test strategy). Through MCP you can ask natural language questions ("What's next?", "Expand task 5", "Move 5.2 under 7") and it maps them to deterministic CLI operations-keeping planning, execution, and refactoring of tasks inside your editor.

Install from https://github.com/eyaltoledano/claude-task-master

4. browser-use

The Problem: You need to check analytics dashboards, export data, or perform the same multi-step browser workflow every week. It’s tedious and error-prone.

How browser-use MCP Solves It: It can automate browser interactions-logging into systems, navigating to dashboards, extracting data, and returning structured results.

Install from https://docs.browser-use.com/customize/mcp-server

5. Knowledge Graph

The Problem: Your team keeps re‑diagnosing the same build, dependency, and environment errors; architectural intent erodes; and Copilot can’t surface prior reasoning because it isn’t stored in a structured, queryable form.

How It Solves It: A local knowledge graph that stores:

- Entities (people, services, domains, features)

- Relations (“service_A depends_on service_B”, “job_X publishes_to queue_Y”)

- Observations (atomic facts: “Rollout uses canary: true”)

- Lessons (error pattern + verified resolution + success rate tracking)

Unlike plain embedding memory, lessons capture error fingerprints (type, message, context) plus evolving remediation steps and verification commands. Success/failure feedback updates the lesson’s effectiveness score.

Real Scenario: A recurring CI failure: `Playwright timeout in headless mode on macOS runners`. Instead of re‑searching, you ask: *”Find similar errors and show the highest success‑rate fix.”* The server returns a prior lesson with the exact environment nuance and validated mitigation steps.

Install from: https://github.com/modelcontextprotocol/servers/tree/main/src/memory

Now, think of the flow:

1. Discover broadly with Chroma (“What did we discuss about circuit breakers?”).
2. Distill durable facts (decision, constraint, error fingerprint) → promote into graph as an entity/observation/lesson.
3. During a future incident: query graph first for precise, curated remediation; fall back to vector search if no lesson exists.

The magic happens when you use multiple MCP servers together. Here are some powerful combinations:

1. Start with local servers (Chroma, DB Introspection, Knowledge Graph Memory)
2. Add network servers gradually for specific use cases
3. Review what data each server accesses before installation
4. Test with non-sensitive data first

The best part? All of these tools are free and open-source. No subscriptions, no API limits, no vendor lock-in.

Your 5-minute action plan:

1. Install Chroma MCP and embed your README
2. Set up Context7 for your main framework
3. Ask Copilot one question that uses both
4. Experience the difference

Once you see the power of having instant access to both your team’s knowledge and fresh external documentation, you’ll wonder how you ever worked without it.

What will you automate first? Share your MCP setup on Twitter and share below — I love seeing creative combinations!

The hooks and context API changed the way we build React apps forever. The new API would help in building components in a more functional and reusable way. In this blog, we are going to see how we could use hooks and context to share the user object across the app.

Note: Before we start

As the original docs say:

Context is primarily used when some data needs to be accessible by many components at different nesting levels. If you only want to avoid passing some props through many levels, component composition is often a simpler solution than context.

Additionally, I’m going to assume that you have an already running React App, so we’re going to work with some high-level methods to shorten the blog.

Why use Context for this use-case, and are there alternatives?

In some apps, we find the need to have access to the user object in many deeply nested components. Previously, to achieve sharing an object across many parts of the app, global state management libraries like redux were used. Even worse, prop drilling was used to pass down the props to child components.

Many opt for the inbuilt Context API for its simplicity, and the fact that it comes with React so you don’t need to install a third-party library.

Defining the context

Let’s consider the example below

We are declaring the context in the UserContext variable, then making it accessible throughout our app with UserContext.Provider . Pretty straightforward right?

However, this solution is limiting because apps are usually more complicated than this. Often we have routing in place, and we’d want some routes protected and some opened.

Extending the implementation

First, let’s move the context into its own file for better readability.

Here, we’ve created a wrapper to handle our user context logic. We’ve also renamed it to AuthContext . The AuthContext now provides the current user, as well as the setter for the current user. Awesome, right?

Okay, let’s define a simple custom hook to consume our context in the child components.

That’s it! So let’s consume it.

First off, we would have to provide the context in our application root as shown below. You can see that we’re getting our user object from sessionStorage We’re assuming that she logged in earlier and we persisted her details in session storage. This is done to prevent us from losing her user info once she refreshes the page, forcing her to log in again.

There are many ways to do this with pros and cons in each. For now, let’s use this as just an example.

Consuming the context

Assuming you’re using react-router, we could define our routes like so:

We’ve defined different routes for our app, some are protected while some are not. For the protected routes, you could see that we have defined a wrapper component called <ProtectedRoute> . The implementation <ProtectedRoute> is shown below:

So far so good! Lastly, let’s see how we can implement and consume the authentication context within our login component.

Finally, let’s assume that we need to implement the ability to logout within a nested child component inside our app. How would that implementation look like?

And that’s it! You can extend the context and add more values as you need, but this pretty much enables you to share the current user state across the app. Feel free to drop in questions/suggestions below! Thanks for reading!

Spring 2020. The birds are singing. The earth is replenishing its beauty with lush greenery. Everything is perfect, well, except for a global pandemic that’s confined human beings in their homes.

What to do?

During the coronavirus pandemic of 2020, most of us find ourselves at home suddenly with a lot of time on our hands, food in the fridge, Netflix, and in 90% of the cases, no bae (thank you social distancing). Unfortunately and fortunately for (dark-themed, hooded and introverted) software engineers like myself, life is the same. We just can’t get the best coffee anymore from that little coffee shop in the neighborhood, but that’s fine.

I’ve spent literally the last seven weeks like a zombie: work, Netflix, eat and, repeat. Frankly, this is the life I thought I always wanted, not having to leave the house. But it gets boring too. So I decided to write a diary about some of my experiences as a software engineer. Some crazy, some fun and some not so pretty. I mean, why not?

In the next few posts, I’ll be talking about the (rough) journey, the panic attacks, the craziest bugs ever (literally), the changes and evolution that engineering brought to me as a person. My goal is to make software engineers, or anyone starting out a career, understand how hard and rewarding it is to be passionate, and most of all, yourself. I’m not the type that writes a lot, but I’m hoping this can be the beginning of my best selling book. Haha, just kidding. Hope you, whoever you are, enjoy this :-)

We have all been there. With tens or hundreds of cron jobs running, and you’re (or your boss is) pulling your hair out trying to figure out why a critical job didn’t run last night. You’re puzzled and confused. Didn’t it run? Did it fail? Why could it have failed?

At AMPATH, our reliance on cron jobs to schedule ETL jobs was becoming increasingly impossible. Our ETL processes involve first denormalizing and flattening data , then use the denormalized data to build calculated tables. The calculated tables then produce real-time reports and decision support systems for the Ministry of Health and clinicians respectively. In the beginning, using cron jobs was a simple and effective way to execute the jobs.

The cron workflow was as follows:

1. Run the denormalizing jobs every second minute of the hour
2. Run the calculated tables jobs every fifth minute, with the hope that the previous jobs will take only three minutes to complete.

Over time as data and the demand for more reports increased, it became difficult to estimate the time a particular job would take and we more than often run into lock wait timeouts and deadlocks. Cron had reached its limit. We needed a way to:

1. Handle complex relationships between jobs.
2. Handle all the jobs centrally with a well defined user interface.
3. Error reporting and alerting.
4. Viewing and analyzing job run times.
5. Security (protecting credentials of databases).

Enter Apache Airflow

Upon searching for solutions to improve or replace our ETL workflow, I stumbled upon an open source tool, Apache Airflow. Airflow’s Direct Acryclic Graph (DAG) concept offered a way to build and schedule complex and dynamic data pipelines in an extremely simple, testable and scalable way. And I freaked out! This is exactly what we have been looking for.

We quickly setup airflow using docker-compose like this. And started transforming some of the main ETL jobs we had into python code for Airflow. Here is the github repo for the scripts.

Airflow Concepts

We are going to cover some of the basic concepts to get you started with Airflow. For a more detailed documentation, head over to this official docs site here

DAGs and Operators

In Airflow, all workflows are considered to be DAG’s. You can think of DAG’s as a set of tasks with some sort of relationship. This is how a DAG looks like in Airflow Graph View:

A DAG usually has a schedule, a start time and a unique ID. The tasks inside DAG’s are made of operators. Operators are define what actually run for a particular task. Examples of Operators in Airflow include:

BashOperator: To execute shell commands/scripts

PythonOperator: To execute Python code.

You can define a simple DAG that simply prints out ‘Hello World!’ every 10 minutes like this:

from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator

default_args = {
'owner': 'airflow',
'email': ['fali@ampath.or.ke'],
'email_on_failure': True,
'email_on_retry': True,
'email_on_success': False,
'start_date': datetime(2019, 5, 31)
}

dag = DAG(
    dag_id='hello_world_dag',
    default_args=default_args,
    schedule_interval= '*/10 * * * *')

task1 = BashOperator(
           task_id="echo_hello_world"
           bash_command='echo Hello World!',
           dag=dag)

Extending Operators

What’s even more exciting is you can extend/create custom operators if none of the inbuilt operators match your needs. Here’s an example of how I extended the inbuilt MySQL Operator to return the results of a statement after execution:

from airflow.hooks.mysql_hook import MySqlHook

class CustomMySqlOperator(MySqlOperator):
    def execute(self, context):
      self.log.info('Executing: %s', self.sql)
      hook = MySqlHook(
             mysql_conn_id=self.mysql_conn_id,
             schema=self.database)
      return hook.get_records(self.sql, parameters=self.parameters);

....

# Use the custom operator in one of your tasks

task = CustomMySqlOperator(
          task_id='custom_mysql_task',
          sql='select * from person;',
          mysql_conn_id='mysql_conn',
          database='etl',
          dag=dag)

Defining relationships between tasks

Often, you find that some of your tasks need to execute one after another. In airflow you can define relationships like this:

task1 = BashOperator(
           task_id="echo_hello_world"
           bash_command='echo I will execute first!',
           dag=dag)

task2 = BashOperator(
           task_id="echo_hello_world"
           bash_command='echo I will execute second!',
           dag=dag)

# There are multiple ways you can define this relationship
# Using the bitwise operators or methods
# Option 1
task1 >> task2

# Option 2
task2 << task1

# Option 3
task1.set_downstream(task2)

# Option 4
task2.set_upstream(task1)

Branching

Sometimes you want to execute tasks depending on certain conditions. That is very possible using airflow, here’s an example:

...

run_task = BashOperator(
           task_id="echo_hello_world"
           bash_command='echo Hello World!',
           dag=dag)

sleep = BashOperator(
           task_id="echo_hello_world"
           bash_command='sleep 1m',
           dag=dag)

### function decides which task to run depending on the time by returning the task_id

def decide_path():
    now = datetime.now(timezone('Africa/Nairobi'))
    if now.hour >= 19:
       return "run_task"
    else:
       return "sleep"

branch = BranchPythonOperator(
                 task_id='check_time',
                 python_callable=decide_path,
                 dag=dag)

...

Connections

One of the best things about Airflow is Security. Airflow handles and encrypts your credentials for you so you will never have to do that by yourself or include it in your code.

Once you add and save the credentials under connections, you can access it simply by adding the connection id in your operator when defining your task.

Conclusion

Airflow is a game changer when it comes to scheduling and monitoring workflows. In this article, I have only covered the basic concepts that helped me get started, but there’s a lot more that I haven’t covered. Feel free to dig deeper and reach out.

In the first part of this series, we went ahead and set up heartbeat to monitor two HTTP servers and send the data to Elasticsearch. We also configured Kibana to automatically create the uptime dashboards for us. For this second part, we are going to include Slack notifications to notify us in case the server goes down.

Before we begin, we have to get the webhook url for your Slack channel. Follow the below instructions to get the webhook url.

1. Log in to slack.com as a team administrator.
2. Go to https://my.slack.com/services/new/incoming-webhook/.
3. Select a default channel for the integration.

4. Click Add Incoming Webhook Integration.

After the incoming webhook has been generated, copy the url and add it with the following configuration in the elasticsearch.yml file in the directory elasticsearch/config/.

xpack.notification.slack:
  account:
    monitoring:
      url: <WEB-HOOK-URL>
      message_defaults:
        from: x-pack
        icon: http://example.com/images/watcher-icon.jpg
        attachment:
          fallback: "X-Pack Notification"
          color: "#36a64f"
          title: "X-Pack Notification"
          title_link: "https://www.elastic.co/guide/en/x-pack/current/index.html"
          text: "One of your watches generated this notification."
          mrkdwn_in: "pretext, text"

After you add this to your elasticsearch.yml, you have to restart all the containers for the changes to take effect. Run:

$docker-compose restart $(docker-compose ps -q)

Once the containers have restarted, head over to Kibana, under Management menu, under Elasticsearch, click the Watcher option and create a new threshold alert

Setting Up Watcher

1. Using the UI (Kibana 6.6 and above)

With the Kibana 6.6, you can now configure the notifications schedule and message with the UI. After entering the name, indices and time field, you will be provided by a set of options like below:

These options make Kibana aware of the conditions requires to fire the actions such as Slack notifications. For this case, the conditions are:

WHEN count() GROUPED OVER top 5 http.response.status_code IS ABOVE 299 FOR THE LAST 5 minutes

HTTP codes 200 to 299 represents a successful request, so we want to fire an action when we get responses with status codes greater than 299. Next we will move to actions.

For this case, you will pick the Slack action on the drop down. (If Slack wasn’t configured properly in elasticsearch.yml, it will be grayed out). The recipient can be a user or a channel (remember to add a # symbol for channels), and the message can be customized to include the error message and code as below:

Encountered  {{ctx.payload.hits.total}} errors in the last 5 minutes: \n {{ctx.payload.hits.hits}} *Error Message*: _{{_source.error.message}}_, \n *HTTP Status*: _{{_source.http.response.status}}_ \n {{ctx.payload.hits.hits}}

2. Using JSON (Kibana 6.5 and below)

The below JSON can be used to create an advanced watch or if you’re using Kibana version 6.5 or below where there is no option to use the UI.

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "heartbeat*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "query_string": {
                    "query": "monitor.status:down"
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-5m"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "send_trigger": {
      "throttle_period_in_millis": 3600000,
      "slack": {
        "message": {
          "from": "heartbeat",
          "to": [
            "#heartbeat-monitoring"
          ],
          "text": "Heartbeat Monitoring",
          "icon": "https://raw.githubusercontent.com/elastic/elasticsearch-net/master/build/nuget-icon.png",
          "attachments": [
            {
              "color": "danger",
              "title": "Server Down",
              "text": "Encountered  {{ctx.payload.hits.total}} errors in the last 5 minutes: \n {{#ctx.payload.hits.hits}} *Error Message*: _{{_source.error.message}}_, \n *HTTP Status*: _{{_source.http.response.status}}_ \n {{/ctx.payload.hits.hits}}"
            }
          ]
        }
      }
    }
  }
}

Further explanations for each of the properties in the above JSON object can be found here.

And that’s it!

You can save your watch at this point and try to simulate it using the simulate menu in Kibana. You should receive a message like this from slackbot:

Feel free to reach out and ask any questions regarding the above set up!

Uptime is an important aspect in any online hosted service as it directly impacts a business. You can have the best software product in the world, but if you experience a lot of downtime, then it’s as good as nothing. In the real world, 100% uptime is cannot be guaranteed. Thus the need to determine the moment downtime happens, and act fast.

Overtime, we also need to know more details, for example, what causes downtime, at what time it occurs etc. And that’s why at AMPATH, we chose to do it with Kibana and Heartbeat.

From the official websites:

Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack, so you can do anything from learning why you’re getting paged at 2:00 a.m. to understanding the impact rain might have on your quarterly numbers.

Heartbeat monitors services for their availability with active probing. Given a list of URLs, Heartbeat asks the simple question: Are you alive? Heartbeat ships this information and response time to the rest of the Elastic Stack for further analysis.

Both of these tools are part of the Elastic stack. You can find a quick intro to the Elastic stack here.

In the example below, we are going to set up Heartbeat to ping a HTTP server to determine if it’s up or down as well as configure Kibana to automatically create visualizations for the data received from heartbeat.

Step 1: Set up the ELK Stack

(Make sure you have docker and docker-compose installed before you begin!)

Fork and clone this repo. The docker-compose.yml file has the essential docker images for the ELK stack, we are going to have to add one more service for Heartbeat.

2: Configure Heartbeat

As you can see, we have added the heartbeat service to the compose file. But before we start the containers, we have to create the volumes we named in order to map the config file to the containers. So let’s create a folder named heartbeat. Inside the heartbeat folder:

(i) Create a Dockerfile with the following content:

(ii) Create a folder called config and inside the config folder, create a file called heartbeat.yml

The heartbeat.yml file is a config file that we’ll use to configure Heartbeat monitors. To better understand what it does, here’s how it should be written for our particular case:

Brief Explanation

This file is responsible for configuring multiple heartbeat monitors. In our case, we have configured a single monitor of the http type. We have also instructed it to hit https://fatmali@github.io and https://google.com every 10 seconds with a GET request, if it gets a response with a 200 status code, then this means a server is up else it means it’s having some trouble. Finally, we have configured an Elasticsearch output and the Kibana host for automatic dashboard creation.

To learn more of about the http monitor and the different types of monitor heartbeat supports, head over to the official docs!

And that’s it, you’re ready to go!

Run:

$ sudo docker-compose up -d

And you should be up and running, once the containers have started, head over to Kibana and you should see the new Heartbeat dashboard automatically created. It should look like this:

The next part will cover how to utilize Elasticsearch Watchers to setup downtime notifications with Slack! Happy Monitoring 😉

When I finally graduated from college and landed my dream job, I was surprised to find myself as the only woman in my office. The first few days sucked. Despite my male colleagues being extremely helpful and welcoming, I felt lonely.

Sure, it can be motivating to be the first/only lady in the office but that doesn’t come without its own set of challenges. Many times, I was mistaken for a front-end developer. Nothing wrong about being a front-end developer, but the idea front-end is feminine and the back-end is too “tough” is what annoyed me.

The biggest challenge most women face in male-dominated careers is the lack of female mentors. Without mentors, we often try to push ourselves to perform better than our male colleagues. And that’s okay, but we shouldn’t forget how and why we got there in the first place.

Here are my top 6 tips to survive and succeed in a male-dominated career environment:

Be Yourself

Always. Don’t apologize for who you are. Understand your strengths and your weaknesses and act on them. Be confident. Remember you are worthy and just as good as everybody else, don’t be afraid to be you.

Find a mentor and mentees

It’s necessary to find someone “who’s been there” to be your mentor. Someone that understands the challenges you face. Here’s an article that can guide you on finding a female mentor. Most importantly, find young girls you can mentor, we need more women in these careers.

Don’t hold back

Don’t be afraid to air out your opinions. Learn to communicate your ideas clearly. When an opportunity comes to lead or demo your skills, don’t, for heaven’s sake, hold back. Take your place on the freaking table.

Become literally indispensable

Work hard, not to compete but to become irreplaceable. Put in more hours and extra effort to get things done. Always get into the details when working on something. And get things done!

Don’t be the office messenger (please!)

Don’t ever allow yourself to be the office coffee-getter or messenger. Well unless you don’t mind being an unpaid assistant/secretary. Do what you’re paid to do.

Do not tolerate disrespect/mistreatment

Finally, if you’re in an environment that you don’t feel respected, you are at liberty to leave and find a place that will offer you better treatment. Never tolerate any mistreatment just because you’re female, stand up for yourself or walk out.

Personally, I try to follow the above tips every day to succeed in my career. I’d love to hear back from anyone who might have any comments related to this topic.

Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.

Unlike creating a virtual OS like VMs, docker allows applications to use the same Linux kernel as the host machine. This makes docker more light-weight and perfomant than VM’s. It is an open source tool made to make the lives of developers and system admins a lot easier by vanquishing the “works on my machine” problem.

Getting Started

For this tutorial, we are going to learn how to set up and basic commands I use almost everyday when working with containers.

Step 1: Install Docker

In this step, you could either install docker, or head over to Docker playground (make sure you have a docker hub account) and get started playing with docker right away!

Step 2: Basic Commands

You can confirm if docker is installed by running the command below and it should return the name of the current version of docker you’re running:

$ docker --version

Docker version 18.03.1-ce, build 9ee9f40

The next step is to fetch a mysql image from docker hub. A container is launched by running an image. An image is an executable package that includes everything needed to run an application — the code, a runtime, libraries, environment variables, and configuration files.

To pull the mysql image, we will run the command below:

$ docker pull mysql

Using default tag: latest
latest: Pulling from library/mysql
07a152489297: Pull complete
Digest: sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
Status: Downloaded newer image for mysql:latest

To see a list of the images now available in your system:

$ docker image ls

REPOSITORY          TAG                 IMAGE ID            SIZE
mysql               latest              8d99edb9fd40        445MB

To run the mysql image, we will run:

$ docker run --name mysql_container -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql

The above command runs the docker image of mysql. The --name tag names the container mysql_container, if we had not provided a name, docker would have given a random name to our container. The -e tag sets the environment variable which in this case is the mysql root password to my-secret-pw.

The -d tag tells docker to run this container in the background as a daemon, thus closing the shell won’t stop the container, (the opposite of this is -it), and finally we call the image name which is mysql.

To see the list of all running containers:

$ docker ps

To log into a running container:

$ sudo docker exec -ti mysql_container bash

once inside the container, you can execute commands such as ls ,cd, mkdir because it is a based on the unix kernel. Once in the container, you could try connecting to your mysql instance using mysql -uroot -pmy-secret-pw. To exit the container just hit exit .

To stop a running container simply hit:

$ docker stop mysql_container

When you run sudo docker ps, you won’t see mysql_container in the list of running containers, to see a list of all running and stopped containers:

$ sudo docker ps -a

The -a tag means all containers. To restart the container

$ sudo docker start mysql_container

To remove a container means you cannot restart it again. You simply do that with this command (note that you cannot remove a running container):

$ sudo docker rm mysql_container

And that concludes the basic docker commands that will get you started with docker. To see all the options available for a particular command, for example start, you can run:

$ docker start --help

If you have reached this far, congrats! :-D In the next tutorial, we are going to look at docker volumes!