Skip to main content
Image

r/UnstructuredIO


Webhooks for Unstructured Pipelines: in the wild…what we're seeing.
Webhooks for Unstructured Pipelines: in the wild…what we're seeing.

This week’s discussion has been about webhooks (see previous posts 1, 2, and 3). Read more about it here.

The most common pattern we’ve seen is routing failed-job events to whatever paging tool a team already uses, so on-call engineers stop finding out about failures secondhand. Right behind that is completion-triggered handoff: a job finishes, and that event kicks off the next stage of a pipeline that lives outside our system entirely, instead of someone manually checking and then manually starting the next step.

We've also seen people use it for plain visibility rather than action. For instance, they’re dropping job events into a Slack channel so a whole team has shared awareness of what's running and what just broke, without anyone needing to check a dashboard at all.

What are you seeing on your end? Does this match how you'd expect to use it, or are you doing something with it we haven't anticipated?


Webhooks in Unstructured Pipelines: our quick take. How about yours?
Webhooks in Unstructured Pipelines: our quick take. How about yours?

We’ve been talking lately about webhooks in Unstructured Pipelines (posts 1, 2). You can read more about it here.

Honestly, the core mechanism is simple by design. And that's mostly a good thing. Five job states, JSON payloads, and your receiver does the rest. Nothing exotic to learn, which means it's been quick for people to adopt.

One thing to note: deleting webhooks is permanent. There's no undo, no soft-delete, no recovery window. If a webhook is wired into something important, double-check before you remove it. 

Oh, and another: we know workflow-scoped webhooks currently exist in the API but not yet in the UI. If you want a no-code approach for workflow-scoped webhooks right now, sorry; you have to go through the API.

What's your take? Does that match your experience setting one up, or did you hit something we haven't mentioned?


What webhooks in Unstructured Pipelines unlocks for you…your thoughts?
What webhooks in Unstructured Pipelines unlocks for you…your thoughts?

We just kicked off a discussion a few days ago about our webhooks. Let’s talk more about why it matters.

The obvious one is alerting: you can route a failed-job event straight to your on-call rotation instead of someone noticing a stale dashboard. But the more interesting unlock is treating job completion as a trigger for actual work, not just a status you check.

A completed event can kick off a downstream sync, fire a CI/CD step, or hand off to whatever automated process is waiting on fresh data. A failed event can do more than alert. It can roll back a deploy, retry with different parameters, or flag a record for human review. Because the payload goes to a receiver URL, you're not locked into any one integration; it's worked through AWS Lambda, Azure Functions, Google Cloud Run, Zapier, Slack, Svix, and plain webhook testing tools, so whatever's already in your stack is probably already supported.

Oh, and something from a security perspective. You can sign your webhook with a secret so the receiving side can verify the request actually came from us and wasn't spoofed. Use this technique if your endpoint triggers anything sensitive.

Here are the docs, if you want to learn more.

What do you think? Any interesting projects you'd reach for this with? Something you'd actually wire a webhook into rather than just checking a dashboard for?