All Systems Operational

Uptime over the past 90 days. View historical uptime.
Weld Sync Management UI Operational
90 days ago
100.0 % uptime
Today
Sync Engine Operational
90 days ago
99.93 % uptime
Today
Landing page Operational
90 days ago
100.0 % uptime
Today
Orchestration Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

Jul 21, 2026

No incidents reported today.

Jul 20, 2026

No incidents reported.

Jul 19, 2026

No incidents reported.

Jul 18, 2026
Elevated job failures due to rate limiting issues with AWS
Resolved - We have implemented multiple fixes to avoid the false positives in our stale monitor service.
We have implemented parallel processing and split our job into singular docker containers. To allow us to scale them automatically depending on queue build up.
And we have also guarded against build up which in turn terminates the stale monitor until the build up is resolved (should it happen again).
Jul 18, 13:01 CEST
Monitoring - We have deployed a fix to make sure our queues gets drained faster.
Everything seems to be back to normal but monitoring the situation now.
Jul 18, 12:34 CEST
Identified - We have been working on improving speeds of our data pipelines to avoid slow starts.
This at midnight caused A LOT of jobs to start at the same time overwhelming our system.
Our system then wasn't able to process all the finishing jobs fast enough and our stale job monitoring kicked in.

This in turn reset a whole bunch of jobs that were just waiting to get flagged as finished. And restarted them. This caused a vicious cycle of jobs restarting despite finishing and keeping our queues maxed out.

We are working on breaking these monitors up into singular service now so we can run them individually and in parallel without worrying about side-effects from running many of all monitors.
Jul 18, 11:22 CEST
Investigating - We are currently trying to resolve an issue where 429s from Amazon is causing jobs to be flagged as stale incorrectly
Jul 18, 08:35 CEST
Jul 17, 2026

No incidents reported.

Jul 16, 2026

No incidents reported.

Jul 15, 2026

No incidents reported.

Jul 14, 2026

No incidents reported.

Jul 13, 2026

No incidents reported.

Jul 12, 2026

No incidents reported.

Jul 11, 2026

No incidents reported.

Jul 10, 2026

No incidents reported.

Jul 9, 2026

No incidents reported.

Jul 8, 2026

No incidents reported.

Jul 7, 2026

No incidents reported.

← Incident History Powered by Atlassian Statuspage