Resolved -
We identified that a change made to the Google Calendar plugin resulted in very long render times for that particular plugin. This delay quickly exhausted our pool of Firefox workers and prevented other plugins from getting a chance to generate. The offending change has been rolled back.
Mar 30, 16:50 EDT
Monitoring -
We have deployed a fix and worker health has improved. We are continuing to monitor the situation.
Mar 30, 16:05 EDT
Update -
On our background worker nodes, we have found that Firefox times out during plugin rendering. We are continuing to investigate the cause.
Mar 30, 14:09 EDT
Investigating -
Some users are reporting that some of their plugins have not updated since about 14:00 UTC. We are investigating.
Mar 30, 13:16 EDT