38

Our quarterly publication of the Stack Exchange Network data (the “data dumps”) is currently delayed. While the process of copying and uploading data was scheduled to begin on Sunday, December 3rd, we’ve run into some issues with not all of the dumps uploading. The team is currently working on this and we’ll release the data dump as soon as we can. We’ll update this post once the data has been uploaded for everyone’s awareness.

Hopefully, we’ll be able to share the data dump as early as today but we wanted to give you a heads-up to keep you fully updated. In September we shared that we’re working to improve this pipeline and process so that it doesn’t need this level of proactive monitoring, which is still the case. Our team is working toward a more predictable timeline for the data dumps to occur each quarter. Again, we’ll update here as we have more information.

Update given by Rosie in comment posted on 2023-12-06 19:15:11Z:

Just wanted to share an ETA. We're hoping the data dump will be completed by the early next week. I'll provide an update once it's completed or if it is further delayed. Thanks for your patience.

Improve this question
8
  • 4
    @AaronBertrand what exactly is failing every time? Commented Dec 5, 2023 at 17:59
  • 3
    I'm going on a limp and say moving all this data at the same time from multiple sources to one big thing is really not something the infrastructure likes. Commented Dec 5, 2023 at 18:03
  • 13
    Thanks for the update! Commented Dec 5, 2023 at 18:13
  • 4
    @ShadowWizardIsSadAndAngry Well, last time, we were getting throttled and rate limited. This time, I thought it was the same, but actually they had an issue (scroll down). Commented Dec 5, 2023 at 22:04
  • 1
    @AaronBertrand thanks. Does it mean the throttle and rate limit issues have been resolved? Commented Dec 6, 2023 at 9:24
  • 3
    @ShadowWizardIsSadAndAngry No, it's still not fast (whether that is intentional / systemic throttling or not, I don't know). We're just no longer completely blocked and, since we only have two files left, I doubt rate limiting will come into play. They just happen to be the two biggest files. Commented Dec 6, 2023 at 13:54
  • 1
    @AaronBertrand fair, what about the plan to have them read from you, and thus making it all more simple and quick? Commented Dec 6, 2023 at 20:29
  • 2
    @ShadowWizardIsSadAndAngry Yep, haven't forgotten about that, let's just say the delay is not on our side. Commented Dec 6, 2023 at 20:56

1 Answer 1

Reset to default
18

I just received confirmation that the Data Dump for this quarter has now successfully been released. Thank you again for your patience.

Improve this answer
0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.