Red faces at Gitlab as sysadmin expunges 300GB of data

Oops

A systems administrator at open source software company Gitlab, which provides a repository management tool as an alternative to Github, has accidentally deleted about 300GB of live product data.

Gitlab is currently down while the company tries frantically to restore the data, with customers kept abreast of progress live on this page.

Gitlab formally launched in 2014 with backing from Khosla Ventures and Y-Combinator as a commercial alternative to services like Github. Before 2014, Gitlab's tools were available as open source downloads.

Github has itself had major outages in the past, once with five days offline due to DDoS attacks.

However, Gitlab would also appear to be struggling to restore the lost data from backup - assuming that it did actually do regular backups. The company claims that it has to take the site down in order to restore from its backups.

The expunged database covers issues and merge requests, but repositories and wikis are safe - otherwise the affected code would be considerably bigger.

It seemed, at first, that things were just sluggish. But then it was spotted that the data was being chewed up like Pacman on an ectoplasmic binge.

At time of writing, GitLab says it's 68 per cent done with restoring the database. For the most part, users have (surprisingly) been very understanding, with comments like "Mankind is not perfect" and "thanks for your work and transparency".

However, as mistakes go, this is on a par with an obliging taxi driver being interview on BBC News about Apple Music in a case of mistaken identity.

The look of horror on Guy Goma's face is probably the same one the GitLab sysadmin had when he realised what had happened.

Forthcoming Computing events:

Computing's popular DevOps Summit returns in March in a programme that also includes the DevOps Excellence Awards. Attendance at the Summit is free to qualifying IT Leaders and senior IT pros, so register early to secure your place.