GitLab says it lost 6 hours of database data, including issues, merge requests, users, comments and snippets for GitLab.com on Tuesday after a spam attack. "Spammers where hammering the database by creating snippets, making it unstable. The problems escalated causing a lockup on writes on the database, this caused some downtime," according to a post on GitLab.com.
Representatives of the code-hosting site said they are currently restoring data from a 6-hours old backup of their database. "Losing production data is unacceptable," the company’s post read.
Once it realized it was under attack, GitLab blocked the spammers based on an IP address, and "removed a user from using a repository as a form of CDN. This resulted in 47,000 IPs signing in using the same account and causing a high database load."
The company said its enterprise customers — which run the GitLab platform on their own servers — were not affected. GitLab enterprise customers include IBM, Macy’s, ING, NASA and VMWare, according to TechCrunch.
Data loss post-backup restoration would affect less than 1% of their user base, according to the company. Git/wiki repositories and self-hosted installations were not affected, the company said.
GitLab said they would post a "5 why's of why this happened and a list of measures we will implement" within a few days. Though it’s hard to say yet what the company could have done differently, CIOs want to look out for the promised document to see what lessons can be gleaned from GitHub’s data base issues. Yet another reminder that technology does occasionally fail. Of course, if you’re one of the 1% that lost data, that’s little comfort.