Bitly hackers stole user credentials from offsite database backup

Bitly has shed a little more light on the serious security breach it suffered 2 weeks.
As you may recall, the URL-shortening service announced last week that it believed the account credentials of Bitly users could have fallen into the hands of hackers, but it fell short of answering how it determined customer privacy had been breached, how securely passwords had been stored, or – indeed – what had actually gone wrong.
Now some of those questions are being answered.
In a follow-up post entitled “More detail”, Bitly explains that it believes the hackers did *not* manage to access its production network or servers, but instead accessed the customer database from an offsite backup.

Over the course of the next few hours, the Security Team determined with a high degree of confidence that there had been no external connections to our production user database or any unauthorized access of our production network or servers. They observed that we had an unusually high amount of traffic originating from our offsite database backup storage that was not initiated by Bitly. At this point, it was clear that the best path forward was to assume the user database was compromised and immediately initiate our response plan, which included steps to protect our users’ connected Facebook and Twitter accounts.

And how did the hackers manage to access that offsite backup? They broke into an employee’s account at an unnamed hosted source code repository where they stole the login credentials for the backup of Bitly’s database.

We audited the security history for our hosted source code repository that contains the credentials for access to the offsite database backup storage and discovered an unauthorized access on an employee’s account. We immediately enabled two-factor authentication for all Bitly accounts on the source code repository and began the process of securing the system against any additional vulnerabilities.

What’s worrying about this is that – for a while at least – not only did the hackers have access to a backup of Bitly’s customer database, but they also could have compromised the company’s source code.
Bitly says it is sending an email to “all users from the domain bitlysupport.com outlining the steps to secure your account”. The fact that they have named the domain they are planning to send the warning email from underlines their concern that the hackers might attempt their own malicious campaigns, targeting customers who have had their accounts exposed through the hack.
Ironically, Bitly’s announcement of the domain name they intend to use may not actually make it trickier for any attackers to exploit the situation – as it will be child’s play for them to forge email headers and pretend the messages are coming from bitlysupport.com.
My advice? Be very careful about *any* messages that you receive which claim to come from Bitly, and be wary of clicking on any links in the emails. Much better to visit the Bitly website directly, and access your account that way.
According to Bitly, the passwords stored in the exposed database were salted and hashed. Unfortunately, users who have not changed their passwords in the last few months may be at greater risk of having had their passwords cracked as Bitly strengthened the way it stored passwords in January:

If you registered, logged in or changed your password after January 8th, 2014, your password was converted to be hashed with BCrypt and HMAC using a unique salt. Before that, it was salted MD5.

No wonder then that the firm is recommending that users change their passwords as a precaution.
In case you’re worried about your own account, here is what Bitly says you need to do:

Following are step-by-step instructions to reset your API key and OAuth token:
1) Log in to your account and click on ‘Your Settings,’ then the ‘Advanced’ tab.
2) At the bottom of the ‘Advanced’ tab, select ‘Reset’ next to ‘Legacy API key.’
3) Copy down your new API key and change it in all applications. These can include social publishers, share buttons and mobile apps.
4) Go to the ‘Profile’ tab and reset your password.
5) Disconnect and reconnect any applications that use Bitly. You can check which accounts are connected under the ‘Connected Accounts’ tab in ‘Your Settings.’

Many Bitly users are believed to have connected their accounts to their social media presences on the likes of Facebook and Twitter, but users will not be able to publish via Bitly to those sites until their profiles have been reconnected following the advice above.
Two factor authentication – are you using it?
It’s good to hear that Bitly has now enabled two factor authentication for all of its employees using the source code repository, but an organisation serious about protecting its crown jewels like its source code, would have done that long ago.
I’ve explained the perils with passwords in the past, including the problems with users re-using the same password in multiple places, choosing easy to guess passwords, falling victim to spyware which hoovers up passwords as they are typed on infected computers, or having their login credentials phished from them via convincing emails.
Two-factor authentication (2FA) helps reduce these risks, requiring users to enter a unique one-time-password alongside their regular credentials.

Everytime you login, a new one-time-password is required.
Even if your regular password is guessed, cracked or stolen by hackers, it won’t be any use to the bad guys because they won’t know what your one-time-password is.
Furthermore, if something like a mobile phone app is generating your one-time password for you then it’s extremely unlikely it will be in the clutches of the hackers trying to break into your account.
So, I strongly recommend that whenever an online service or website offers you the option of hardening your account using two-factor authentication you should turn it on.
Furthermore, if you are an organisation running an online service or providing mechanisms for your staff to access company information remotely, it also makes sense for you to consider offering two-factor authentication to reduce the risks.
Two-factor authentication isn’t a magical solution which will stop all online criminal activity, but it certainly makes life harder for the hackers who want to break into your accounts.
Oh, and in case you were wondering, Bitly says it is “accelerating” its efforts to provide two-factor authentication for its customers account as well. That means, if users’ passwords fall into the wrong hands in future – they will be an awful lot harder for the bad guys to exploit.