How I got slapped with a $4,644.27 bill and why you should always secure your account.

Back in January, my AWS account was compromised and it was used to perform deep learning tasks with massive compute instances that amounted to a bill of $4644.27. Now you might be thinking, there is no way something like this would have gone unnoticed by AWS or me. For example, when the unauthorized user requested limit increase on one of the instances, I should have received an email with the conversation the user had with AWS (I did). In this post, I will detail the mistakes I made, what I should have done in the first place, and how to solve it should you find yourself in the same predicament.

It was a pretty good scare to start 2021.

The Mistakes

Mistakes were made.

• I did not set up MFA (Multi-Factor Authentication) for my account. This is one of the first defenses against unauthorized use by requiring another device or method before logging in to my account. Setting up MFA turned out to be super easy and it took only a few minutes for me. Here is the official guide for it, Using multi-factor authentication (MFA) in AWS .
• I performed everything on AWS with the root user. The root user is the account that you log in with your e-mail which can perform sensitive operations. It enables you to freely provision resources, create new roles, or request limit increases. This is a big NO because if the account becomes compromised, there will be serious implications (case in point).
• I disregarded the notifications that AWS had sent me. I ignored the e-mails as marketing messages before clicking into them. Being subscribed to multiple topics on AWS, I had begun to assume that most of the e-mails they send would be promotional messages (big mistake). Interestingly, this was how I discovered the misuse when I finally realized my mistake.

The Measures

Prevention makes things easier.

• Always set up MFA for your account. Like I said, this is one of the first defenses against unauthorized logins and it is also effective.
• Create IAM users and assign roles to perform specific tasks. This will restrict the access these users have to only specific resources. AWS recommends only using the root user once to create an administrative user that will be used to do the former (creating other users and assigning roles), AWS account root user .
• Pay attention to whatever you receive via e-mail. This is pretty obvious and it applies to everything, always be alert!

If you’re in the same position…

1. Don’t panic.
2. Take whatever evidence you have: screenshots, e-mail conversations /notifications, bills, and etc.
3. Contact customer service with said evidence(s) and explain your situation with clarity.
4. Follow their instructions (e.g. terminate instances, deleting access keys, removing users, etc.) and wait for their response.
5. Reexamine everything like what I did in this article and note down what you can do to prevent this from happening again.

Note: In my experience, a customer representative may call from a U.S. number so don’t be surprised if you get a call.

If you are wondering what happened to me in the end, the charge was cancelled and everything was resolved within 2 weeks. The reason it was so quick was because I had little money left in the card associated with my account. When you have insufficient funds, they cannot charge you but will attempt to do so continuously. And fortunately for that, I did not have to go through the refund process with the card issuer.