A Basic Solution to Shutting Down AWS Infrastructure on a Schedule
Even in the most AWS native solutions utilizing serverless technologies and cloud-managed “billing by volume” services, there is usually some infrastructure that is still explicitly provisioned and billed hourly such as containerized services, bastion hosts, relational databases on RDS. Combined with test environments that fully mirror a production environment, this can lead to pointlessly running up the AWS bill by having servers on standby even on nights and weekends, when (hopefully) not much development is happening.
I’m in a hurry and want to shut down my infrastructure for the weekend, give me the goods!
Wait — why do I need to configure it, instead of using a simple tagging based approach?
Tagging is all fine and dandy if you have simple standalone resources such as EC2 instances that can be flicked down and up again. It won’t work as nicely with auto scaling services, since you need to retain the knowledge of how much to ramp it up again, which requires state. This could be stored in SSM, DynamoDB or a myriad of other places, but as the purpose here is to provide a very simple basic solution, the configuration is deployed with the scheduler.
Okay, but why were these specific services included?
A good grip on your infrastructure starts when first provisioning your first environment. This means using infrastructure as code, or infra-code for short, to provision your resources — hopefully a globally adopted standard way of working at this point. Using infra-code means you could as easily tear down your environments as they are built up, but doing this constantly is often not feasible due to the raw amount of time required (f.ex. CloudFront distributions taking up to 40 minutes to provision) and the data loss incurred in tearing down many resources. It’s easier to just ramp down the necessary resources temporarily while keeping up the overall infrastructure.
In practice, I’ve found the most obvious candidates for easy rampdown are:
- ECS service capacity (containers)
- Autoscaling group capacity (virtual servers)
- RDS clusters (relational databases)
If using AWS Fargate to run your containerized services, it’s sufficient to bring down the desired capacity of your ECS services to zero to avoid any costs, and this can be done with a single API call. If you use self-provisioned ECS host servers instead of Fargate, you’d also need to stop these host instances.
As for EC2 instances, hopefully you have set them up to auto scaling groups, and AWS recommends setting even single bastion hosts up this way. There are many benefits, mainly fault tolerance and ease of management, and also more involved tricks such as the possibility of using spot instances bidding the on-demand price for an almost guaranteed discount (though don’t do this for critical services, as AWS can intermittently raise the spot bid price over the on-demand price to lessen the load during server maintenance). Again, ramping up or down an autoscaling group is a single API call.
For RDS, either instances or whole clusters can be stopped. However, if at all possible it’s usually preferable to use AWS Aurora, which only supports stopping whole clusters, or go the whole nine yards to use AWS Aurora Serverless, so you are only billed on the capacity you actually use.
For scheduling, the simplest approach is to rely on Cloudwatch Events which support a simple cron-like syntax which is already familiar to most developers. The example solution uses Serverless framework to configure these events.
Other services could be added, or do it in a fancier way — go ahead and implement your own solution, hopefully getting a head start from here!
