Photo by Lisanto 李奕良 on Unsplash

Auto-scaling AWS ECS Fargate Task using SQS and Lambda

Understanding the use case is crucial in this article because there may be several different designs and solutions. It relies on your current infrastructure and business logic whether you can implement one of them.

So without any introduction, let’s understand the use case right off.

Use case

1. Suppose we want to run compute on a set of events that do not need to be processed as and when they arrive.
2. Compute can take more than 15 minutes on average per event.
3. Triggering of the events in this article depends on your use case and your current architecture.

Based on this knowledge, a simple solution could be to trigger a lambda for each event.

However, this has some flaws.

First, this is near real-time computing and can take more than 15 minutes (max lambda timeout reached) on average per event which does not satisfy our use case. Second, lambdas are by default throttled if the number of events generated is significantly more than 1000. Third, cost, which can be extremely high depending on the amount of…