The Path to Microservices CI/CD Nirvana

By Sam Banks on March 18, 2020

By Sam Banks

March 18, 2020

Why we chose Buildkite + Ansible + Cloudformation to deploy our Infrastructure

At Stackchat, over the course of 24 months, we have built a global chat platform, with a focus on security, scalability and auditability.

We were able to build our (mostly-serverless) platform and scale it out with a single-person devops team, avoiding many of the common pitfalls that I've encountered in my 10+ year career in the field. This has been possible thanks to a new breed of excellent tooling that favours simplicity over features, and ensuring that we strictly adhere to good engineering principals.

In this post I will give a description of those engineering principals and will lay out the strengths and weaknesses of the tools that we chose.

I will also give some example code and screenshots at the end, with deep-dives on specific tasks and pipelines to follow in subsequent posts.

The engineering principals we live by

  • DRY

    "Don't Repeat Yourself" or the DRY principle is stated as, "Every piece of knowledge or logic must have a single, unambiguous representation within a system."

    Not only does DRY make code more efficient and readable, but it also encourages coders to observe other best practice conventions. For example, DRY motivates us to create reusable modules or templates which can be used and improved by other members of the team. The practice also encourages us to look for the most elegant solutions to any problems we encounter.

  • KISS

    “Keep it simple, stupid” is thought to have been coined by the late Kelly Johnson, who was the lead engineer at the Lockheed Skunk Works a place responsible for the S-71 Blackbird spy plane amongst many other notable achievements). Kelly explained the idea to others with a simple story. He told the designers at Lockheed that whatever they made had to be something that could be repaired by a man in a field with some basic mechanic’s training and simple tools.

    To us, this translates directly to infrastructure design. All tooling should be easily maintained and modified by junior engineers with miminal assistance.

    This can also be seen as a counterpoint to dogmatically adhering to DRY. Before creating a new module or script, We need to ask ourselves if it provides enough advantages to justify the increased complexity.

  • YAGNI

    Yagni stands for "You Aren't Gonna Need It". The Yagni principle prevents over-engineering by limiting the development of speculative future software features because, more likely than not, "you aren't gonna need it".

    Designing for future hypothetical use cases (espeically related to performance optimization) is a common pitfall in devops which leads to time-wasting and bloated code bases that are hard to maintain. Remember, features are easy to add, but hard to remove so only build what you need right now.

  • Principle Of Least Astonishment

    This principle means that your code should be intuitive and obvious, and not surprise another engineer when reviewing your code. If you hear your team mate muttering "what the f*!$%?" under their breath after you send them a PR, you are most likely in breach of this principal.

    For variables, modules, roles, etc. your naming should always reflect the component's purpose, striking a balance between wordy and ambiguous, and the logic you create should be easy to follow.

  • Over-Engineering

    This one is especially close to our hearts and encapsulates most of the previous principals.

    The pernicious effects of over-engineering can cripple an engineering team as a business scales, especially if it happens quickly.

    Keeping your infrastructure code small and modular and not reinventing functionality already present in your chosen tooling (RTM before writing a new module), will allow your infrastructure to scale exponentially without requiring your team to.

    Some common over-engineering crimes and their outcomes:

    • Abstraction

      The temptation to wrap wrappers in wrapped wrappers is often too strong to resist for engineers in our industry. Abstraction is essential and exists at every level of an application, however it should be used sparingly to avoid the compounding costs involved.

      It may save you some future typing to use the latest so-hot-right-now-on-hacker-news tool in order to avoid the manual creation of a new module. However when it mysteriously breaks your integration pipeline six months down the track after an operating system update, and all traces of the tool and its author have dissappeared off the internet, you may regreat having added so many layers of indirection and abstraction to your stack.

    • State

      Avoid state when you can design a system without it. Like abstraction, state is everywhere and is a core building block of applications, but like abstraction, too much will lead to increased complexity and will add to your accumulative technical debt and maintenance burden.

Buildkite, Ansible, and Cloudformation

With these principals in mind, I set out to design the Stackchat devops empire. In this section I'll briefly describe our chosen tooling and will then deep dive into the strengths and weaknesses of each one, providing comparisons to other popular tooling.

As with all tooling assessments, the strengths and weaknesses I've outlined are highly subjective and may not ring true with you, but hopefully there is value in sharing our decision making process and subsequent outcomes.

  • Ansible

    Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced devops tasks such as continuous deployments or zero downtime rolling updates.

    Ansible’s goals are foremost those of simplicity and maximum ease of use. This approach has resulted in it overtaking Chef to become the most popular configuration management tool in the world.

  • Buildkite

    Buildkite is a CI and build automation tool that combines the power of your own build servers with the convenience of a managed, centralized web UI. Buildkite allows us to automate complicated delivery pipelines and it gives us crazy levels of flexibility around custom checkout logic and dynamically building pipelines as part of a build.

  • Cloudformation

    AWS CloudFormation allows us to use programming languages or simple config files to model and automatically provision all the AWS resources needed for Stackchat across all our supported regions and accounts. This gives us a single source of truth for our infrastructure that can actually live in a project's code base next to all the other associated files.

Ansible

Ansible Logo

The Swiss Army Knife of Continuous Integration, Continuous Deployment and Configuration Management.

Strengths

  • Simple to learn

    Ansible playbooks are imperative, rather than declarative. Most automation and infrastructure management tools work by declaring a state of configuration. With Ansible you define a series of steps that are executed in order. This makes Ansible much easier to learn for engineers coming from scripting backgrounds.

    Writing Ansible is similar in many ways to scripting, supporting popular imperative programming constructs such as conditionals, loops, and dynamic includes. Ansible modules are written in YAML, which is extremely popular, being one of the easiest configuration languages to use.

  • Declarative Modules

    While automation code written in Ansible is written in simple imperative playbooks and roles, the modules provided by Ansible work in a declarative fashion. This gives you the best of both worlds.

  • Agentless

    Chef, Puppet, Saltstack and so on have a master and a client. They need to be installed and configured on both the master and the client. Ansible requires installation only on the master server. It communicates with the other nodes through SSH.

    While this isn't a concern for us, since our infrastructure is serverless with Ansible running standalone (no master) on our build agents, it is worth mentioning.

  • Idempotent

    When you write a playbook for configuring your nodes, Ansible first checks if the state defined by each step is different from the current state of the nodes and only makes changes if required. Therefore if a playbook is executed multiple times, it will still result in the same system state.

  • Batteries Included

    Ansible comes out-of-the-box ready to use, with everything you need to manage the infrastructure, networks, operating systems and services that you are already using via the 3000+ included modules.

    These modules make it incredibly easy to perform complex tasks across virtually any Public Cloud and Private Infrastructure running any Operating System and Software Stack.

  • Multi Purpose

    Unlike orchestration tools like Terraform, Ansible supports orchestration and configuration management, as well as much more.

    While one tool might do specific things better than another, applying the 80/20 rule and keeping your stack lean, at the expense of non-core functionality, pays huge dividends as your business scales.

Weaknesses

  • Speed

    Ansible is slower than many other tools, possibly due to it's serial nature and it's "push" model. This gets worse at scale. This isn't much of a problem for us due to our masterless setup and the awesomeness of Buildkite, which we will get into soon.

Example Ansible playbook

1---
2- hosts: all
3  gather_facts: true
4  roles:
5    - add_groups
6    - aws_sts_assume_role
7    - get_facts
8    - npm_token
9    - node_test
10    - role: node_build
11      when: deploy == true
12    - role: artifact_upload
13      when: deploy == true
14    - role: aws_cloudformation_deploy
15      template: "{{bk_root}}/cloudformation_template.yml.j2"
16      when: deploy == true
17

Buildkite

Buildkite LogoBuildkite Logo

In a crowded market Buildkite distinguishes itself by being simple, fast and intelligently designed.

Strengths

  • Simple

    Like Ansible, Buildkite pipelines are configured with YAML. A common configuration language is a big plus for us.

    Compared to venerable stallwarts of CI/CD such as Jenkins and Bamboo, Buildkite has a fraction of the features and plugins. For us this is actually a huge plus, as we like to do everything with Ansible and use the CI solely to bootstrap it. This lack of bloat makes it a joy to use in this fashion.

  • Intelligently Designed

    While Buildkite may be light on features, it isn't missing anything we need. This is no mean feat and is testament to a company with great engineering that listens to it's customers.

    They may not have a plugin to integrate with HP Operations Orchestration, or a Skype Notifier, or 1714 other plugins like Jenkins. They do however provide, via github, an excellent Elastic CI Stack for AWS codebase that allows you to easily implement an autoscaling fleet of build agents running on AWS Spot Instances, that can scale all the way down to zero when not in use. This gives us infinite horizontal scale for large deploys, overnight maintenance, and integration and load testing, at a very cheap cost, with very little engineering investment on our end.

  • Hosted

    As a Serverless company, hosted is essential for us and Buildkite's speed and uptime has been flawless.

  • Open Source

    Buildkite is also open source, which aligns with our company values and future goals. They host all of their code publicly on their github, even their own website!

  • Support

    Buildkite support desk is top notch.

    There are no levels to work your way through to get to someone who can help and no attempts to defer blame. You hit a knowledgeable engineer straight away and more often then not get the solution first time. If there is no immediate solutions they go out of their way to help with workarounds. If it's a bug in their system they tell you straight up and they fix it.

    They also provide a weekly community summary via email containing announcements and features, as well as help requests from the forum. It has been surprisingly helpful to see other companies issues and how they solved them.

  • Company Culture

    While not a technical requirement, it's nice to work with companies who's values align with yours. Their values from their website:

    • Transparency
    • Empathy
    • Quality
    • Collaboration
    • Diversity
    • Sustainable Growth
    • Independence

Weaknesses

  • Lack of Features and Plugins

    While this was a plus for us, people who prefer batteries included solutions should look elsewhere.

Example Buildkite pipeline

1---
2steps:
3  - label: ":ansible: Run stack_node.yml playbook"
4    command:
5      cd environment-automation/ansible/ &&
6      ansible-playbook
7        -e product=common
8        stack_node.yml
9
Buildkite ScreenshotBuildkite Screenshot

Cloudformation

Cloudformation LogoCloudformation Logo

Amazon's infrastructure-as-code solution provides a feature-rich automation and deployment platform.

Strengths

  • Language Support

    Cloudformation Vanilla allows you to write templates in YAML or JSON. But if you would prefer to use a full fat programming language of your choice the recently released Cloud Development Kit allows you to define your application using TypeScript, Python, Java, and .NET.

  • Vendor Support

    You can orchestrate infrastructure in AWS using external tooling such as Terraform, Salt, Puppet and even Ansible via modules. While this does work and is an approach preferred by many, it violates several of our engineering principals and can lead to significant problems:

    • Complexity

      The level of abstraction that make tools like Terraform more attractive to many newcomers inevitably leads to sprawling codebases that are hard to maintain and even harder to uplift.

      Many are also stateful, such as Terraform which uses a state file (why???), and require complex state modification when importing resources or resolving conflicts.

    • Features

      Newly released Amazon products are immediately available in Cloudformation. When using external tooling you have to wait for someone to write (and hopefully test) a new module to support the product.

    • Stability

      When you deploy a change via cloudformation that requires a new resource, Amazon creates a new resource and waits until it is available and healthy before seamlessly replacing and deleting the old resources. If there are any issues a rollback is easy, as the old resources are kept until the cleanup stage of the deployment and it also results in reliably zero-downtime deploys. While some preference tools like Terraform for the speed of their in-place modifications, the ability to avoid downtime and roll back automatically is more important to us.

  • Extensibility

    The recently released AWS Cloudformation Registry means you can now define third party resources in Cloudformation. For us being able to define our Datadog alerts in the Cloudformation template for the microservice that they are monitoring is a huge win.

  • Dependency Management

    AWS CloudFormation automatically manages dependencies between your resources during stack management actions. You do not need to worry about specifying the order in which resource are created, updated, or deleted.CloudFormation determines the correct sequence of actions to use for each resource when performing stack operations.

    This, along with the imperative nature of Ansible, means we get to completely avoid dependency management, which is a huge maintenance overhead in many other tools such as puppet.

Weaknesses

  • Slow

    Due to the deployment style of Cloudformation, which preferences zero-downtime and rollbacks over speed, it is slower than tools that do in-place updates.

  • (Mostly) Single Vendor

    While they have recently introduced the Cloudformation Registry, this isn't for deploying to other clouds like Azure and GCP. For multi-cloud look elsewhere.

Example Cloudformation template

1#jinja2: trim_blocks: True, lstrip_blocks: True
2---
3AWSTemplateFormatVersion: '2010-09-09'
4Transform: AWS::Serverless-2016-10-31
5
6Resources:
7  IoUserApi:
8    Type: AWS::Serverless::Api
9    Properties:
10      StageName: Live
11      MethodSettings:
12        - HttpMethod: '*'
13          ResourcePath: /*
14          LoggingLevel: INFO
15          DataTraceEnabled: true
16          MetricsEnabled: true
17      DefinitionBody:
18        swagger: 2.0
19        info:
20          title: {{stack_name}}
21        x-amazon-apigateway-policy:
22          Version: 2012-10-17
23          Statement:
24            - Effect: Allow
25              # Note from the console: If the Principal is set to AWS, then authorization will fail for all resources not secured with AWS_IAM auth, including unsecured resources.
26              Principal: '*'
27              Action:
28                - execute-api:Invoke
29              Resource: execute-api:/*
30        securityDefinitions:
31          sigv4:
32            type: apiKey
33            name: Authorization
34            in: header
35            x-amazon-apigateway-authtype: awsSigv4
36          tenant-id-authorizer:
37            type: apiKey
38            name: Authorization
39            in: header
40            x-amazon-apigateway-authtype: custom
41            x-amazon-apigateway-authorizer:
42              authorizerResultTtlInSeconds: 300
43              authorizerUri:
44                !Sub arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/{{AuthorizerArn}}/invocations
45              type: token
46        paths:
47          /admin/apps/{appId}/users:
48            options:
49              x-amazon-apigateway-integration:
50                httpMethod: POST
51                type: aws_proxy
52                uri: !Sub arn:${AWS::Partition}:apigateway:{{cf_region}}:lambda:path/2015-03-31/functions/${IoUserAdminApiFn.Arn}/invocations
53            x-amazon-apigateway-any-method:
54              security:
55                - tenant-id-authorizer: []
56              x-amazon-apigateway-integration:
57                httpMethod: POST
58                type: aws_proxy
59                uri: !Sub arn:${AWS::Partition}:apigateway:{{cf_region}}:lambda:path/2015-03-31/functions/${IoUserAdminApiFn.Arn}/invocations
60          /admin/apps/{appId}/users/{proxy+}:
61            options:
62              x-amazon-apigateway-integration:
63                httpMethod: POST
64                type: aws_proxy
65                uri: !Sub arn:${AWS::Partition}:apigateway:{{cf_region}}:lambda:path/2015-03-31/functions/${IoUserAdminApiFn.Arn}/invocations
66            x-amazon-apigateway-any-method:
67              security:
68                - tenant-id-authorizer: []
69              x-amazon-apigateway-integration:
70                httpMethod: POST
71                type: aws_proxy
72                uri: !Sub arn:${AWS::Partition}:apigateway:{{cf_region}}:lambda:path/2015-03-31/functions/${IoUserAdminApiFn.Arn}/invocations
73          /sdk/apps/{appId}/users:
74            options:
75              parameters:
76                - name: appId
77                  in: path
78                  required: true
79                  type: string
80              responses: {}
81              x-amazon-apigateway-integration:
82                uri: https://{{IoClusterServiceApiUrl}}/users/sdk/apps/{appId}/users
83                responses:
84                  default:
85                    statusCode: 200
86                requestParameters:
87                  integration.request.path.appId: method.request.path.appId
88                passthroughBehavior: when_no_match
89                httpMethod: ANY
90                type: http_proxy
91            x-amazon-apigateway-any-method:
92              produces:
93                - application/json
94              parameters:
95                - name: appId
96                  in: path
97                  required: true
98                  type: string
99              responses: {}
100              security:
101                - sigv4: []
102              x-amazon-apigateway-integration:
103                uri: https://{{IoClusterServiceApiUrl}}/users/sdk/apps/{appId}/users
104                responses:
105                  default:
106                    statusCode: 200
107                requestParameters:
108                  integration.request.path.appId: method.request.path.appId
109                  integration.request.header.x-cognito-identity-id: context.identity.cognitoIdentityId
110                  integration.request.header.x-cognito-identity-pool-id: context.identity.cognitoIdentityPoolId
111                passthroughBehavior: when_no_match
112                httpMethod: ANY
113                type: http_proxy
114          /sdk/apps/{appId}/users/{proxy+}:
115            options:
116              parameters:
117                - name: appId
118                  in: path
119                  required: true
120                  type: string
121                - name: proxy
122                  in: path
123                  required: true
124                  type: string
125              responses: {}
126              x-amazon-apigateway-integration:
127                uri: https://{{IoClusterServiceApiUrl}}/users/sdk/apps/{appId}/users/{proxy}
128                responses:
129                  default:
130                    statusCode: 200
131                requestParameters:
132                  integration.request.path.appId: method.request.path.appId
133                  integration.request.path.proxy: method.request.path.proxy
134                passthroughBehavior: when_no_match
135                httpMethod: ANY
136                type: http_proxy
137            x-amazon-apigateway-any-method:
138              produces:
139                - application/json
140              parameters:
141                - name: appId
142                  in: path
143                  required: true
144                  type: string
145                - name: proxy
146                  in: path
147                  required: true
148                  type: string
149              responses: {}
150              security:
151                - sigv4: []
152              x-amazon-apigateway-integration:
153                uri: https://{{IoClusterServiceApiUrl}}/users/sdk/apps/{appId}/users/{proxy}
154                responses:
155                  default:
156                    statusCode: 200
157                requestParameters:
158                  integration.request.path.appId: method.request.path.appId
159                  integration.request.path.proxy: method.request.path.proxy
160                  integration.request.header.x-cognito-identity-id: context.identity.cognitoIdentityId
161                  integration.request.header.x-cognito-identity-pool-id: context.identity.cognitoIdentityPoolId
162                passthroughBehavior: when_no_match
163                httpMethod: ANY
164                type: http_proxy
165
166  IoUserApiBasePathMapping:
167    Type: AWS::ApiGateway::BasePathMapping
168    DependsOn: IoUserApiLiveStage
169    Properties:
170      BasePath: users
171      DomainName: {{IoSharedApiGatewayUrl}}
172      RestApiId: !Ref IoUserApi
173      Stage: Live
174
175  IoUserAdminApiFnLambdaExecRole:
176    Type: AWS::IAM::Role
177    Properties:
178      AssumeRolePolicyDocument:
179        Version: 2012-10-17
180        Statement:
181          - Effect: Allow
182            Action:
183              - sts:AssumeRole
184            Principal:
185              Service:
186                - lambda.amazonaws.com
187      Path: /
188      Policies:
189        - PolicyName: AttachedPolicy
190          PolicyDocument:
191            Version: 2012-10-17
192            Statement:
193              - Effect: Allow
194                Action: dynamodb:*
195                Resource:
196                  - {{IoAppTableArn}}
197                  - {{IoAppTableArn}}/index/*
198              - Effect: Allow
199                Action:
200                  - ssm:GetParametersByPath
201                  - ssm:GetParameters
202                  - ssm:GetParameter
203                Resource: !Sub arn:${AWS::Partition}:ssm:{{cf_region}}:{{cf_account}}:parameter/{{env}}/*
204              - Effect: Allow
205                Action: ssm:DescribeParameters
206                Resource: '*'
207              - Effect: Allow
208                Resource: '*'
209                Action:
210                  - xray:PutTelemetryRecords
211                  - xray:PutTraceSegments
212      ManagedPolicyArns:
213        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
214
215  IoUserAdminApiFn:
216    Type: AWS::Serverless::Function
217    Properties:
218      CodeUri: {{code_uri}}
219      Role: !GetAtt IoUserAdminApiFnLambdaExecRole.Arn
220      Runtime: nodejs10.x
221      Handler: adminHandler/adminHandler.adminHandler
222      Timeout: 300
223      MemorySize: 1024
224      AutoPublishAlias: live
225
226  InvokeIoUserAdminApiPermission:
227    Type: AWS::Lambda::Permission
228    Properties:
229      Action: lambda:InvokeFunction
230      FunctionName: !Ref IoUserAdminApiFn
231      Principal: apigateway.amazonaws.com
232
233  IoUserSdkApiLogGroup:
234    Type: AWS::Logs::LogGroup
235    Properties:
236      RetentionInDays: 30
237
238  IoUserSdkApiRole:
239    Type: AWS::IAM::Role
240    Properties:
241      AssumeRolePolicyDocument:
242        Version: 2012-10-17
243        Statement:
244          - Effect: Allow
245            Action:
246              - sts:AssumeRole
247            Principal:
248              Service:
249                - ecs-tasks.amazonaws.com
250      Path: /
251      Policies:
252        - PolicyName: AttachedPolicy
253          PolicyDocument:
254            Version: 2012-10-17
255            Statement:
256              - Effect: Allow
257                Action: dynamodb:*
258                Resource:
259                  - {{IoAppTableArn}}
260                  - {{IoAppTableArn}}/index/*
261              - Effect: Allow
262                Action:
263                  # - xray:PutTraceSegments
264                  # - xray:PutTelemetryRecords
265                  - ecr:*
266                  - sts:AssumeRole
267                  - iam:GetRole
268                  - iam:PassRole
269                Resource: '*'
270              - Effect: Deny
271                Action:
272                  - logs:CreateLogStream
273                  - logs:PutLogEvents
274                Resource: '*'
275              - Effect: Allow
276                Action:
277                  - ssm:GetParameterHistory
278                  - ssm:GetParametersByPath
279                  - ssm:GetParameters
280                  - ssm:GetParameter
281                Resource: !Sub arn:${AWS::Partition}:ssm:{{cf_region}}:{{cf_account}}:parameter/{{env}}/*
282              - Effect: Allow
283                Action: ssm:DescribeParameters
284                Resource: '*'
285              - Effect: Allow
286                Action: iot:Publish
287                Resource:
288                  !Sub arn:${AWS::Partition}:iot:${AWS::Region}:${AWS::AccountId}:topic/live-chat/*
289      ManagedPolicyArns:
290        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
291        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole
292
293  IoUserSdkApiTaskDefinition:
294    Type: AWS::ECS::TaskDefinition
295    Properties:
296      Cpu: '{{ecs_cpu_multiplier*256}}'
297      Memory: '{{ecs_mem_multiplier*512}}'
298      NetworkMode: awsvpc
299      RequiresCompatibilities:
300        - {{ecs_launch_type}}
301      ExecutionRoleArn: !GetAtt IoUserSdkApiRole.Arn
302      TaskRoleArn: !GetAtt IoUserSdkApiRole.Arn
303      ContainerDefinitions:
304        - Name: IoUserSdkApi
305          # LogConfiguration:
306          #   LogDriver: awslogs
307          #   Options:
308          #     awslogs-group: !Ref IoUserSdkApiLogGroup
309          #     awslogs-region: !Ref AWS::Region
310          #     awslogs-stream-prefix: IoUserSdkApiService
311          Image:
312            !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.${AWS::URLSuffix}/{{IoUserApiEcrRepositoryId}}:{{container_name}}
313          PortMappings:
314            - ContainerPort: {{cluster_service_port}}
315{% if datadog|bool %}
316        - Name: datadog-agent
317          Image: datadog/agent:latest
318          Cpu: 10
319          Memory: 256
320          PortMappings:
321            - ContainerPort: 8126
322          Essential: true
323          Environment:
324            - Name: DD_API_KEY
325              Value: {{DataDogKey}}
326            - Name: ECS_FARGATE
327              Value: true
328            - Name: DD_PROCESS_AGENT_ENABLED
329              Value: true
330            - Name: DD_APM_ENABLED
331              Value: true
332
333  IoUserSdkApiLogExporter:
334    Type: AWS::Logs::SubscriptionFilter
335    Properties:
336      DestinationArn: {{DatadogLogFunctionArn|default(omit)}}
337      LogGroupName: !Ref IoUserSdkApiLogGroup
338      FilterPattern: ''
339{% endif %}
340
341  IoUserSdkApiService:
342    Type: AWS::ECS::Service
343    Properties:
344      Cluster: {{IoEcsClusterId}}
345      LaunchType: {{ecs_launch_type}}
346      DesiredCount: {{ecs_container_multiplier}}
347      DeploymentConfiguration:
348        MaximumPercent: 200
349        MinimumHealthyPercent: 100
350      NetworkConfiguration:
351        AwsvpcConfiguration:
352          SecurityGroups:
353            - {{IoContainerSecurityGroup}}
354          Subnets:
355            - {{PrivateSubnetA}}
356            - {{PrivateSubnetB}}
357 {% if PrivateSubnetC is defined %}
358            - {{PrivateSubnetC}}
359 {% endif %}
360      TaskDefinition: !Ref IoUserSdkApiTaskDefinition
361      LoadBalancers:
362        - ContainerName: IoUserSdkApi
363          ContainerPort: {{cluster_service_port}}
364          TargetGroupArn: !Ref IoUserSdkApiLoadBalancerTargetGroup
365
366  IoUserSdkApiLoadBalancerTargetGroup:
367    Type: AWS::ElasticLoadBalancingV2::TargetGroup
368    Properties:
369      HealthCheckIntervalSeconds: 6
370      HealthCheckPath: /healthcheck
371      HealthCheckProtocol: HTTP
372      HealthCheckPort: '{{cluster_service_port}}'
373      HealthCheckTimeoutSeconds: 5
374      HealthyThresholdCount: 2
375      TargetType: ip
376      TargetGroupAttributes:
377        - Key: deregistration_delay.timeout_seconds
378          Value: '30'
379      Port: {{cluster_service_port}}
380      Protocol: HTTP
381      UnhealthyThresholdCount: 2
382      VpcId: {{VPC}}
383
384  IoUserSdkApiLoadBalancerListenerRule:
385    Type: AWS::ElasticLoadBalancingV2::ListenerRule
386    Properties:
387      Actions:
388        - TargetGroupArn: !Ref IoUserSdkApiLoadBalancerTargetGroup
389          Type: forward
390      Conditions:
391        - Field: path-pattern
392          Values:
393            - /users
394            - /users/*
395      ListenerArn: {{IoSharedSdkApiLoadBalancerListenerArn}}
396      Priority: 1
397
398{% if 'prd' not in acc %}
399  AutoScalingTarget:
400    Type: AWS::ApplicationAutoScaling::ScalableTarget
401    Properties:
402      MinCapacity: {{ecs_container_multiplier}}
403      MaxCapacity: {{ecs_container_multiplier*2}}
404      ResourceId: !Sub service/{{IoEcsClusterId}}/${IoUserSdkApiService.Name}
405      ScalableDimension: ecs:service:DesiredCount
406      ServiceNamespace: ecs
407      RoleARN: !GetAtt IoUserSdkApiRole.Arn
408      ScheduledActions:
409        - ScalableTargetAction:
410            MinCapacity: 0
411            MaxCapacity: 0
412          Schedule: {{ecs_scale_in_schedule|default(omit)}}
413          ScheduledActionName: ScaleIn
414        - ScalableTargetAction:
415            MinCapacity: {{ecs_container_multiplier}}
416            MaxCapacity: {{ecs_container_multiplier*2}}
417          Schedule: {{ecs_scale_out_schedule|default(omit)}}
418          ScheduledActionName: ScaleOut
419{% endif %}
420

Conclusion

TODO