By using cloud platforms, we can take advantage of different resource configurations and compute capacities. However, deploying containerized applications on cloud platforms is proving to be quite challenging, especially for new users who have no expertise on how to use that platform. As each platform may provide specific APIs, orchestrating the deployment of a containerized application can become a hassle.
Docker Compose is a very popular tool used to manage containerized applications deployed on Docker hosts. Its popularity is maybe due to the simplicity on how to define an application and its components in a Compose file and the compact commands to manage its deployment.
Since cloud platforms for containers have emerged, being able to deploy a Compose application on them is a most-wanted feature by many developers that use Docker Compose for their local development.
In this blog post, we discuss how to use Docker Compose to deploy containerized applications to Amazon ECS. We aim to show how the transition from deploying to a local Docker environment to deploying to Amazon ECS is effortless, the application being managed in the same way for both environments.
Requirements
In order to exercise the examples in this blogpost, the following tools need to be installed locally:
Windows and MacOS: install Docker DesktopLinux: install Docker Engine and Compose CLITo deploy to Amazon ECS: an AWS account
For deploying a Compose file to Amazon ECS, we rely on the new Docker Compose implementation embedded into the Docker CLI binary. Therefore, we are going to run docker compose commands instead of docker-compose. For local deployments, both implementations of Docker Compose should work. If you find a missing feature that you use, report it on the issue tracker.
Throughout this blogpost, we discuss how to:
Build and ship a Compose Application. We exercise how to run an application defined in a Compose file locally and how to build and ship its images to Docker Hub to make them accessible from anywhere.Create an ECS context to target Amazon ECS. Run the Compose application on Amazon ECS.
Build and Ship a Compose application
Let us take an example application with the following structure:
$ tree myproject/myproject/├── backend│ ├── Dockerfile│ ├── main.py│ └── requirements.txt├── compose.yaml└── frontend ├── Dockerfile └── nginx.conf2 directories, 6 files
The content of the files can be found here. The Compose file define only 2 services as follows:
$ cat compose.yamlservices:frontend: build: frontend ports: – 80:80 depends_on: – backendbackend: build: backend
Deploying this file locally on a Docker engine is quite straightforward:
$ docker compose up -d[+] Running 3/3⠿ Network “myproject_default” Created 0.5s⠿ Container myproject_backend_1 Started 0.7s⠿ Container myproject_frontend_1 Started 1.4s
Check the application is running locally:
$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESeec2dd88fd67 myproject_frontend “/docker-entrypoint….” 4 seconds ago Up 3 seconds 0.0.0.0:80->80/tcp myproject_frontend_12c64e62b933b myproject_backend “python3 /app/main.py” 4 seconds ago Up 3 seconds myproject_backend_1
Query the frontend:
$ curl localhost:80 ## . ## ## ## ==## ## ## ## ## ===/”””””””””””””””””___/ ==={ / ===-______ O __/ __/ ___________/Hello from Docker!
To remove the application:
$ docker compose down[+] Running 3/3⠿ Container myproject_frontend_1 Removed 0.5s⠿ Container myproject_backend_1 Removed 10.3s⠿ Network “myproject_default” Removed 0.4s
In order to deploy this application on ECS, we need to have the images for the application frontend and backend stored in a public image registry such as Docker Hub. This enables the images to be pulled from anywhere.
To upload the images to Docker Hub, we can set the image names in the compose file as follows:
$ cat compose.yamlservices:frontend: image: myhubuser/starter-front build: frontend ports: – 80:80 depends_on: – backendbackend: image: myhubuser/starter-back build: backend
Build the images with Docker Compose:
$ docker compose build[+] Building 1.2s (16/16) FINISHED => [myhubuser/starter-front internal] load build definition from Dockerfile 0.0s=> => transferring dockerfile: 31B 0.0s=> [myhubuser/starter-back internal] load build definition from Dockerfile 0.0s…
In the build output we can notice the image has been named and tagged according to the image field from the Compose file.
Before pushing the images to Docker Hub, check to be logged in:
$ docker login…Login Succeeded
Push the images:
$ docker compose push[+] Running 0/16⠧ Pushing Pushing frontend: f009a503aca1 Pushing [===========================================… 2.7s…
The images should be stored now in Docker Hub.
Create an ECS Docker Context
To make Docker Compose target the Amazon ECS platform, we need first to create a Docker context of the ECS type. A docker context is a mechanism that allows redirecting commands to different Docker hosts or cloud platforms.
We assume at this point that we have AWS credentials set up in the local environment for authenticating with the ECS platform.
To create an ECS context run the following command:
$ docker context create ecs myecscontext? Create a Docker context using: [Use arrows to move, type to filter] An existing AWS profile AWS secret and token credentials> AWS environment variables
Based on the familiarity with the AWS credentials setup and the AWS tools use, we are prompted to choose between 3 context setups. To skip the details of AWS credential setup, we choose the option of using environment variables.
$ docker context create ecs myecscontext? Create a Docker context using: AWS environment variablesSuccessfully created ecs context “myecscontext”
This requires to have the AWS_ACCESS_KEY and AWS_SECRET_KEY set in the local environment when running Docker commands that target Amazon ECS.
The current context in use is marked by * in the output of context listing:
$ docker context lsNAME TYPE DESCRIPTION DOCKER ENDPOINT default * moby Current DOCKER_HOST based configuration unix:///var/run/docker.sockmyecscontext ecs credentials read from environment
To make all subsequent commands target Amazon ECS, make the newly created ECS context the one in use by running:
$ docker context use myecscontextmyecscontext$ docker context lsNAME TYPE DESCRIPTION DOCKER ENDPOINT default moby Current DOCKER_HOST based configuration unix:///var/run/docker.sockmyecscontext * ecs credentials read from environment
Run the Compose application on Amazon ECS
An alternative to having it as the context in use is to set the context flag for all commands targeting ECS.
WARNING: Check in advance the cost that the ECS deployment may incur for 2 ECS services, load balancing (ALB), cloud map (DNS resolution) etc.
For the following commands, we keep ECS context as the current context in use. Before running commands on ECS, make sure the Amazon account credentials grant access to manage resources for the application as detailed in the documentation.
We can now run a command to check we can successfully access ECS.
$ AWS_ACCESS_KEY=”*****” AWS_SECRET_KEY=”******” docker compose lsNAME STATUS
Export the AWS credentials to avoid setting them for every command.
$ export AWS_ACCESS_KEY=”*****”$ export AWS_SECRET_KEY=”******”
The deploy the sample application to ECS, we can run the same command as in the local deployment:
$ docker compose upWARNING services.build: unsupported attribute WARNING services.build: unsupported attribute [+] Running 18/18⠿ myproject CreateComplete 206.0s⠿ FrontendTCP80TargetGroup CreateComplete 0.0s⠿ CloudMap CreateComplete 46.0s⠿ FrontendTaskExecutionRole CreateComplete 19.0s⠿ Cluster CreateComplete 5.0s⠿ DefaultNetwork CreateComplete 5.0s⠿ BackendTaskExecutionRole CreateComplete 19.0s⠿ LogGroup CreateComplete 1.0s⠿ LoadBalancer CreateComplete 122.0s⠿ Default80Ingress CreateComplete 1.0s⠿ DefaultNetworkIngress CreateComplete 0.0s⠿ BackendTaskDefinition CreateComplete 2.0s⠿ FrontendTaskDefinition CreateComplete 3.0s⠿ FrontendServiceDiscoveryEntry CreateComplete 1.0s⠿ BackendServiceDiscoveryEntry CreateComplete 2.0s⠿ BackendService CreateComplete 65.0s⠿ FrontendTCP80Listener CreateComplete 3.0s⠿ FrontendService CreateComplete 66.0s
Docker Compose converts the Compose file to a CloudFormation template defining a set of AWS resources. Details on the resource mapping can be found in the documentation. To review the CloudFormation template generated, we can run the command:
$ docker compose convertWARNING services.build: unsupported attribute WARNING services.build: unsupported attribute AWSTemplateFormatVersion: 2010-09-09Resources: BackendService: Properties: Cluster: Fn::GetAtt: – Cluster – Arn DeploymentConfiguration: MaximumPercent: 200 MinimumHealthyPercent: 100…
To check the state of the services, we can run the command:
$ docker compose psNAME SERVICE STATUS PORTStask/myproject/8c142dea1282499c83050b4d3e689566 backend Running task/myproject/a608f6df616e4345b92a3d596991652d frontend Running mypro-LoadB-1ROWIHLNOG5RZ-1172432386.eu-west-3.elb.amazonaws.com:80->80/http
Similarly to the local run, we can query the frontend of the application:
$ curl mypro-LoadB-1ROWIHLNOG5RZ-1172432386.eu-west-3.elb.amazonaws.com:80 ## . ## ## ## ==## ## ## ## ## ===/”””””””””””””””””___/ ==={ / ===-______ O __/ __/ ___________/Hello from Docker!
We can retrieve logs from the ECS containers by running the compose logs command:
$ docker compose logsbackend | * Serving Flask app “main” (lazy loading)backend | * Environment: productionbackend | WARNING: This is a development server. Do not use it in a production deployment.backend | Use a production WSGI server instead….frontend | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.shfrontend | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.shfrontend | /docker-entrypoint.sh: Configuration complete; ready for start upfrontend | 172.31.22.98 – – [02/Mar/2021:08:35:27 +0000] “GET / HTTP/1.1″ 200 212 “-” “ELB-HealthChecker/2.0″ “-“backend | 172.31.0.11 – – [02/Mar/2021 08:35:27] “GET / HTTP/1.0″ 200 -backend | 172.31.0.11 – – [02/Mar/2021 08:35:57] “GET / HTTP/1.0″ 200 -frontend | 172.31.22.98 – – [02/Mar/2021:08:35:57 +0000] “GET / HTTP/1.1″ 200 212 “-” “curl/7.75.0″ “94.239.119.152”frontend | 172.31.22.98 – – [02/Mar/2021:08:35:57 +0000] “GET / HTTP/1.1″ 200 212 “-” “ELB-HealthChecker/2.0″ “-”
To terminate the Compose application and release AWS resources, run:
$ docker compose down[+] Running 2/4⠴ myproject DeleteInProgress User Initiated 8.5s⠿ DefaultNetworkIngress DeleteComplete 1.0s⠿ Default80Ingress DeleteComplete 1.0s⠴ FrontendService DeleteInProgress 7.5s…
The Docker documentation provides several examples of Compose files, supported features, details on how to deploy and how to update a Compose application running in ECS, etc.
The following features are discussed in detail:
use of private imagesservice discoveryvolumes and secrets definitionAWS-specific service properties for auto-scaling, IAM roles and load balancinguse of existing AWS resources
Summary
We have covered the transition from local deployment of a Compose application to the deployment on Amazon ECS. We have used a minimal generic example for demonstrating how to use the Docker Compose cloud-capability. For a better understanding on how to update the Compose file and use specific AWS features, the documentation provides much more details.
Resources:
Docker Compose embedded in the Docker CLIhttps://github.com/docker/compose-cli/blob/main/INSTALL.mdCompose to ECS support https://docs.docker.com/cloud/ecs-integration/ECS-specific Compose examples:https://docs.docker.com/cloud/ecs-compose-examples/Deploying Docker containers to ECS:https://docs.docker.com/cloud/ecs-integration/Sample used to demonstrate Compose commands:https://github.com/aiordache/demos/tree/master/ecsblog-demo
The post Docker Compose: From Local to Amazon ECS appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/
