Kategorie: Kubernetes

Docker V2 Github Action is Now GA

Docker is happy to announce the GA of our V2 Github Action. We’ve been working with @crazy-max over the last few months along with getting feedback from the wider community on how we can improve our existing Github Action. We have now moved from our single action to a clearer division and advanced set of options that not only allow you to just build & push but also support features like multiple architectures and build cache.

The big change with the advent of our V2 action is also the expansion of the number of actions that Docker is providing on Github. This more modular approach and the power of Github Actions has allowed us to make the minimal UX changes to the original action and add a lot more functionality.

We still have our more meta build/push action which does not actually require all of these preconfiguration steps and can still be used to deliver the same workflow we had with the previous workflow! To Upgrade the only changes are that we have split out the login to a new step and also now have a step to setup our builder. 

–
name: Setup Docker Buildx
uses: docker/setup-buildx-action@v1

This step is setting up our builder, this is the tool we are going to use to build our Docker image. 

This means our full Github Action is now: 

name: ci

on:
push:
branches: main

jobs:
main:
runs-on: ubuntu-latest
steps:
–
name: Setup Docker Buildx
uses: docker/setup-buildx-action@v1
–
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
–
name: Build and push
id: docker_build
uses: docker/build-push-action@v2
with:
push: true
tags: bengotch/samplepython:latest

For users looking for more information on how to move from V1 of the Github Action to V2, check out our release migration notes. 

Let’s now look at some of the more advanced features we have unlocked by adding in this step and the new QEMU option.

Multi platform

By making use of BuildKit we now have access to multi-architecture builds, this allows us to build images targeted at more than one platform and also build Arm images.

To do this, we will need to add in our QEMU step: 

name: Set up QEMU
uses: docker/setup-qemu-action@v1

And then within our build & push step we will need to specify out platform to use: 

–
name: Build and push
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
push: true
tags: |
bengotch/app:latest

Cache Registry 

To make use of caching to speed up my builds I can now make use of the 

name: ci

on:
push:
branches: master

jobs:
registry-cache:
runs-on: ubuntu-latest
steps:
–
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
–
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
–
name: Build and push
uses: docker/build-push-action@v2
with:
push: true
tags: user/app:latest
cache-from: type=registry,ref=user/app:latest
cache-to: type=inline

To see more examples of the best practices for using our latest version of the Github Action check out Chads example repohttps://github.com/metcalfc/docker-action-examples. You can make use of the features in here or some of the more advanced features we can now offer with the V2 action such as push to multiple registries, use of a local registry for e2e test, export an image to the Docker client and more! 

To learn more about the changes to our Github Action, have a read through our updated usage documentation or check out our blog post on the best practices with Docker and Github Actions. If you have questions or feedback on the changes from V1 to V2 please raise tickets on our repo or our public roadmap
The post Docker V2 Github Action is Now GA appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Docker Hub Image Retention Policy Delayed, Subscription Updates

Today we are announcing that we are pausing enforcement of the changes to image retention until mid 2021. Two months ago, we announced a change to Docker image retention policies to reduce overall resource consumption. As originally stated, this change, which was set to take effect on November 1, 2020, would result in the deletion of images for free Docker account users after six months of inactivity. After this announcement, we heard feedback from many members of the Docker community about challenges this posed, in terms of adjusting to the policy without visibility as well as tooling needed to manage an organization’s Docker Hub images. Today’s announcement means Docker will not enforce image expiration enforcement on November 1. Instead, Docker is focusing on consumption-based subscriptions that meet the needs of all of our customers. In this model, as the needs of a developer grow, they can upgrade to a subscription that meets their requirements without limits.

This change means that developers will get a base level of consumption to start, and can extend their subscriptions as their needs grow and evolve, only paying for what is actually needed. The community of 6.7 million registered Docker developers is incredibly diverse–the requirements of someone getting started with containers is different than the needs of an OSS project organizer which are also different for a 40,000 person software development team. Our new model gives each individual developer or organization the opportunity to scale their usage and consumption along the dimensions that make most sense to them. 

As we make this move to consumption-based subscriptions, we are also creating new capabilities to help users understand and manage their usage of various resources on the Docker platform. As an example of this, for image storage on Docker Hub we will soon release an experimental Hub CLI tool, a Hub Dashboard and new APIs. Our goal is to give developers the insights required to effectively understand and manage their image storage in Docker Hub. We will be delivering the first tools in the coming weeks, and will announce the timeline for new image retention policies early in 2021.

Reminder: Image pull consumption tiers

Continuing with our move towards consumption-based limits, customers will see the new rate limits for Docker pulls of container images at each tier of Docker subscriptions starting from November 1, 2020. Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction as long as the quantities are not excessive or abusive. We want our Docker Pro subscription to be the best way for individual developers to work with Docker and our Team subscription to continue to add value for teams to come together as they scale their usage with tools like CI/CD. The levels for what is abusive/excessive will be managed with these two goals in mind. 

Excessive usage/abuse of Pro and Team limits will be initially managed through a process where a customer will be informed about the usage overage through email as well as the ability to get usage information in response headers from Docker Hub. Continued abuse may be followed with hard restriction in usage. Details about Docker subscription levels and differentiators are available on the Docker Pricing Page. 

Going forward you will see this model extended to other capabilities available from Docker in order to provide maximum flexibility for developers. With millions of developers pulling billions of images per month, any change we make to the system has to be considered with our community in mind. We appreciate the feedback and suggestions from the Docker community, and we are excited to share more new features with you in the coming weeks and months.
The post Docker Hub Image Retention Policy Delayed, Subscription Updates appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Understanding Inner Loop Development and Pull Rates

We have heard feedback that given the changes Docker introduced relating to network egress and the number of pulls for free users, that there are questions around the best way to use Docker as part of your development workflow without hitting these limits. This blog post covers best practices that improve your experience and uses a sensible consumption of Docker which will mitigate the risk of hitting these limits and how to increase the limits depending on your use case. 

If you are interested in how these limits are addressed in a CI/CD pipeline, please have a look at our post: Best Practices for using Docker Hub for CI/CD. If you are using Github Action, have a look at our Docker Github Actions post.

Prerequisites

To complete this tutorial, you will need the following:

Free Docker Account You can sign-up for a free Docker account and receive free unlimited public repositoriesDocker running locallyInstructions to download and install DockerAn IDE or text editor to use for editing files. I would recommend VSCode

Determining Number of Pulls

Docker defines pull rate limits as the number of manifest requests to Docker Hub. Rate limits for Docker pulls are based on the account type of the user requesting the image – not the account type of the image’s owner. For anonymous (unauthenticated) users, pull rates are limited based on the individual IP address. 

We’ve been getting questions from customers and the community regarding container image layers. We are not counting image layers as part of the pull rate limits. Because we are limiting on manifest requests, the number of layers (blob requests) related to a pull is unlimited at this time

As an anonymous user, you are able to perform up to 100 pulls within a six hour window. This is a high enough limit to allow individual developers to build their images on their local development machine without worry of reaching the pull limits.

If you need to perform more than 100 pulls per six hour window, create a free Docker account which will allow you to perform up to 200 pulls per six hour window. Doubling the amount of pulls compared to the anonymous limit.

To get a good idea of how many pulls a build will incur, you can take a look at the number of FROM commands in your Docker file. Once an image has been pulled to your local machine, it will not incur a pull on subsequent builds.

So for example, if we had an application that was made up of a frontend UI, a REST service and a database. We would have two Dockerfiles. One for building the UI and one for building the REST service. We would  then combine these images with our database inside of a compose file. Inside of this compose file, we would include our database image.

Let’s take a look at this scenario using the react-express-mongodb example in the Awesome Compose repository. Clone the awesome-compose repository and open the react-express-mongodb folder in your favorite editor.

$ git clone git@github.com:docker/awesome-compose.git

$ cd awesome-compose/react-express-mongodb

Expand the frontend fold and open the Dockerfile.

As you can see on line 1, we are using the node:lts-buster-slim image. If we do not already have this image locally then when we perform a build, this image will be pulled from Docker Hub and count as one pull.

Likewise in the backend folder, we see a Dockerfile that is used to build the backend image. On line 1 of this file, we are also using the node:lts-buster-slim image. Again, if you have not already pulled this image from Docker Hub, when you run a docker build, then Docker will pull this image and count it as one pull.

To recap, since we are using the same base image (node:lts-buster-slim) for each of our application images, we will only have to pull that image once and therefore only incur one pull.

The same is true for the mongo:4.2.0 image. When you run the docker-compose up command, the mongo image will be pulled, if not present locally, and increase the pull limit counter by one.

So in the above example, with zero images present locally, you will incur two pull requests to Hub. Even if we expanded this out to a slightly more complex architecture with a few more services that are also written in node, we would still only incur two pull requests. One for the node image and one for the mongodb image.

Now let’s take a look at a more advanced build scenario below.

Here is an example of a Dockerfile that uses multi-stage builds:

1 # syntax=docker/dockerfile:1.1.72 ARG GO_VERSION=1.13.7-buster34 FROM golang:${GO_VERSION} AS golang56 FROM golang AS build7 ….8 FROM debian:buster AS foo9 ….10 FROM scratch AS final11 COPY –from=build /bin/foo /bin/foo

Simply counting the number of FROM‘s will not work in all situations but it is a good general proxy. If I have a FROM command that references an image that I only have locally, then a Hub pull will not occur. We can also use  FROM commands in a multi-stage build to reference other build-stages located in the same Dockerfile. 

In the above Dockerfile, we have multiple FROM statements of which the total comprises a multi-stage build. What will actually be pulled from Hub depends on the state of your local cache, which build target is set and whether or not you’re using BuildKit.

Let’s walk through a scenario where we are not using BuildKit.

On line 4 we can see that we are referencing the GO_VERSION build argument:

FROM golang:${GO_VERSION} AS golang

The value of GO_VERSION is dependent on whether we have passed a value using the –build-arg option or not. 

So, for example, let’s say we have the golang:1.13.7-buster image on our local machine. If we do not override the GO_VERSION then we will not incur a pull. On the other hand, if we set the GO_VERSION to 1.15.2-buster and do not have this image locally, then we will incur a pull from Hub.

Another point to keep in mind when counting FROMs, is that a FROM command can reference the scratch image. The scratch image is not an image on Hub but is treated specially by Docker and never pulls anything from Hub but is used as a starting point for creating an empty image.

Now let’s take a look at building an image using BuildKit. We’ll use the same sample Dockerfile from above.

When we run a build, the FROM scratch AS final stage is started which will trigger the following flow:

FROM golang AS build is startedWhich triggers the FROM golang:$(GO_VERSION} AS golang stageWhich will pull the golang:1.13.7-buster image from Hub if it is not present locally

In this scenario, the FROM debian:buster AS foo stage on line 8 is not used in the final image and therefore will not be built and the debian:buster image will not be pulled from Hub. Even though it is a FROM statement in the Dockerfile, it is not used and need not be counted when figuring out the number of pulls that will occur.

Unlimited Pulls

If you are working on a larger project that has a lot of different base images or you are building images and removing them often, then the best option is to purchase a Docker Pro Account or a Docker Team Account.

Both the Pro and Team accounts give you unlimited pulls and therefore not subject to rate limiting. You also receive unlimited private repositories with these plans.

Conclusion

In this article we discussed how pulls are counted when building images using Docker. We first talked about a common application that has a frontend, backend and a datastore and how this scenario will not reach the 100 pull limit for anonymous users. Then we discussed a more advanced Dockerfile that uses multi-stage builds and how this can potentially affect your pull limits. Although not enough to reach the 200 pull limit for authenticated accounts. 

For more information and common questions, please read our FAQ. As always, please feel free to reach out to us on Twitter (@docker) or to me directly (@pmckee).

To get started using Docker sign up for a free Docker account and take a look at our getting started guide.
The post Understanding Inner Loop Development and Pull Rates appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Docker and Snyk Extend Partnership to Docker Official and Certified Images

Today we are pleased to announce that Docker and Snyk have extended our existing partnership to bring vulnerability scanning to Docker Official and certified images. As the exclusive scanning partner for these two image categories, Snyk will work with Docker to provide developers with insights into our most popular images. It builds on our previous announcement earlier this year where Snyk scanning was integrated into the Docker Desktop and Docker Hub. This means that developers can now incorporate vulnerability assessment along each step of the container development and deployment process.

Docker Official images represent approximately 25% of all of the pull activity on Docker Hub. Docker Official images are used extensively by millions of developers and developer world wide teams to build and run tens of millions of containerized applications. By integrating vulnerability scanning from Snyk users are now able to get more visibility into the images and have a higher level of confidence that their applications are secure and ready for production.

Docker Official images that have been scanned by Snyk will be available early next year.

You can read more about it from Snyk here and you can catch Docker CEO Scott Johnson and Snyk CEO Peter McKay discuss the partnership during the Snykcon user conference keynote Thursday morning October 22 at 8:30 AM Pacific. You can register for Snykcon at http://bit.ly/SnykConDocker

Additional Resources

Get started with scanning in the desktop nowhttps://www.docker.com/get-started

Learn more about scanning in Docker Hubhttps://goto.docker.com/on-demand-adding-container-security.htmlLearn more about scanning in Docker Desktop https://goto.docker.com/on-demand-find-fix-container-image-vulnerabilities.html
The post Docker and Snyk Extend Partnership to Docker Official and Certified Images appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/