Kategorie: Kubernetes

Kubernetes Meets High-Performance Computing

Editor’s note: today’s post is by Robert Lalonde, general manager at Univa, on supporting mixed HPC and containerized applications  Anyone who has worked with Docker can appreciate the enormous gains in efficiency achievable with containers. While Kubernetes excels at orchestrating containers, high-performance computing (HPC) applications can be tricky to deploy on Kubernetes. In this post, I discuss some of the challenges of running HPC workloads with Kubernetes, explain how organizations approach these challenges today, and suggest an approach for supporting mixed workloads on a shared Kubernetes cluster. We will also provide information and links to a case study on a customer, IHME, showing how Kubernetes is extended to service their HPC workloads seamlessly while retaining scalability and interfaces familiar to HPC users.HPC workloads unique challengesIn Kubernetes, the base unit of scheduling is a Pod: one or more Docker containers scheduled to a cluster host. Kubernetes assumes that workloads are containers. While Kubernetes has the notion of Cron Jobs and Jobs that run to completion, applications deployed on Kubernetes are typically long-running services, like web servers, load balancers or data stores and while they are highly dynamic with pods coming and going, they differ greatly from HPC application patterns.Traditional HPC applications often exhibit different characteristics: In financial or engineering simulations, a job may be comprised of tens of thousands of short-running tasks, demanding low-latency and high-throughput scheduling to complete a simulation in an acceptable amount of time.A computational fluid dynamics (CFD) problem may execute in parallel across many hundred or even thousands of nodes using a message passing library to synchronize state. This requires specialized scheduling and job management features to allocate and launch such jobs and then to checkpoint, suspend/resume or backfill them.Other HPC workloads may require specialized resources like GPUs or require access to limited software licenses. Organizations may enforce policies around what types of resources can be used by whom to ensure projects are adequately resourced and deadlines are met.HPC workload schedulers have evolved to support exactly these kinds of workloads. Examples include Univa Grid Engine, IBM Spectrum LSF and Altair’s PBS Professional. Sites managing HPC workloads have come to rely on capabilities like array jobs, configurable pre-emption, user, group or project based quotas and a variety of other features.Blurring the lines between containers and HPCHPC users believe containers are valuable for the same reasons as other organizations. Packaging logic in a container to make it portable, insulated from environmental dependencies, and easily exchanged with other containers clearly has value. However, making the switch to containers can be difficult. HPC workloads are often integrated at the command line level. Rather than requiring coding, jobs are submitted to queues via the command line as binaries or simple shell scripts that act as wrappers. There are literally hundreds of engineering, scientific and analytic applications used by HPC sites that take this approach and have mature and certified integrations with popular workload schedulers. While the notion of packaging a workload into a Docker container, publishing it to a registry, and submitting a YAML description of the workload is second nature to users of Kubernetes, this is foreign to most HPC users. An analyst running models in R, MATLAB or Stata simply wants to submit their simulation quickly, monitor their execution, and get a result as quickly as possible. Existing approachesTo deal with the challenges of migrating to containers, organizations running container and HPC workloads have several options:Maintain separate infrastructuresFor sites with sunk investments in HPC, this may be a preferred approach. Rather than disrupt existing environments, it may be easier to deploy new containerized applications on a separate cluster and leave the HPC environment alone. The challenge is that this comes at the cost of siloed clusters, increasing infrastructure and management cost.Run containerized workloads under an existing HPC workload managerFor sites running traditional HPC workloads, another approach is to use existing job submission mechanisms to launch jobs that in turn instantiate Docker containers on one or more target hosts. Sites using this approach can introduce containerized workloads with minimal disruption to their environment. Leading HPC workload managers such as Univa Grid Engine Container Edition and IBM Spectrum LSF are adding native support for Docker containers. Shifter and Singularity are important open source tools supporting this type of deployment also. While this is a good solution for sites with simple requirements that want to stick with their HPC scheduler, they will not have access to native Kubernetes features, and this may constrain flexibility in managing long-running services where Kubernetes excels.Use native job scheduling features in KubernetesSites less invested in existing HPC applications can use existing scheduling facilities in Kubernetes for jobs that run to completion. While this is an option, it may be impractical for many HPC users. HPC applications are often either optimized towards massive throughput or large scale parallelism. In both cases startup and teardown latencies have a discriminating impact. Latencies that appear to be acceptable for containerized microservices today would render such applications unable to scale to the required levels.All of these solutions involve tradeoffs. The first option doesn’t allow resources to be shared (increasing costs) and the second and third options require customers to pick a single scheduler, constraining future flexibility.Mixed workloads on KubernetesA better approach is to support HPC and container workloads natively in the same shared environment. Ideally, users should see the environment appropriate to their workload or workflow type.One approach to supporting mixed workloads is to allow Kubernetes and the HPC workload manager to co-exist on the same cluster, throttling resources to avoid conflicts. While simple, this means that neither workload manager can fully utilize the cluster. Another approach is to use a peer scheduler that coordinates with the Kubernetes scheduler. Navops Command by Univa is a solution that takes this third approach, augmenting the functionality of the Kubernetes scheduler. Navops Command provides its own web interface and CLI and allows additional scheduling policies to be enabled on Kubernetes without impacting the operation of the Kubernetes scheduler and existing containerized applications. Navops Command plugs into the Kubernetes architecture via the ‘schedulerName’ attribute in the pod spec as a peer scheduler that workloads can choose to use instead of the Kubernetes stock scheduler as shown below.With this approach, Kubernetes acts as a resource manager, making resources available to a separate HPC scheduler. Cluster administrators can use a visual interface to allocate resources based on policy or simply drag sliders via a web UI to allocate different proportions of the Kubernetes environment to non-container (HPC) workloads, and native Kubernetes applications and services.From a client perspective, the HPC scheduler runs as a service deployed in Kubernetes pods, operating just as it would on a bare metal cluster. Navops Command provides additional scheduling features including things like resource reservation, run-time quotas, workload preemption and more. This environment works equally well for on-premise, cloud-based or hybrid deployments.Deploying mixed workloads at IHMEOne client having success with mixed workloads is the Institute for Health Metrics & Evaluation (IHME), an independent health research center at the University of Washington. In support of their globally recognized Global Health Data Exchange (GHDx), IHME operates a significantly sized environment comprised of 500 nodes and 20,000 cores running a mix of analytic, HPC, and container-based applications on Kubernetes. This case study describes IHME’s success hosting existing HPC workloads on a shared Kubernetes cluster using Navops Command.For sites deploying new clusters that want access to the rich capabilities in Kubernetes but need the flexibility to run non-containerized workloads, this approach is worth a look. It offers the opportunity for sites to share infrastructure between Kubernetes and HPC workloads without disrupting existing applications and businesses processes. It also allows them to migrate their HPC workloads to use Docker containers at their own pace.
Quelle: kubernetes

My Three Favorite New Features in Docker Enterprise Edition

I’ve been at Docker for just over two years now, and I’ve worked with every version of Docker Enterprise Edition (née Docker Datacenter) since before there even was a Docker Enterprise Edition (EE). I’m more excited about this new release than any previous release.
There are several new features that are going to ease the management of your applications (both traditional and cloud-native) wherever you need them to run: the cloud or the data center, virtual or physical, Linux or Windows – and now even IBM Z mainframes.
It would take too long to discuss all of the new features, so with that in mind, I’m going to talk about my three favorite features in Docker EE 17.06.

Hybrid-OS Clusters
Docker and Microsoft introduced support for Windows Server containers last fall. This was a major milestone that helped Docker move towards the goal of embracing apps across the entirety of the data center. With this latest release Docker extends hybrid OS operations even further: IT admins can now build and manage clusters comprised of Linux, Windows Server 2016, and IBM Z mainframes  – all from the same management plane. This means you can manage applications comprised of both Windows and Linux components from Docker Universal Control Plane. For instance, you can run your web front end on Linux and connect that to Microsoft SQL Server running on Windows.
Docker EE 17.06 is the first Containers-as-a-Service platform to offer production-level support for the integrated management and security of Windows Server Containers.
For more information on hybrid-OS clusters, check out this video.
Enhanced Role-based Access Control (RBAC)
Docker EE has always featured RBAC. With Docker EE 17.06 we’ve enhanced these capabilities to further extend the way administrators manage access to cluster resources.
To better understand how RBAC works in Docker EE 17.06 it’s probably best if I define four concepts:

Custom Roles: A role is essentially a set of permissions that define what operations someone can perform on cluster resources. As in previous releases, Docker EE 17.06 has a set of predefined roles (View Only, Full Control, etc). What’s new in this release is the ability for administrators to choose from dozens of individual capabilities to define custom roles.

For instance, an admin could define a ‘network-ops’ role that only grants the ability to perform a subset of tasks specifically related to network functionality.
 Note: This image only shows a small subset of all the various operation permissions available in Docker EE 17.06
In short roles are what someone can do when working with your Docker EE cluster.

Subject: Subjects define who can perform certain tasks. Subjects can be Docker EE users, teams or organizations.

Collections: Collections are a new concept in Docker EE. They provide a mechanism for administrators to group cluster resources (services, containers, volumes, networks, secrets, etc) together. An admin assigns a special Docker label (com.docker.ucp.access.label) to a particular resource to define what collection the resource belongs to.
Collections can be nested into a directory-like hierarchy. For instance an admin user can create a prod collection, and then a webserver collection beneath that.

Nested collections will inherit permissions from their parent collections.
You can think of collections as where someone can perform tasks.
 

Grant: A grant defines who (subject) can do what (role) where (collection). For example, you can create a grant that specifies that the “Dev Team” gets “View Only” access against resources in the “/Production” collection.

In addition to these new capabilities, Docker EE Advanced 17.06 extends the concept of RBAC to nodes as well. So now administrators can subdivide cluster servers between teams, and ensure that those dedicated resources are only accessed by individuals who have been explicitly granted permission. These features give administrators nearly infinite flexibility with regards to how they want to secure their cluster resources.
For more information on RBAC in Docker EE 17.06 check out this video.
Automated Image Promotion and Immutable Repos
Ok, this is technically two features, but they’re both awesome: Automated Image Promotion and Immutable Repos. These two capabilities allow administrators to further ensure the integrity of Docker images.
Automated image promotion gives IT practitioners the ability to define criteria that, when met, will automatically promote an image from one Docker Trusted Registry (DTR) repository to another.
For instance, today you might create a new version of an application, run it through QA, and then – if it passes – manually promote it to the production repo. The QA process could include steps such as scanning for vulnerabilities or the usage of components with certain licenses.
With Docker EE 17.06, you can automate portions of this process. You can define criteria based on the the image tag, the number of vulnerabilities in the image, presence of certain packages, or the type of license found in the image. If those criteria are met, the image will automatically be promoted from one repo to the other.

Additionally, you can apply multiple policies to create sophisticated automated promotion scenarios.
Immutable repos work alongside image promotion (as well as the existing security scanning and image signing features) to help protect the integrity of your Docker images. As the name implies, immutable repos allow administrators to prevent image tags from being changed in a given repository.
This is aimed at stopping a scenario where someone pushes a version of an image with a given tag, and then someone else overwrites that image by pushing a different version using the same tag as the original user. With immutable repos you can be assured that your images will not be accidentally (or intentionally) overwritten.
For more information on image promotion and immutable repos, please see this video.
Secure and Manage More Applications
Ok – I know I said I was going to talk about my three favorite new features, but I have to add my other favorite new feature: Docker Security Scanning for Windows images. Docker Security Scanning, part of Docker EE Advanced, automatically scans images for common vulnerabilities and exploits as they are pushed to DTR. Previously this has only worked with Linux images, but with Docker EE Advanced 17.06 it now also works with Windows images!
So there you have it: my three (or four or five depending on how you counted) favorite new features in Docker EE 17.06.
Thanks for taking the time to learn what’s new in Docker EE 17.06. Like I said, there are plenty of other new features. Heck, I didn’t even talk about multi-stage builds or the new UI. I hope after reading this, that you’re as excited about Docker EE 17.06 as I am.
Continue your Docker journey with these helpful links:

Try Docker Enterprise Edition for free
Register for an upcoming Docker webinar
Review What’s New with Docker EE
Read the documentation

Are you excited about #Docker EE ? @mikegcoleman is – read about his fave featuresClick To Tweet

 
The post My Three Favorite New Features in Docker Enterprise Edition appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Docker is Headed to Gartner Catalyst 2017

The Docker team will be in sunny San Diego, CA, August 21-23 for Gartner Catalyst. Come by and visit us in Booth #508 to meet with our Docker Enterprise Edition (EE) experts, see a demo of Docker EE, and ask us any questions you may have before and after any of the Gartner sessions on Docker and containers. Better yet, schedule a meeting with us and we’ll not only answer all your questions, you will also get a special gift.

This year’s Catalyst event includes an entire topic dedicated to Docker and containers, which you can find by looking for the topic “Docker & Containers” in the schedule builder. If you are still trying to separate all the fact from fiction about Docker and want a specific recommendation, there is a great Tech Demo session by Gartner analyst Richard Watson we think you might like titled Seven Docker & Container Myths We Need To Bust. 
We hope you will join us at Gartner Catalyst to get the latest research on the next big trends for IT, but if you are not in San Diego, we hope to see you at one of these other upcoming events:

VMworld, Las Vegas, NV, August 27-31 (Booth #1206)
Microsoft Ignite, Orlando, FL, September 25-29
DockerCon Europe, Copenhagen, DK, October 16-19

To learn more about Docker solutions for the enterprise:

Test drive Docker Enterprise Edition for free
Read more about Docker or view pricing
Learn what’s new with Docker Enterprise Edition

Attending @gartner_inc #gartnercatalyst next week in San Diego? Visit us in Booth #508 to meet…Click To Tweet

The post Docker is Headed to Gartner Catalyst 2017 appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Announcing the New Release of Docker Enterprise Edition

We are excited to share the new release of Docker Enterprise Edition. By supporting IBM Z and Windows Server 2016, this release puts us further in the lead with the first Containers-as-a-Service (CaaS) solution in the market for the modernization of all applications without disruption to you and your IT environment.
 

 
Docker Enterprise Edition (EE) 17.06 embraces Windows, Linux and Linux-based mainframe applications, bringing the key benefits of CaaS to the enterprise application portfolio. Most enterprises manage a diverse set of applications that includes both traditional applications and microservices, built on Linux and Windows, and intended for x86 servers, mainframes, and public clouds. Docker EE unites all of these applications into single platform, complete with customizable and flexible access control, support for a broad range of applications and infrastructure, and a highly automated software supply chain. These capabilities allow organizations to easily layer Docker EE onto existing processes and workflows, aligning to existing organizational structures while delivering improved resource utilization and reduced maintenance time.
This release includes UCP 2.2 and DTR 2.3 and establishes Docker EE as a key IT platform for both new application development as well as application modernization across both on-premises and cloud environments.
 

Multi-Architecture Orchestration
Docker EE is the only solution for modernizing Windows, Linux, and mainframe applications across on-premises and cloud, without requiring code changes. With organizations dedicating large portions of their IT budget towards maintaining existing apps and the digital era forcing everyone to focus on innovation, Docker EE provides a non-disruptive way to modernize existing applications to make them more portable, more scalable, and easier to update. Most enterprise organizations have a mixture of .NET, Java apps and mainframe applications in their portfolio. Docker EE provides a way to modernize all these different applications by packaging them in a standard format which does not require software development teams to change their code. Organizations can containerize traditional apps and microservices and deploy them in the same cluster, either on-premises or in the cloud.
Key new features include:

Support for full lifecycle management of Docker Windows containers including image scanning, secrets management, and overlay networking
Integrate Windows and Linux applications through the use of overlay networking to support hybrid applications
Ability to intelligently orchestrate across mixed clusters of Windows, Linux, and mainframe worker nodes
With added support of Linux on IBM z Systems, Docker delivers a consistent experience (Compose files, networking, security, lifecycle management) across Linux, Windows, and Linux-on-mainframe applications

 
Secure Multi-Tenancy
As container adoption grows across an organization, roles and responsibilities need to align with existing organizational structures and processes. The latest release of Docker EE allows organizations to customize role-based access control and define both physical and logical boundaries for different teams sharing the same Docker EE environment. These new capabilities allow teams to bring their own organizational models to a Docker environment whether that is a shared IT services model where different teams rent their own nodes, multiple teams share resources, or a specific team is granted access to a collection of specific resources. The enhancements allow complex organizations to easily onboard new lines of business while keeping application owners separate across a shared environment.
Key new features include:

Leverage built-in default roles or create custom roles with granular permissions from the entire Docker API
Assign grants to users and teams for resource collections that include services, containers, volumes, networks, and secrets
Leverage RBAC for nodes to segment a team’s access to a specific set of nodes within a Docker EE environment

 
Policy-Based Automation
Docker EE is a platform solution that supports a wide variety of applications, and a key priority is ensuring that this diversity does not add complexity nor slow down the software supply chain. In a dynamic container environment, organizations need to automate as much of the process as they can without sacrificing security. New features in Docker EE allow organizations to create predefined policies that can remove bottlenecks in the process to maintain compliance and prevent human errors, while still accelerating application delivery.
Key new features include:

Automatic image promotion using pre-defined policies to move images from one repository to another within the same registry
Immutable repositories prevent image tags from being modified or deleted, ensuring that production app repositories are secure and locked down
New APIs for:

Access control permissions
User / Team / Org management
Cluster configuration

 
Next Steps:
There are many new and exciting capabilities with this release of Docker Enterprise Edition and over the next few weeks, we’ll be going into more detail on each of them. To learn more, check out these additional resources:

See the new features in action in our new hosted demo environment. With no software to install, you’re just minutes away from experiencing Docker EE 17.06 for yourself.
Register for these upcoming webinars:

Thursday, Aug. 24th: What’s New with Docker Enterprise Edition
Tuesday, Aug. 29th: Docker Captains on Deck: Swarm 
Thursday, Aug. 31st: Deploying Multi-OS Applications with Docker
Also stay tuned for What’s New sessions in your local region

Read the documentation or learn more about Docker EE

Announcing new release of #Docker Enterprise Edition – platform for modernization without disruptionClick To Tweet

The post Announcing the New Release of Docker Enterprise Edition appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

DockerCon Europe Diversity Scholarship!

Each year, DockerCon brings the community together to learn, belong and contribute. With generous support from the  OCI) our team has created the DockerCon Europe Scholarship Program to provide members of the Docker community, who are traditionally underrepresented, mentorship and a financial scholarship to attend DockerCon in Copenhagen this year. This scholarship program aims to foster inclusivity by supporting members of our community through access to resources, tools and mentorship needed to facilitate career and educational development.

If you are interested in applying for the DockerCon Scholarship, follow the steps below:
Application Process:
The application process includes completing one of the five self-paced trainings along with the scholarship application form.
Step 1
Complete at least one of the free self-paced courses available in the Docker Playground. These courses are intended for both Dev and Ops beginner and intermediate level Docker users. Select which course you feel best fits you.
Step 2
After you’ve finished one of the courses, complete the application here. In the application, you will need to provide the name(s) of the lab(s) you completed along with the answers to the quiz at the end of the course.  
Deadline to Apply:
Tuesday, 5 September, 2017 at 5:00PM PST
Selection Process
A committee of Docker community members will review and select the scholarship recipients based on the completion of one of the courses and application. Recipients will be notified by the week of 18 September 2017.  
What’s included if you are selected for the scholarship:

Full DockerCon Conference Pass
Round-trip airfare
Hotel accommodations for 4 nights (16 October, 17 October, 18 October, 19 October )
1:1 mentorship session with a member of the  Docker community on-site at DockerCon

Requirements

All applicants must complete at least one self-paced online course found at: http://training.docker.com/category/self-paced-online 
Must be able to attend DockerCon Europe 2017 (16 October – 19 October, 2017)
Must be 18 years old or older to apply
Must have a valid passport to travel to Copenhagen, Denmark.
Learn more about the DockerCon Diversity Scholarship here. 

Have questions or concerns? Reach us at dockercon@docker.com
More free Docker ressources:

Attend local Docker meetups
Check out the Docker Playground
View the Docker Youtube Channel

#DockerCon Europe Diversity Scholarship is now open! Learn more and apply here!Click To Tweet

The post DockerCon Europe Diversity Scholarship! appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Be a Budget Hero with Docker Enterprise Edition

We recently started a multi-part learning series for SysAdmins and IT professionals called IT Starts With Docker. We started with the basics, covering container technology and Docker solutions for the enterprise. Now, we shift to the important question: Is it worth your time and your company’s investment to further explore Docker Enterprise Edition (EE)?
The resounding answer to that question is YES. IT teams who have adopted Docker EE are finding it faster and easier to deploy and maintain their applications, plus drive better infrastructure utilization, all without touching the underlying code. Developer teams are realizing productivity gains of their own by being able to onboard new developers faster, shortening the cycles from development to production, and elimination of the burdensome “it worked on my machine” problems.

Try the simple ROI calculator for yourself. It takes just a couple of minutes and allows you to estimate your own savings with Docker EE and how you can become the budget hero of your department. Then, register for our live webinar on Tuesday, August 15th, The Business Value of Docker, where we will outline how organizations like your own are saving 50% on their total costs with Docker EE. 
The calculator is based on real data from Docker EE customers across a broad range of industries. In just the first 5 days of a Proof-of-Concept through the Modernize Traditional Apps program, customers are seeing how they can:

cut application deployment times from days or even weeks to hours or minutes
streamline application maintenance, cutting time spent on upgrades by as much as 99%
consolidate and maximize infrastructure utilization (yes, it works with your existing VMs)
and enable development teams and IT operations teams to work efficiently together to accelerate application delivery

Where IT Begins
If you’re new to IT Starts With Docker, learn the basics of Docker and containers with our online hands-on learning environment. Within a few minutes of following the labs, you will see why so many organizations are now running containers in production. These labs may trigger your own ideas of how Docker might help you in your job: smoother deployment and maintenance of applications, securing apps throughout your software supply chain, and managing apps across disparate infrastructure. Docker can help with all of these and more.
From the first application that gets containerized with Docker, you can show positive returns. We invite you to sign up for updates on this special learning series and see how Docker can make you a budget hero.
To learn more about Docker EE:

Read more about Docker Enterprise Edition 
Try the ROI calculator and download the whitepaper
Visit IT Starts with Docker and sign up for updates
Explore and register for other upcoming webinars or join a local Meetup
Learn everything you need to know about Docker at DockerCon Europe

Four Paths To Become an IT Budget Hero with #Docker #ITStartsWithDockerClick To Tweet

The post Be a Budget Hero with Docker Enterprise Edition appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

What is containerd ?

We have done a few talks in the past on different features of containerd, how it was designed, and some of the problems that we have fixed along the way.  Containerd is used by Docker, Kubernetes CRI, and a few other projects but this is a post for people who may not know what containerd actually does within these platforms.  I would like to do more posts on the featureset and design of containerd in the future but for now, we will start with the basics.
I think the container ecosystem can be confusing at times. Especially with the terminology that we use. Whats this? A runtime. And this? A runtime…  containerd as the name implies, not contain nerd as some would like to troll me with, is a container daemon.  It was originally built as an integration point for OCI runtimes like runc but over the past six months it has added a lot of functionality to bring it up to par with the needs of modern container platforms like Docker and Kubernetes.

Since there is no such thing as Linux containers in the kernelspace, containers are various kernel features tied together, when you are building a large platform or distributed system you want an abstraction layer between your management code and the syscalls and duct tape of features to run a container.  That is where containerd lives.  It provides a client layer of types that platforms can build on top of without ever having to drop down to the kernel level.  It’s so much nicer towork with Container, Task, and Snapshot types than it is to manage calls to clone() or mount().
Containerd was designed to be used by Docker and Kubernetes as well as any other container platform that wants to abstract away syscalls or OS specific functionality to run containers on linux, windows, solaris, or other OSes.  With these users in mind, we wanted to make sure that containerd has only what they need and nothing that they don’t.  Realistically this is impossible but at least that is what we try for.  Things like networking are out of scope for containerd.  The reason for this is, when you are building a distributed system, networking is a very central aspect.  With SDN and service discovery today, networking is way more platform specific than abstracting away netlink calls on linux.  Most of the new overlay networks are route based and require routing tables to be updated each time a new container is created or deleted.  Service discovery, DNS, etc all have to be notified of these changes as well.  It would be a large chunk of code to be able to support all the different network interfaces, hooks, and integration points to support this if we added networking to containerd.  What we did instead is opted for a robust events system inside containerd so that multiple consumers can subscribe to the events that they care about.  We also expose a task API that lets users create a running task, have the ability to add interfaces to the network namespace of the container, and then start the container’s process without the need for complex hooks in various points of a container’s lifecycle.
Another area that has been added to containerd over the past few months is a complete storage and distribution system that supports both OCI and Docker image formats.  You have a complete content addressed storage system across the containerd API that works not only for images but also metadata, checkpoints, and arbitrary data attached to containers.
We also took the time to rethink how “graphdrivers” work.  These are the overlay or block level filesystems that allow images to have layers and you to perform efficient builds.  Graphdrivers were initially written by Solomon and I when we added support for devicemapper.  Docker only supported AUFS at the time so we modeled the graphdrivers after the overlay filesystem.  However, making a block level filesystem such as devicemapper/lvm act like an overlay fillesystem proved to be much harder to do in the long run.  The interfaces had to expand over time to support different features than what we originally thought would be needed.  With containerd, we took a different approach, make overlay filesystems act like a snapshotter instead of vice versa.  This was much easier to do as overlay filesystems provide much more flexibility than snapshotting filesystems like BTRFS, ZFS, and devicemapper as they don’t have a strict parent/child relationship.  This helped us build out a smaller interface for the snapshotters while still fulfilling the requirements needed from things like a builder as well as reduce the amount of code needed, making it much easier to maintain in the long run.
So what do you actually get using containerd?  You get push and pull functionality as well as image management.  You get container lifecycle APIs to create, execute, and manage containers and their tasks. An entire API dedicated to snapshot management.  Basically everything that you need to build a container platform without having to deal with the underlying OS details.  I think the most important part of containerd is having a versioned and stable API that will have bug fixes and security patches backported.

What’s #containerd? All you need to know about #Docker’s open and reliable #container runtime Click To Tweet

Learn more about containerd:

Check out the containerd GitHub Repo
Join the containerd Slack channel
Register for the Moby Summit LA alongside Open Source Summit North America
Register for DockerCon Europe and DockerCon Moby Summit

The post What is containerd ? appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Docker 101: Introduction to Docker webinar recap

Docker is standardizing the way to package applications, making it easier for developers to code and build apps on their laptop or workstation and for IT to manage, secure and deploy into a variety of infrastructure platforms
In last week’s webinar, Docker 101: An Introduction to Docker, we went from describing what a container is, all the way to what a production deployment of Docker looks like, including how large enterprise organizations and world-class universities are leveraging Docker Enterprise Edition (EE)  to modernize their legacy applications and accelerate public cloud adoption.
If you missed the webinar, you can watch the recording here:

We ran out of time to go through everyone’s questions, so here are some of the top questions from the webinar:
­Q: How does Docker get access to platform resources, such as I/O, networking, etc.­ Is it a type of hypervisor?
A: Docker EE is not a type of hypervisor. Hypervisors create virtual hardware: they make one server appear to be many servers but generally know little or nothing about the applications running inside them. Containers are the opposite: they make one OS or one application server appear to be many isolated instances. Containers explicitly must know the OS and application stack but the hardware underneath is less important to the container. In Linux operating systems, the Docker engine is a daemon installed directly in a host operating system kernel that isolates and segregates different procedures for the different containers running on that operating system. The platform resources are accessed by the host operating system and each container gets isolated access to these resources through segregated namespaces and control groups (cgroups). cgroups allow Docker to share available hardware resources to containers and optionally enforce limits and constraints. You can read more about this here.
Q: ­Are containers secure since they run on the same OS?­
Yes, cgroups, namespaces, seccomp profiles and the “secure by default” approach of Docker all contribute to the security of containers. Separate namespaces protects processes running within a container meaning it cannot see, and even less affect, processes running in another container, or in the host system. Cgroups help ensure that each container gets its fair share of memory, CPU, disk I/O; and, more importantly, that a single container cannot bring the system down by exhausting one of those resources. And Docker is designed to limit root access of containers themselves by default, meaning that even if an intruder manages to escalate to root within a container, it will be much harder to do serious damage, or to escalate to the host. These are just some of the many ways Docker is designed to be secure by default. Read more about Docker security and security features here. 
Docker Enterprise Edition includes additional advanced security options including role-based access control (RBAC), image signing to validate image integrity, secrets management, and image scanning to protect images from known vulnerabilities. These advanced capabilities provide an additional layer of security across the entire software supply chain, from developer’s laptop to production.
Q: ­Can a Docker image created under one OS (e.g Windows) be used to run on a different operating system (e.g RedHat 7.x)?
A: Unlike VMs, Docker containers share the OS kernel of the underlying host so containers can go from one Linux OS to another but not from Windows to Linux. So you cannot run a .NET app natively on a Linux machine, but you can run a RHEL-based container on a SUSE-based host because they both leverage the same OS kernel.
Q: Is there another advantage other than DevOps for implementing Docker in enterprise IT infrastructure?
A: Yes! Docker addresses many different IT challenges and aligns well with major IT initiatives including hybrid/multi-cloud, data center and app modernization. Legacy applications are difficult and expensive to maintain. They can be fragile and insecure due to neglect over time while maintaining them consumes a large portion of the overall IT budget. By containerizing these traditional applications, IT organizations save time and money and make these applications more nimble. For example:

Cloud portability: By containerizing applications, they can be easily deployed across different certified platforms without requiring code changes.
Easier application deployment and maintenance: Containers are based on images which are defined in Dockerfiles. This simplifies the dependencies of an application, making them easier to move between dev, test, QA, and production environments and also easier to update and maintain when needed. 62% of customers with Docker EE see a reduction in their mean time to resolution (MTTR).
Cost savings: Moving to containers provides overall increased utilization of available resources which means that customers often see up to 75% improved consolidation of virtual machines or CPU utilization. That frees up more budget to spend on innovation,

To learn more about how IT can benefit from modernizing traditional applications with Docker, check out www.docker.com/MTA.
Q: Can you explain more about how Docker EE can be used to convert apps to microservices?
A: Replacing an existing application with a microservices architecture is often a large undertaking that requires significant investment in application development. Sometimes it is impossible as it requires systems of record that cannot be replaced. What we see many companies do is containerize an entire traditional application as a starting point. They then peel away pieces of the application and convert those to microservices rather than taking on the whole application. This allows the organization to modernize components like the web interface without complete re-architecture, allowing the application to have a modern interface while still accessing legacy data.
­Q: Are there any tools that will help us manage private/corporate images? ­Can we have host our own image repository in-house vs using the cloud?
A: Yes! Docker Trusted Registry (DTR) is a private registry included in Docker Enterprise Edition Standard and Advanced. In addition, DTR provides additional advanced capabilities around security (eg. image signing, image scanning) and access controls (eg. LDAP/AD integration, RBAC). It is intended to be a private registry for you to install either in your data center or in your virtual private cloud environment.
Q: ­Is there any way to access the host OS file system(s)?  I want to put my security scan software in a Docker container but scan the host file system.
A: The best way to do this is to mount the host directory as a volume in the container with “-v /:/root_fs” so that the file system and directory are shared and visible in both places. More information around storage volumes, mounting shared volumes, backup and more are here.

Top 7 questions from #Docker 101 – Webinar recapClick To Tweet

Next Steps:

If you’re an IT professional, join our multi-part learning series: IT Starts with Docker
If you’re a developer, check out the Docker Playground 
Learn more about Docker Enterprise Edition or try the new hosted demo environment
Explore and register for other upcoming webinars or join a local Meetup

The post Docker 101: Introduction to Docker webinar recap appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/