Introduction to Kubernetes: The Ultimate Guide

The post Introduction to Kubernetes: The Ultimate Guide appeared first on Mirantis | Pure Play Open Cloud.
Given the importance of Kubernetes in today’s environment, we wanted to give you a “one stop shop” for any information you may need.
What is Kubernetes?
Simply put, Kubernetes, or K8s, is a container orchestration system. In other words, when you use Kubernetes, a container based application can be deployed, scaled, and managed automatically.

The objective of Kubernetes is to abstract away the complexity of managing a fleet of containers that represent packaged applications and include everything needed to run wherever they’re provisioned. By interacting with the Kubernetes REST API, you can describe the desired state of your application, and Kubernetes does whatever is necessary to make the infrastructure conform. It deploys groups of containers, replicates them, redeploys if some of them fail, and so on.

Because it’s open source, a k8s cluster can run almost anywhere, and the major public cloud providers all provide easy ways to consume this technology. Private clouds based on OpenStack can also run Kubernetes, and bare metal servers can be leveraged as worker nodes for it. So if you describe your application with Kubernetes building blocks, you’ll then be able to deploy it within VMs or bare metal servers, on public or private clouds.

Let’s take a look at the basics of how Kubernetes works so that you will have a solid foundation to dive deeper.
What is a Kubernetes cluster? The Kubernetes architecture
The Kubernetes architecture is relatively simple. You never interact directly with the nodes hosting your application, but only with the control plane, which presents an API and is in charge of scheduling and replicating groups of containers named Pods. Kubectl is the command line interface you can use to interact with the API to share the desired application state or gather detailed information on the infrastructure’s current state.

Let’s look at the various pieces.
Nodes
Each node that hosts part of your distributed application does so by leveraging Docker or a similar container technology, such as Rocket from CoreOS. The nodes also run two additional pieces of software: kube-proxy, which gives access to your running app, and kubelet, which receives commands from the k8s control plane. Nodes can also run flannel, an etcd backed network fabric for containers.
Master
The control plane itself runs the API server (kube-apiserver), the scheduler (kube-scheduler), the controller manager (kube-controller-manager) and etcd, a highly available key-value store for shared configuration and service discovery implementing the Raft consensus Algorithm.

Now let’s look at some of the terminology you might run into.
Terminology
Kubernetes has its own vocabulary which, once you get used to it, gives you some sense of how things are organized. These terms include:

Pods: Pods are a group of one or more containers, their shared storage, and options about how to run them. Each pod gets its own IP address.
Labels: Labels are key/value pairs that Kubernetes attaches to any objects, such as pods, Replication Controllers, Endpoints, and so on.
Annotations: Annotations are key/value pairs used to store arbitrary non-queryable metadata.
Services: Services are an abstraction, defining a logical set of Pods and a policy by which to access them over the network.
Replication Controller: Replication controllers ensure that a specific number of pod replicas are running at any one time.
Secrets: Secrets hold sensitive information such as passwords, TLS certificates, OAuth tokens, and ssh keys.
ConfigMap: ConfigMaps are mechanisms used to inject containers with configuration data while keeping containers agnostic of Kubernetes itself.

Why Kubernetes?
So what is Kubernetes used for?  In order to justify the added complexity that Kubernetes brings, there need to be some benefits. At its core, a cluster manager such as k8s exists to serve developers so they can serve themselves without having to involve the operations team.

Reliability is one of the major benefits of Kubernetes; Google has over 10 years of experience when it comes to infrastructure operations with Borg, their internal container orchestration solution, and they’ve built Kubernetes based on this experience. Kubernetes can be used to prevent failure from impacting the availability or performance of your application, and that’s a great benefit.

Scalability is handled by Kubernetes on different levels. You can add cluster capacity by adding more worker nodes, which can even be automated in many public clouds with autoscaling functionality based on CPU and Memory triggers. The Kubernetes Scheduler includes affinity features to spread your workloads evenly across the infrastructure, maximizing availability. Finally, k8s can autoscale your application using the Pod autoscaler, which can be driven by custom triggers.
Ultimate guide to Kubernetes
Now that you have the basics, we can look at more information. Here at Mirantis we’re committed to making things easy for you to get your work done, so we’ve decided to put together this guide to Kubernetes.

If you have suggestions for topics you’d like us to cover, or links to resources you find particularly valuable, please let us know.
Introduction to Kubernetes
Don’t Be Scared of Kubernetes
Kubernetes has the broadest capabilities of any container orchestrator available today, which adds up to a lot of power and complexity. That can be overwhelming for a lot of people jumping in for the first time – enough to scare people off from getting started. Here are five things you might be afraid of, and 5 ways to get started.
Deploying Kubernetes
Building Your First Certified Kubernetes Cluster On-Premises
Where following entries have shown how to create a basic dev/test cluster, this article explains how to create a production cluster using Docker Enterprise.
How to install Kubernetes with Kubeadm: A quick and dirty guide
Sometimes you just need a Kubernetes cluster, and you don’t want to mess around. This article is a quick and dirty guide to creating a single-node Kubernetes cluster using Kubeadm, a tool the K8s community created to simplify the deployment process.
Multi-node Kubernetes with KDC: A quick and dirty guide
Kubeadm-dind-cluster, or KDC, is a configurable script that enables you to easily create a multi-node cluster on a single machine by deploying Kubernetes nodes as Docker containers (hence the Docker-in-Docker (dind) part of the name) rather than VMs or separate bare metal machines. 
Create and manage an OpenStack-based KaaS child cluster
Once you’ve deployed your KaaS management cluster, you can begin creating actual Kubernetes child clusters. These clusters will use the same cloud provider type as the management cluster, so if you’ve deployed your management nodes on OpenStack, your child cluster will also run on OpenStack.
How to deploy Airship in a Bottle: A quick and dirty guide
Airship is designed to deploy OpenStack, but it deploys it on Kubernetes, so the first thing it does is deploy a Kubernetes cluster, so it’s another option for getting a cluster up and running.
Configuring Kubernetes and components
Virtlet: run VMs as Kubernetes pods
Virtlet enables you to run VMs as first class citizens within Kubernetes; this article explains how and why to make that work.
Everything you ever wanted to know about using etcd with Kubernetes v1.6 (but were afraid to ask)
The etcd key-value store is the only stateful component of the Kubernetes control plane. This makes matters for an administrator simpler, but when etcd went from v2 to v3, it was a headache for operators.
Development
Modeling complex applications with Kubernetes AppController
When you’re first looking at Kubernetes applications, it’s common to see a simple scenario that may include several pieces — but not explicit dependencies. But what happens when you have an application that does include dependencies. For example, what happens if the database must always be configured before the web servers, and so on? It’s common for situations to arise in which resources need to be created in a specific order, which isn’t easily accommodated with today’s templates.
Designing Kubernetes-based applications
How do you build 12-factor apps using Kubernetes?
It’s said that there are 12 factors that define a cloud-native application.  It’s also said that Kubernetes is designed for cloud native computing. So how do you create a 12-factor application using Kubernetes?  Let’s take a look at exactly what twelve factor apps are and how they relate to Kubernetes.
Creating YAML
Introduction to Kustomize, Part 1: Creating a Kubernetes app out of multiple pieces
Kustomize is a tool that lets you create an entire Kubernetes application out of individual pieces — without touching the YAML for the individual components.  For example, you can combine pieces from different sources, keep your customizations — or kustomizations, as the case may be — in source control, and create overlays for specific situations.
Introduction to Kustomize, Part 2: Overriding values with overlays
In part 1 of this tutorial, we looked at how to use Kustomize to combine multiple pieces into a single YAML file that can be deployed to Kubernetes. In doing that, we used the example of combining specs for WordPress and MySQL, automatically adding a common app label. Now we’re going to move on and look at what happens when we need to override some of the existing values that aren’t labels.
Introduction to YAML, Part 1: Creating a Kubernetes deployment
In previous articles, we’ve been talking about how to use Kubernetes to spin up resources. So far, we’ve been working exclusively on the command line, but there’s an easier and more useful way to do it: creating configuration files using YAML. In this article, we’ll look at how YAML works and use it to define first a Kubernetes Pod, and then a Kubernetes Deployment.
Introduction to YAML, Part 2: Kubernetes Services, Ingress, and repeated nodes
In part 1 of this series, we looked at the basics behind YAML and showed you how to create basic Kubernetes objects such as Pods and Deployments using the two basic structures of YAML, Maps and Lists. Now we’re going to look at enhancing your YAML documents with repeated nodes in the context of Kubernetes Services, Endpoints, and Ingress.
Containerize an application
How do I build a containerized app on Mirantis OpenStack with native Docker tools?
In this article, we take a look at what’s really going on behind the scenes of a containerized application by building one on OpenStack using native Docker tools.
Create an application
Best of 2019 Blogs: Designing Your First App in Kubernetes
Kubernetes is a powerful container orchestrator and has been establishing itself as IT architects’ container orchestrator of choice. But Kubernetes’ power comes at a price; jumping into the cockpit of a state-of-the-art jet puts a lot of power under you, but knowing how to actually fly it is not so simple. 
Creating and accessing a Kubernetes cluster on OpenStack, part 3: Run the application
In part 2, you created the actual cluster, so finally, you’re ready to actually interact with the Kubernetes API that you installed. To do that, you’ll need to define the security credentials for accessing your applications, deploy a containerized app to the cluster, and expose the app to the outside world so you can access it.
Docker: (a few) Best Practices
As Docker continues to evolve, it is important to stay up to date with best practices. We joined JFrog to go over the challenges of Dockerization, Dockerfile tips, and configuration tweaks for production.
Multi-container pods and container communication in Kubernetes
Containers are often intended to solve a single, narrowly defined problem, such as a microservice, but in the real world, problems require multiple containers for a complete solution. In this article, we’re going to talk about combining multiple containers into a single Kubernetes Pod, and what it means for inter-container communication.
Deploy an application with Helm
Using Kubernetes Helm to install applications: A quick and dirty guide
Deploying an application using containers can be much easier than trying to manage deployments of a traditional application over different environments, but trying to manage and scale multiple containers manually is much more difficult than orchestrating them using Kubernetes. But even managing Kubernetes applications looks difficult compared to, say, “apt-get install mysql”. Fortunately, the container ecosystem has now evolved to that level of simplicity. Enter Helm.
Infrastructure and operations
Kubernetes Lifecycle Management with Docker Kubernetes Service (DKS)
There are many tutorials and guides available for getting started with Kubernetes. Typically, these detail the key concepts and  outline the steps for deploying your first Kubernetes cluster. However, when organizations want to roll out Kubernetes at scale or in production, the deployment is much more complex and there are new requirements around both the initial setup and configuration and the ongoing management – often referred to as “Day 1 and Day 2 operations.”
We installed an OpenStack cluster with close to 1000 nodes on Kubernetes. Here’s what we found out.
We did a number of tests that looked at deploying close to 1000 OpenStack nodes on a pre-installed Kubernetes cluster as a way of finding out what problems you might run into, and fixing them, if at all possible. In all we found several, and though in general, we were able to fix them, we thought it would still be good to go over the types of things you need to look for.
Scale and Performance Testing of Kubernetes
Managing thousands of containers can be challenging, but if you want to know how Kubernetes will behave at scale we might be able to provide an answer. At KubeCon in Seattle, we shared the data we collected in our scale lab, which consists of 500 physical nodes. Using virtual machines, we can simulate up to 5000 Kubernetes minions running actual workloads, and our tests are designed to reveal how Kubernetes behaves while managing a complex application (in this case, OpenStack services) at large scale.
Kubernetes as an Edge substrate
How to build an edge cloud part 1: Building a simple facial recognition system
If you look at the internet, there’s a lot of talk about edge clouds and what they are — from a conceptual level.  But not too many people are telling you how to actually build one. Today we’re going to start to change that.
Open Source IoT Platform based on OpenStack and Kubernetes
This blog post explains open source IoT platform introduced at OpenStack Summit Keynote at Austin in more detail. First we explain our approach and vision to IoT, technical overview and show two sample use cases.
Scaling your application
Clustered RabbitMQ on Kubernetes
There are a lot of possible approaches to setting up clustered RabbitMQ on Kubernetes. Today I’m going to talk about the most common pitfalls or all approaches to RabbitMQ clustering, so if you want to come up with your own solution, you should find a good bit of the material meaningful to you.
Kubernetes Replication Controller, ReplicaSet and Deployments: Understanding replication options
As a container management tool, Kubernetes was designed to orchestrate multiple containers and replication, and in fact there are currently several ways to do it. In this article, we’ll look at three options: Replication Controllers, ReplicaSets, and Deployments.
Scaling with Kubernetes DaemonSets
We’re used to thinking about scaling from the point of view of a Deployment; we want it to scale up under different conditions, so it looks for appropriate nodes, and puts pods on them. DaemonSets, on the other hand, take a different tack: any time you have a node that belongs to the set, it runs the pods you specify.  For example, you might create a DaemonSet to tell Kubernetes that any time you create a node with the label app=webserver you want it to run Nginx.  Let’s take a look at how that works.
Security
Make your container images safer and more reliable with Harbor, the cloud native registry
Container registries such as DockerHub have made container application development much easier, but they have introduced another problem: how do you know downloaded images are production-ready and secure? To solve this problem, private registries such as Harbor enable your developers to get the benefits of pre-defined images while enabling you to designate what images can be used.
Controlling access to Kubernetes using RBAC
Role-based Access Control for Kubernetes with Docker Enterprise
Docker Enterprise Edition 2.0 provides a single management control plane for both Swarm and Kubernetes-based clusters – including clusters made up of both Swarm and Kubernetes workers. It also provides a web interface enabling you to manage user access to those clusters using RBAC.
Networking
Calico
BGPaaS in OpenStack – Kubernetes with Calico in OpenStack with OpenContrail
It’s been a while since the new version 3.X of OpenContrail was released, so let’s take a good look at new features of this most deployed SDN/NFV with OpenStack, and specifically use cases and how to use BGP as a Service in OpenStack private cloud.
Kubernetes and OpenStack
Kubernetes and OpenStack multi-cloud networking
The use of real bare metal Kubernetes clusters for application workloads from a networking point of view.
Using a service mesh
What is Istio? It’s a service mesh. Great. What’s a service mesh?
Istio has been all over the ecosystem wherever there’s talk about service meshes, but it’s important that we take a look at what all of that means.
Containers aren’t a game: industry gets serious about Kubernetes development with Draft and Istio
As the infrastructure market settles down, more attention is being paid to what happens after you have your cloud up and running. This week, we saw the announcement of not one, but two frameworks aimed at developers of Kubernetes-based applications.  
Spinnaker Shpinnaker and Istio Shmistio to make a shmesh! (Part 1)
I’m guessing that whenever your manager approaches you and says “We have a problem,” you sort of know that it really means “I have a problem for you to solve.” Such is often the case with our customers, who are frequently attempting to move from a cascading (waterfall) style of delivering application services on bare metal to a more modern way of approaching continuous delivery geared toward cloud native applications.
Return of the Smesh (Spinnaker Shpinnaker and Istio Shmistio to make a Smesh! Part 2)
One of the first things I learned on my sojourn through the open source world is that there are ALWAYS new and different approaches to building the better mouse trap when it comes to component design within a given architecture, and that a single project doesn’t usually contain all of the answers to questions created when developing new application architectures.
OpenDaylight
What’s in OpenDaylight?
The momentum to recognize the OpenDaylight Project as the standard open source software-defined networking (SDN) continues to grow. Established to accelerate the adoption of SDN and Network Functions Virtualization (NFV), OpenDaylight provides an open platform for network programmability designed to enable SDN and create a solid NFV foundation for all sizes of networks.

What are the resources you’ve found to be most helpful? Let us know in the comments!The post Introduction to Kubernetes: The Ultimate Guide appeared first on Mirantis | Pure Play Open Cloud.
Quelle: Mirantis

Understanding Inner Loop Development and Pull Rates

We have heard feedback that given the changes Docker introduced relating to network egress and the number of pulls for free users, that there are questions around the best way to use Docker as part of your development workflow without hitting these limits. This blog post covers best practices that improve your experience and uses a sensible consumption of Docker which will mitigate the risk of hitting these limits and how to increase the limits depending on your use case. 

If you are interested in how these limits are addressed in a CI/CD pipeline, please have a look at our post: Best Practices for using Docker Hub for CI/CD. If you are using Github Action, have a look at our Docker Github Actions post.

Prerequisites

To complete this tutorial, you will need the following:

Free Docker Account You can sign-up for a free Docker account and receive free unlimited public repositoriesDocker running locallyInstructions to download and install DockerAn IDE or text editor to use for editing files. I would recommend VSCode

Determining Number of Pulls

Docker defines pull rate limits as the number of manifest requests to Docker Hub. Rate limits for Docker pulls are based on the account type of the user requesting the image – not the account type of the image’s owner. For anonymous (unauthenticated) users, pull rates are limited based on the individual IP address. 

We’ve been getting questions from customers and the community regarding container image layers. We are not counting image layers as part of the pull rate limits. Because we are limiting on manifest requests, the number of layers (blob requests) related to a pull is unlimited at this time

As an anonymous user, you are able to perform up to 100 pulls within a six hour window. This is a high enough limit to allow individual developers to build their images on their local development machine without worry of reaching the pull limits.

If you need to perform more than 100 pulls per six hour window, create a free Docker account which will allow you to perform up to 200 pulls per six hour window. Doubling the amount of pulls compared to the anonymous limit.

To get a good idea of how many pulls a build will incur, you can take a look at the number of FROM commands in your Docker file. Once an image has been pulled to your local machine, it will not incur a pull on subsequent builds.

So for example, if we had an application that was made up of a frontend UI, a REST service and a database. We would have two Dockerfiles. One for building the UI and one for building the REST service. We would  then combine these images with our database inside of a compose file. Inside of this compose file, we would include our database image.

Let’s take a look at this scenario using the react-express-mongodb example in the Awesome Compose repository. Clone the awesome-compose repository and open the react-express-mongodb folder in your favorite editor.

$ git clone git@github.com:docker/awesome-compose.git

$ cd awesome-compose/react-express-mongodb

Expand the frontend fold and open the Dockerfile.

As you can see on line 1, we are using the node:lts-buster-slim image. If we do not already have this image locally then when we perform a build, this image will be pulled from Docker Hub and count as one pull.

Likewise in the backend folder, we see a Dockerfile that is used to build the backend image. On line 1 of this file, we are also using the node:lts-buster-slim image. Again, if you have not already pulled this image from Docker Hub, when you run a docker build, then Docker will pull this image and count it as one pull.

To recap, since we are using the same base image (node:lts-buster-slim) for each of our application images, we will only have to pull that image once and therefore only incur one pull.

The same is true for the mongo:4.2.0 image. When you run the docker-compose up command, the mongo image will be pulled, if not present locally, and increase the pull limit counter by one.

So in the above example, with zero images present locally, you will incur two pull requests to Hub. Even if we expanded this out to a slightly more complex architecture with a few more services that are also written in node, we would still only incur two pull requests. One for the node image and one for the mongodb image.

Now let’s take a look at a more advanced build scenario below.

Here is an example of a Dockerfile that uses multi-stage builds:

1 # syntax=docker/dockerfile:1.1.72 ARG GO_VERSION=1.13.7-buster34 FROM golang:${GO_VERSION} AS golang56 FROM golang AS build7 ….8 FROM debian:buster AS foo9 ….10 FROM scratch AS final11 COPY –from=build /bin/foo /bin/foo

Simply counting the number of FROM‘s will not work in all situations but it is a good general proxy. If I have a FROM command that references an image that I only have locally, then a Hub pull will not occur. We can also use  FROM commands in a multi-stage build to reference other build-stages located in the same Dockerfile. 

In the above Dockerfile, we have multiple FROM statements of which the total comprises a multi-stage build. What will actually be pulled from Hub depends on the state of your local cache, which build target is set and whether or not you’re using BuildKit.

Let’s walk through a scenario where we are not using BuildKit.

On line 4 we can see that we are referencing the GO_VERSION build argument:

FROM golang:${GO_VERSION} AS golang

The value of GO_VERSION is dependent on whether we have passed a value using the –build-arg option or not. 

So, for example, let’s say we have the golang:1.13.7-buster image on our local machine. If we do not override the GO_VERSION then we will not incur a pull. On the other hand, if we set the GO_VERSION to 1.15.2-buster and do not have this image locally, then we will incur a pull from Hub.

Another point to keep in mind when counting FROMs, is that a FROM command can reference the scratch image. The scratch image is not an image on Hub but is treated specially by Docker and never pulls anything from Hub but is used as a starting point for creating an empty image.

Now let’s take a look at building an image using BuildKit. We’ll use the same sample Dockerfile from above.

When we run a build, the FROM scratch AS final stage is started which will trigger the following flow:

FROM golang AS build is startedWhich triggers the FROM golang:$(GO_VERSION} AS golang stageWhich will pull the golang:1.13.7-buster image from Hub if it is not present locally

In this scenario, the FROM debian:buster AS foo stage on line 8 is not used in the final image and therefore will not be built and the debian:buster image will not be pulled from Hub. Even though it is a FROM statement in the Dockerfile, it is not used and need not be counted when figuring out the number of pulls that will occur.

Unlimited Pulls

If you are working on a larger project that has a lot of different base images or you are building images and removing them often, then the best option is to purchase a Docker Pro Account or a Docker Team Account.

Both the Pro and Team accounts give you unlimited pulls and therefore not subject to rate limiting. You also receive unlimited private repositories with these plans.

Conclusion

In this article we discussed how pulls are counted when building images using Docker. We first talked about a common application that has a frontend, backend and a datastore and how this scenario will not reach the 100 pull limit for anonymous users. Then we discussed a more advanced Dockerfile that uses multi-stage builds and how this can potentially affect your pull limits. Although not enough to reach the 200 pull limit for authenticated accounts. 

For more information and common questions, please read our FAQ. As always, please feel free to reach out to us on Twitter (@docker) or to me directly (@pmckee).

To get started using Docker sign up for a free Docker account and take a look at our getting started guide.
The post Understanding Inner Loop Development and Pull Rates appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

How rapid integrated clinical data helps prioritize care for a vulnerable population

Editor’s note: When the COVID-19 pandemic arrived in the U.S. earlier this year, healthcare organizations had to quickly change or speed up their plans to serve patients. Commonwealth Care Alliance (CCA) used data analytics from Google Cloud to connect clinicians and care managers with high-risk members. Valmeek Kudesia, vice president of clinical informatics and advanced analytics at CCA, shares their story. CCA is a community-based healthcare organization nationally recognized as a leader in providing and coordinating care for high-cost, high-need individuals who are dually eligible for Medicaid and Medicare. CCA integrates the roles of healthcare payor, care management organization, and direct care provider to serve these individuals (known as “members”). Our members live with medical, behavioral health, and social challenges. Many have complicated lives and almost all are vulnerable or marginalized. When news of COVID-19 arrived here in the U.S. last winter, we knew our members would need enhanced levels of care and attention. We needed to deliver on our mission of serving our members while accounting for many new and rapidly changing factors. Our dedicated staff and clinicians needed reliable data that was available quickly and integrated across many domains and sources. We had already put in place an advanced analytics platform with Google Cloud’s BigQuery and Looker. Six months later, we’ve established trust in our data and provide our clinicians with a more holistic view of members’ needs—extending our ability to provide compassion and care to our members, i.e., human-centered use of data and analytics. We will continue to lean upon our data to best serve our members during the upcoming combination of COVID-19 and influenza.Related ArticleData warehouse migration tips: preparation and discoveryIn this post, we’ll explore a few important questions that come up during the initial preparation and discovery phase for data warehouse …Read ArticleThe data required to make decisions fasterThe platform we’d chosen was built for a situation where many users must move fast and in different directions and quickly change what data or information they need. This meant we didn’t have to pivot when COVID-19 arrived. Our data science team used Looker and BigQuery in combination with other technologies (see figure below) to develop and deploy our data ops and ML ops capabilities. All of Google Cloud was available under a single business associate agreement (BAA) to meet HIPAA requirements and BigQuery was (and still is) elastic and available as a service. These two features made it easier for our small data science team to stay focused and move fast, while remaining compliant and enjoy great platform performance.As we used query abstraction and a columnar-based data engine, we discovered an advantage during our response to the COVID-19 pandemic. We had a rapidly increasing number of ways to describe individual “things” combined with very rapid cycles to define “what mattered” at a moment of the day or week. We took advantage of the columnar format to pre-answer as many questions as possible and used straightforward query virtualization to quickly iterate on the idea of what clinicians wanted to see at the speed that they needed to see new or different data. Consequently, we could “move with” clinicians and provide data and predictive insights via general dashboards and role-specific dashboards called action boards. “Action boards” go beyond dashboards—they offer the information needed to help decide which action a clinician will take next, depending on the specific needs of each member. Incorporating daily or even hourly data from various sources was essential to ensure our members got what they needed, whether home-delivered meals, medications, or another service. In some cases, we already had all the required data. For example, in less than 30 minutes, we implemented the CDC’s definition of high-risk for COVID-19 complications in LookML (Looker’s query abstraction layer) and linked the concept “high-risk COVID-19 complications” into our information model. Within a day, we built our main COVID-19 monitoring dashboards and infused relevant pandemic data into other clinical dashboards and action boards. The combination of concept flexibility via abstraction and wide delivery allowed us to quickly identify every member at high risk of a poor COVID-19 outcome and provide that knowledge to their assigned CCA clinicians and integrated care team.Some of the data we needed wasn’t easily accessible—for example, at the time there wasn’t a repository of COVID-19 data or data services on GitHub. But it was essential that we incorporated all needed data to serve all our members—a data-omnivorous approach. In many cases, we had to gather the data ourselves. For example, in the early stages of the COVID-19 pandemic in Massachusetts, adult day health (ADH) centers, community centers that provide critical services for seniors, were closing one-by-one and then abruptly closed en masse. We were able to distribute that knowledge to each member’s assigned CCA clinician action board within minutes of learning of new ADH closings. At a later stage of the COVID-19 pandemic, we started to ingest Massachusetts Department of Public Health data describing test positivity rates to provide insights describing the concentration of high-risk members living in areas with high or escalating COVID-19 risk. The journey from “just data” to “essential for care and support” As the COVID-19 pandemic and its effects continue to evolve, we continually use the latest available information to update and guide our member support and care strategies. Our teams have become more comfortable using data. Typically, we have more than 450 weekly active users and, on average, someone looks at data nearly once every second during the workday. The questions have gotten more interesting and we’re able to ask more about how a piece of data fits, or doesn’t fit, into the member’s bigger picture. Instead of thinking of data as a separate factor, it fades away and becomes part of the surrounding context in which we see the person who is the member—just like how we don’t think specifically about GPS anymore, but rather we focus on the destination. Now the use of data is really built into the way we care for members–it’s part of daily routines. With this kind of reliability, the data science team has moved on to deeper feature engineering and causal inference to enrich the insights delivered to our staff and clinicians. Additionally, our staff and clinicians have come to expect that data can help them take care of vulnerable people. Retrospective for data-informed decisionsThe road to data-informed decision making involves time to build trust in the system. COVID-19 helped solidify that trust and now our clinicians expect data to help toward the care of vulnerable members. We learned that you must build up the ability to quickly iterate to get the data and platform good enough (not perfect) to start being meaningful —and the technology has to allow for that iteration speed. As humans, we see what we know and when people don’t see the big picture or don’t have all the information, they may make near-sighted or counterproductive choices. They can fall into habits or heuristics that reinforce existing problems.Now we’re dealing with human-to-machine interactions as well as human-to-data interactions. When we get it right, we start to incorporate data into our decision making without adding another step. Eventually, it’s just a natural part of the decision process—in this case to take care of a person. For example, evaluation of chest pain frequently requires an electrocardiogram (ECG) interpretation, which encapsulates a large amount of data, and clinicians expect an ECG to help them take better care of the person vs. focusing on the data itself. The COVID-19 pandemic showed us that data used in the right way can extend the human touch and become a therapeutic ally.  You can learn more about CCA in our JOIN@Home session and the full case study.
Quelle: Google Cloud Platform

Scale in at your own pace with Compute Engine autoscaler controls

In a Compute Engine environment, managed instance groups (MIGs) offer autoscaling that lets you automatically change an instance group’s capacity based on current load, so you can rightsize your environment—and your costs. Autoscaling adds more virtual machines (VMs) to a MIG when there is more load (scaling out), and deletes VMs when the need for VMs is lesser (scaling in).When load declines, the autoscaler removes all unused capacity. This allows you to save costs but might cause the autoscaler to scale in abruptly. For example, if the load goes down by 50%, the autoscaler removes ~50% of your VMs immediately after a 10-minute stabilization period. Deleting VMs so abruptly might not work well for some workloads. For example, if your VMs take many minutes to initialize you might want to slow down the rate at which you scale in to maintain capacity for imminent load spikes.Introducing scale-in controlsNew scale-in controls in Compute Engine let you limit the VM deletion rate by preventing the autoscaler from reducing a MIG’s size by more VM instances than your workload can tolerate to lose.When you configure autoscaler scale-in controls, you control the speed at which you scale in. The autoscaler never scales in faster than your configured rate, as shown in this diagram:When load declines, the autoscaler maintains the size for the group at a level required to serve the peak load observed in the last 10 minutes (the stabilization period). This works the same with and without scale-in controls.An autoscaler without scale-in controls keeps only enough instances required to handle the recently observed load. After the stabilization period, the autoscaler removes all unneeded instances in one step. A sudden drop in load can lead to a dramatic reduction of instance group size.An autoscaler with scale-in controls limits how many VM instances can be removed in a given period of time (here 10 VMs in 20 minutes). This slows down the instance reduction rate.With the load spikes again, the autoscaler adds new instances. However, because VMs take a long time to initialize, the new VMs aren’t ready to serve the load. With scale-in controls, the previous capacity wasn’t deleted yet, allowing existing VMs to serve the spike.Getting startedYou can set up scale-in controls in the Google Cloud Console. Select an autoscaled MIG from the instance groups page and click Edit group. Under the Autoscaling section, set your scale-in controls:You can also configure scale-in controls programmatically. Here’s the same command written using CLI (gcloud):For more details including configuration using API refer to the documentation.Try scale-in controls todayScale-in controls are generally available across all regions. For more information on how you can control the scale-in rate, check out the docs. Related Article5 best practices for Compute Engine cost optimizationBest practices for saving money on Google Compute EngineRead Article
Quelle: Google Cloud Platform