How Lumiata democratizes AI in healthcare with Google Cloud

Editor’s note: Today’s guest post comes from AI for healthcare platform Lumiata. Here’s the story of how they use Google Cloud to power their platform—performing data prepping, model building, and deployment to tackle inherent challenges in healthcare organizations. If ever there was a year for healthcare innovation—2020 was it. At Lumiata, we’ve been on a mission to deliver smarter, more cost-effective healthcare since 2013, but the COVID-19 pandemic added new urgency to our vision of making artificial intelligence (AI) easy and accessible. Using AI in healthcare went from a nice-to-have to a must-have for healthcare organizations. Just imagine how differently you could plan or assess risk if you could identify communities that are more likely to develop long-term effects or co-morbidities and end up in the hospital. Our Lumiata AI Platform helps healthcare organizations use AI to improve quality of care, minimize risk, and reduce costs, without having to build those predictive analytics capabilities themselves. We’re making it possible for healthcare organizations to benefit from the latest technologies to derive valuable insights from their data at scale—and ultimately offer better care to their patients. As an AI-first organization, we’ve been able to try new things to meet our customers’ changing needs, and our Google Cloud infrastructure lets us experiment and develop solutions quickly (check out our previous post on why we chose Google Cloud to help us deliver AI). What our customers need: fast, accessible AIAI provides an enormous opportunity for healthcare organizations, with an almost unlimited number of practical applications. AI isn’t just a solution you can switch on—it requires implementing the right, often purpose-built, solutions to extract the right insights from your data. AI in healthcare is the next frontier, but the transformation can be slow. Without Lumiata’s help, many organizations struggle to operationalize AI, from data prepping to model building and deployment, even if they have identified the problems they would like to solve. Having advanced data science teams isn’t enough—you need to establish a fast, flexible, and resilient infrastructure to deploy projects. Healthcare AI projects are often plagued by a lack of understanding of the complexity of the high-dimensional data and what it takes to simplify it, as well as what’s required from engineering to productionize AI. In addition, it can be difficult to get the appropriate buy-in to make the changes needed to be successful.In addition, healthcare organizations are building technology on waterfall methodologies, which lack the feedback loops and continual improvement to deliver the promised results. Without fast proof that AI is worth the investment, many projects fail before they’ve even started.This is where Lumiata comes in. Our goal is to get customers up and running with the ability to perform fast queries and accurate, AI- and ML-driven predictions in a few weeks. The wealth of healthcare data is ripe for generating AI-powered insights and predictions, but it’s often trapped in legacy systems. Also, many organizations simply don’t have the resources to build everything themselves. We provide predictive products to healthcare businesses looking to leverage machine learning without the heavy lift by offering low- to no-code data modeling tools and solutions—all based on Google Cloud. That way, organizations are empowered to get started and run models when they don’t necessarily have the team do it themselves. We selected Google Cloud because of its security infrastructure, intuitive AI tools, and multi-cloud application management technologies. BigQuery, Google’s serverless data warehouse, enables us to provide access to huge amounts of data. With Google Cloud Dataflow and Apache Beam, we built a data ingestion and extraction process to join and normalize disparate patient records and datasets. The entire system is built on a Google Kubernetes Engine, allowing us to scale quickly to meet infrastructure requirements, and Kubeflow helps us develop and deliver our machine learning pipelines. Additionally, Google Cloud’s fully managed services mean we don’t have to think about building and operating our infrastructure. Instead, we invest our resources in doing more work for our customers and addressing their data needs. Let’s take a look at how Google Cloud helps us deliver AI solutions to our customers through the steps of a typical ML building process.1. Data prepping—from raw input to a 360-degree viewHealthcare organizations often suffer from information data silos that lack interoperability, so there’s no true understanding of the total amount of data and insights available. Most companies rarely have a comprehensive longitudinal person record (LPR) into every person’s health history. When it comes to machine learning, cleaning and preparing data is where teams spend the majority of their time. It’s slow and incredibly time-consuming. In addition, working with on-premises environments doesn’t provide enough elasticity–you need to move quickly, and only the cloud has the capacity to support data prepping for AI. We’ve created a data preparation process that takes raw, messy data and transforms it into fully prepped data for machine learning. Our data management pipeline ingests raw, disparate patient datasets and turns them into what we call Lumiata Person360 records. Using BigQuery and Dataflow, we ingest raw data dumps, link with existing or synthesized identifiers, validate, clean, and normalize it based on medications, procedures, diagnosis codes, and lab results. The data is then tied into a single person record and tagged with proprietary disease codes. Our automated pipeline gives us incredible speed to intake data, and Google Cloud ensures we can scale to handle massive datasets seamlessly. For instance, we’ve been able to take 63 million person records (roughly 2.5 terabytes of data) and run them through our entire data management pipeline in less than four hours. As healthcare organizations handle protected health information and must ensure Health Insurance Portability and Accountability (HIPAA) compliance, it’s imperative that we have the highest level of security and compliance at all times. To ensure this, we deploy single-tenant instances of the entire platform as its own Google Cloud Platform project with its own Kubernetes, networking, buckets, BigQuery tables, and services. 2. Removing the burden of training data modelsOne of the biggest challenges with model building is developing the infrastructure that enables transparent access to various data sources. Infrastructure setup takes time and resources, and often creates complexity when determining how to manage diverse data representations, architecture, and data quality. This is compounded by the fact that ML pipelines must continuously scale as the data for analysis increases. Ultimately, we don’t want our customers to have to worry about the underlying infrastructure. We use Kubernetes and Kubeflow to build scalable ML pipelines and deep learning architectures that can support massive datasets. Our platform unlocks millions of input variables (machine learning features) from Person360 patient records and mixes them with our own internal 110 million-member data asset. We then use this data for training our complex data models to predict cost, risk, disease onset, and medical events.Google’s AI Platform also makes it easier for us to experiment faster with large training datasets from our 120 million records. For instance, we have shifted from more traditional machine learning (like gradient boosted decision trees) toward larger deep learning models that can take multiple predictions across more than 140 medical conditions and analyze them across a specific time dimension. The real value here is the speed at which we can service our customers from the time they drop their data into our platform to the first datasets. Our automated machine learning pipeline enables us to reduce the time it takes to deliver the first outputs—from months to weeks. For instance, we can now train our models with feature matrices containing 11 million people in less than two hours—and all without having to waste time setting up infrastructure for distributed training. 3. Deploying and serving models in productionProductizing complex ML models comes with its own host of challenges. After models are trained and ready for deployment, it can be difficult to maintain consistency as you scale model deployments to meet new use cases or requirements across the organization. Our data science and machine learning engineering teams run offline experiments (outside of Kubeflow) using the Google AI Platform, allowing a single team member to run numerous experiments a day. Once we have a model that works, we version the training pipeline, pre-trained models, and inference pipelines before deploying it onto Kubeflow. The Lumiata AI Platform allows us to benefit from serverless and distributed training—our data scientists are training more models per week, and we have made quicker leaps forward using BERT-based deep learning models. Building on top of Kubernetes and Kubeflow gives us a fast, scalable path to deploy and serve models to our customers. Kubeflow’s reusable components allow us to scale without having to build from scratch. There’s no need to worry about the nuances of training, tuning, and deploying models—customers can simply upload their data and get predictions out on the other end. Running ML and AI in productionThe real impact of simplifying AI implementation is that it opens up previously undiscovered paths for improvement. For instance, we recently launched Pharmacy Intelligence, an AI-powered intervention tool that leverages pharmacy visit data to improve chronic disease management. We partnered up with FGC Health, a Canadian retail pharmacy chain, to help them identify diabetic patients at risk for cardiovascular complications who have gaps in care. The tool will then recommend a simple, actionable intervention, such as a visit to a specialist, drug titration, or adjustments to their existing drug regimen. This is a wonderful example of how using AI to address common gaps in care has the power to save lives. As a company, we see Google Cloud as the core of our platform, enabling us to innovate more rapidly. As a result, we’re delivering solutions for new and interesting problems, such as claims payment integrity, predicting hospital admission and readmission, and identifying disease progression fingerprints for more personalized care. We’re helping healthcare companies become smarter, more powerful, and more effective—leveraging the information they already have in new ways to power the next generation of patient care.Related ArticleRoad to recovery: How Google Cloud is helping states get the COVID-19 vaccine to more peopleGoogle Cloud is proud to partner with a number of states across the U.S. to support vaccination efforts at scale.Read Article
Quelle: Google Cloud Platform

Four consecutive years of 100% renewable energy—and what’s next

We’re proud to announce that in 2020 Google again matched 100 percent of its global electricity use with purchases of renewable energy. We were the first company of our size to achieve this milestone back in 2017, and we’ve repeated the accomplishment in every year since. All told, we’ve signed agreements to buy power from more than 50 renewable energy projects, with a combined capacity of 5.5 gigawatts – about the same as a million solar rooftops.Achieving 100 percent renewable energy year after year is no easy feat, because the amount of computing done in Google data centers continues to grow. This was especially true in 2020 – a year when many peoples’ work, school, doctor’s appointments, first dates, and visits with loved ones moved online. Even as Google Meet and Duo hosted over a trillion minutes of video calls in 2020, our renewable energy procurement kept pace.The path to 100% starts with reducing the amount of energy we use in the first place. Researchers recently found that worldwide data center electricity stayed close to flat in the last decade, even as computing needs grew 550 percent. And Google has led this trend: compared with five years ago, we now deliver around seven times as much computing power with the same amount of electrical power.Last year’s accomplishment was also due to a global package of renewable energy deals that we announced in late 2019. As those projects came online over the course of 2020, hundreds of new turbines and hundreds of thousands of new solar panels began converting wind and sun into electrons.Our renewable energy projects that started operations in 2020 spanned four continents. Some highlights we’re especially excited about:  Google’s first offshore wind project, in the blustery North Sea, began contributing electrons to the grid where we operate our Belgium data center.In Chile, we began purchasing power  from a new solar farm in the Antofagasta region  to match our growing load in South America. Solar panels distributed across hundreds of public housing rooftops helped us source new clean energy in land-constrained Singapore.Across the U.S., large-scale solar and wind projects gave a boost to data centers from Oklahoma to Alabama to Virginia. So what’s next? Though we’re thrilled to have matched Google’s annual electricity consumption with renewable energy for four years running, we’re now building on our progress to target an even larger ambition: by 2030, Google aims to run on entirely 24/7 carbon-free energy, everywhere we operate. As we discuss in a new explainer, achieving this goal means shifting away from a net-zero model of “emit and compensate” and instead targeting “absolute zero,” where we simply never emit carbon from our operations in the first place. Solving this challenge is not only important for Google, but will also be essential for fully transitioning electric grids to carbon-free energy. We hope our efforts to develop solutions for our own operations can lead the way. For more, check out Google CEO Sundar Pichai’s 2021 Earth Day update on our progress.
Quelle: Google Cloud Platform

LinuxKit as a Commodity for Building Linux Distributions

Guest post by Docker Captain Gianluca Arbezzano

Recently Corey Quinn from LastWeekInAWS wrote an article that made me think “Nobody Cares About the Operating System Anymore”. Please have a look at it! I like the idea that nobody cares about where their application runs. Developers only want them running.

A bit of context about Tinkerbell

I am one of the maintainers for the Tinkerbell project. A bare metal workflows engine that heavily relies on containers and Docker to get its work done. It tries to find an answer for a reasonable question: how do we manage rooms of pieces of hardware? More in practice, how can we bring an API on top of everybody’s data centers?

Containers are the abstraction we decided to use when running reusable code (that we call actions) in somebody else’s hardware. Mainly because distribution, packaging, and runtime are solved issues. Everyone knows how to build, push and run a container.

I think this scenario compares well with the story Corey highlighted. Operating systems are an established, well-known abstraction for the majority of the use cases.

The special operating system for bare metal provisioning

The lifecycle of a bare metal server can be summarised as follows:

You mount SSDs, RAMs and you rack your server, cabling it with power and dataIf you have a BMC, you can remotely start the server. Otherwise, you have to push the power button manuallyThe BIOS looks for bootable devices, USBs, disks, but it can’t find anything to boot because the disk driver is not something that server has nowadays, and there are not operators running with USB sticks in modern datacentersUsually, the last chance a server has to boot when nothing works is via netbooting. A popular technology for that is called PXE (the new one iPXE).PXE makes a DHCP request; you can imagine it as the last SOS: “PLEASE tell me what to do.”If there is a DHCP listening, it can replay with a rescue message to simplify things with a script that PXE can execute.

Usually, the script contains boot information for an operating system, a Linux operating system that can run on RAM. For example, this is how you can Netboot Alpine.

Now the hardware has an ephemeral (on RAM) fully operational operating system. Tinkerbell distributes two of them, one called OSIE and the second one called Hook:

OSIE is the one ran by Equinix Metal internally to provision their entire cloud offerHook is a more recent one the Tinkerbell community develop using LinuxKit

OSIE or Hooks are essential for bare metal provisioning because they are the source of power for what you can do on the hardware itself. Tinkerbell starts a docker daemon, and it downloads and executes a set of actions (as you read Docker containers).

The actions all together build workflows that looks like:

Provisioning: flash and install the end is the operating system on the disk, so next boot, you can access Ubuntu, CentOS, CoreOS, or what you needDeprovision: you can wipe disks and make the server available for a next brand new use

This article is about why we choose LinuxKit, and I hope it will give you more information about when you should think about using it as well!

Nobody cares about the operating system, but you can not avoid one

As you can imagine reading briefly about Tinkerbell when it comes to bare metal provisioning, every bit counts because the hardware lifecycle is cold and not that fast. Stay in control of every step is crucial, from when the server power to when it makes the DHCP request when the hardware boots the in-memory operating system until you finally get what you want executed!

When it comes to operating systems and Linux maintaining a distribution is a lot of work! Even if it dedicated to a specific use case like the one we have to Tinkerbell (it is just a temporary execution environment that relays on Docker) we still have to take care of:

Compatibilities: there are many hardware devices, drivers, kernel modules, architecturesSize: the operating system runs on RAM, yeah it is not that expensive nowadays, but still, we have to be carefulNeeds: We can not assume that all the environments where Tinkerbell runs are the same. For example somebody will make like the idea to run an SSH server as part of the in memory environment because their server do not have a serial console and SSH is a good option for troubleshooting. Or they want to run agents for service discovery like Consul, or for monitoring like Telegraf to improve observability and monitoring. Or some scanner for security purposes.

We can’t make one that works for everything and everybody. That’s why with Hook we decided to adopt LinuxKit.

LinuxKit is now part of the Linux Foundation, initially developed by Docker specifically to release Docker for Mac. You can think about it as a Linux builder focused on containers.

You can add a program on boot as init, or as long-running services. The cool part is that everything runs as a container giving us the ability to package building blocks as Docker containers, leaving a clear path from end-user to build their environment, based on their needs reusing LinuxKit itself (if they want) and the building blocks we developed.

One of the building blocks I am referring to is a Docker container who overrides the logic used by LinuxKit to start the docker daemon. Along that it also starts what we call tink-worker. An agent who reaches out to the tink server obtaining the next workflow to run. You can think about them as api-server and kubelet for Kubernetes but instead of running pods tink-worker reaches out to the Docker daemon running actions such as:

streaming a new operating system to a particular diskformatting or partitioning a diskExecuting commands with a different chrootBut it can be every container, you can even run an action that notifies you on slack when your workflows reaches a certain point

LinuxKit provides facilities for multi-architecture and output format as well. We are working for ARM support, for example.

Being part of something bigger than ourselves

LinuxKit has a supportive community with docs, examples and even a Slack channel. End users of Tinkebell can make use of hundreds of people and maintainers dedicated to only building distros with LinuxKit. This makes the effort of maintaining Tinkerbell scoped to a reasonable size. Allowing us to stay strict to what matters most. Provisioning bare metal quickly.
The post LinuxKit as a Commodity for Building Linux Distributions appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Neue Datasets im Registry of Open Data on AWS von United States Geological Survey (USGS), dem Schweizer Institut für Bioinformatik, iNaturalist.org und anderen Organisationen verfügbar

Achtundzwanzig neue oder aktualisierte Datasets von der US Geological Survey, dem Schweizer Institut für Bioinformatik, iNaturalist.org, der National Oceanic and Atmospheric Agency (NOAA) und anderen Organisationen sind im Registry of Open Data auf Amazon Web Services (AWS) in den folgenden Kategorien verfügbar.
Quelle: aws.amazon.com