Kategorie: Microsoft Azure

Bringing confidential computing to Kubernetes

Historically, data has been protected at rest through encryption in data stores, and in transit using network technologies, however as soon as that data is processed in the CPU of a computer it is decrypted and in plain text. New confidential computing technologies are game changing as they provide data protection, even when the code is running on the CPU, with secure hardware enclaves. Today, we are announcing that we are bringing confidential computing to Kubernetes workloads.

Confidential computing with Azure

Azure is the first major cloud platform to support confidential computing building on Intel® Software Guard Extensions (Intel SGX). Last year, we announced the preview of the DC-series of virtual machines that run on Intel® Xeon® processors and are confidential computing ready.

This confidential computing capability also provides an additional layer of protection even from potentially malicious insiders at a cloud provider, reduces the chances of data leaks and may help address some regulatory compliance needs.

Confidential computing enables several previously not possible use-cases. Customers in regulated industries can now collaborate together using sensitive partner or customers data to detect fraud scenarios without giving the other party visibility into that data. In another example customers can perform mission critical payment processing in secure enclaves.

How it works for Kubernetes

With confidential computing for Kubernetes, customers can now get this additional layer of data protection for their Kubernetes workloads with the code running on the CPU with secure hardware enclaves. Use the open enclave SDK for confidential computing in code. Create a Kubernetes cluster on hardware that supports Intel SGX, such as the DC-series virtual machines running Ubuntu 16.04 or Ubuntu 18.04 and install the confidential computing device plugin into those virtual machines. The device plugin (running as a DaemonSet) surfaces the usage of the Encrypted Page Cache (EPC) RAM as a schedulable resource for Kubernetes. Kubernetes users can then schedule pods and containers that use the Open Enclave SDK onto hardware which supports Trusted Execution Environments (TEE).

The following pod specification demonstrates how you would schedule a pod to have access to a TEE by defining a limit on the specific EPC memory that is advertised to the Kubernetes scheduler by the device plugin available in preview.

Now the pods in these clusters can run containers using secure enclaves and take advantage of confidential computing. There is no additional fee for running Kubernetes containers on top of the base DC-series cost.

The Open Enclave SDK was recently open sourced by Microsoft and made available to the Confidential Computing Consortium, under the Linux Foundation for standardization to create a single uniform API to use with a variety of hardware components and software runtimes across the industry landscape.

Try out confidential computing for Kubernetes with Azure today. Let us know what you think in our survey or on GitHub.
Quelle: Azure

Microsoft cloud in Norway opens with availability of Microsoft Azure

Today, we’re announcing the availability of Microsoft Azure from our new cloud datacenter regions in Norway, marking a major milestone as the first global cloud provider to deliver enterprise-grade services in country. These new regions demonstrate our ongoing investment to help enable digital transformation and advance intelligent cloud and intelligent edge computing technologies across both commercial and public sectors.

DNB, Equinor, Lånekassen, and Posten are just a few of the customers and partners leveraging our cloud services to accelerate innovation and increase computing resources. This new offering of Microsoft Azure delivers scalable, highly available, and resilient cloud services to Norwegian companies and organizations while meeting data residency, security, and compliance needs.

Our President, Brad Smith, recently visited Norway to celebrate this important launch and to discuss how vital trust is for those we serve, not only to help bring forth innovation but to ensure our customers are protected.

“Our customers have entrusted us to protect, operate, and develop our platform in a way that keeps their data private and secure. This is an immense responsibility that we can’t just claim, but a responsibility that we must earn every single day.” – Brad Smith, President, Microsoft

Accelerating digital transformation in Norway

As we further our expansion commitment, we consider the demand for locally delivered cloud services and the opportunity for digital transformation in the market. Azure enables our customers and partners to increase their utilization of public cloud services and accelerate investments into private and hybrid cloud solutions. Norwegian organizations can now embrace these benefits to further innovation and build digital businesses at scale. Below are just a few of the customers and partners embracing Microsoft Azure in Norway.

The Norwegian banking industry is recognized for its rapid technology adoption, digitalizing the services that build the best products for customers. As Norway’s largest financial services group, DNB Group is a major operator in several industries, for which they also have a Nordic or international strategy. With Microsoft Azure, DNB will be able to migrate to the cloud in accordance with Norwegian data handling regulations to modernize, gain operational efficiency, and secure the best experience for its customers. 

“The possibility of data residency was a decisive factor in choosing Microsoft’s datacenter regions. Now we are looking forward to using the cloud to modernize and achieve efficiency and agility in order to ensure the best experience for our customers.” – Alf Otterstad, Executive Vice President, Group IT, DNB

Equinor, a broad energy company developing oil, gas, wind, and solar energy in more than 30 countries worldwide, has chosen Microsoft Azure to enable its digital transformation journey through a seven-year consumption and development agreement. With this strategic partnership, anchored in cloud-enabled innovation, and by moving its whole system portfolio to Azure, Equinor is aiming to achieve a more cost-efficient, safer, and more reliable operation. Equinor will utilize a variety of cloud services like machine learning and advanced analytics to improve performance, decrease costs, and increase safety. Through the partnership with Microsoft and leveraging capabilities within Azure, Equinor seeks to be a leader in the transformation of the energy industry worldwide and a growing force in renewables.

“Equinor’s ambition is to become a global digital leader within our industry. We have a long history of innovation and technology development. The strategic partnership will, through cloud services, involve development of the next-generation IT workplace, extended business application platforms, and mixed-reality solutions.” – Åshild Hanne Larsen, CIO and SVP, Corporate IT Equinor

Lånekassen, the Norwegian State Educational Loan Fund, has over 1.1 million customers, composed of former and current students. By moving to Azure, it seeks to develop new and transformative citizen services, based on cognitive and analytical technologies. Lånekassen’s purpose is to make education possible, and to provide the Norwegian workforce with relevant competences. It aims to strengthen student funding as well as maintain and increase the already high level of automatized customer services and application processes.

“It has been a priority for Lånekassen to focus on how we can utilize new technology to deliver an even better service for our students and manage our student financing schemes even more efficiently. As we move our core solutions into the cloud, it will give us increased opportunities to innovate. We have already had great success with using machine learning, and we are now looking forward to optimizing our operations further.” – Nina Schanke Funnemark, CEO, Lånekassen

Posten Norge AS has chosen to use the Microsoft Azure platform to meet ever-changing market demands by modernizing some of its existing applications estate and creating new services for its customers and partners. Posten’s next-generation logistics system will provide its workforce with new digital toolsets to deliver even better customer experiences.

“Posten’s vision is to make everyday life simpler and the world smaller. With this vision, we aim to simplify and increase the value of trade and communication for people and businesses in the Nordic region. With the opening of Norwegian datacenter regions, we hope to accelerate and fuel our vision further.” – Arne Erik Berntzen, CIO, Posten AS

Bringing the complete cloud to Norway

The new cloud regions in Norway connect with Microsoft’s 54 regions via our global network, one of the largest and most innovative on the planet, spanning more than 130,000 miles of terrestrial fiber and subsea cable systems to deliver services to customers. Microsoft brings the global cloud closer to home for Norwegian organizations and citizens through our transatlantic system Marea, the highest-capacity subsea cable to cross the Atlantic.

The new cloud regions in Norway are targeted to expand in 2020 with Office 365, one of the world’s leading cloud-based productivity solutions, and Dynamics 365 and Power Platform, the next generation of intelligent business applications and tools.

Learn more about the new cloud services in Norway and the availability of Azure regions and services across the globe.
Quelle: Azure

Introducing Azure Cost Management for partners

As a partner, you play a critical role in successful planning and managing long-term cloud implementations for your customers. While the cloud grants the flexibility to scale the cloud infrastructure to the changing needs, it does become challenging to control the spend when cloud costs can fluctuate dramatically with demand. This is where Azure Cost Management comes in to help you track and control cloud cost, prevent overspending and increase predictability for your cloud costs

Announcing general availability of Azure Cost Management for all cloud solution partners (CSPs) who have onboarded their customers to the new Microsoft Customer agreement. With this update, partners and their customers can take advantage of Azure Cost Management tools available to manage cloud spend, similar to the cost management capabilities available for pay-as-you-go (PAYG) and enterprise customers today.

This is the first of the periodic updates to enable cost management support for partners that enables partners to understand, analyze, dissect and manage cost across all their customers and invoices.

With this update, CSPs use Azure Cost Management to:

Understand invoiced costs and associate the costs to the customer, subscriptions, resource groups, and services.
Get an intuitive view of Azure costs in cost analysis with capabilities to analyze costs by customer, subscription, resource group, resource, meter, service, and many other dimensions.
View resource costs that have Partner Earned Credit (PEC) applied in Cost Analysis.
Set up notifications and automation using programmatic budgets and alerts when costs exceed budgets.
Enable the Azure Resource Manager policy that provides customer access to Cost Management data. Customers can then view consumption cost data for their subscriptions using pay-as-you-go rates.

For more information see, Get Started with Azure Cost Management as a Partner.

Analyze costs by customer, subscription, tags, resource group or resource using cost analysis

Using cost analysis, partners can group by and filter costs by customer, subscription, tags, resource group, resource, and reseller Microsoft partner Network identifier (MP NID), and have increased visibility into costs for better cost control. Partners can also view and manage the costs in the billing currency and in US dollars for billing scopes.

Reconcile cost to an invoice

Partners can reconcile costs by invoice across their customers and their subscriptions to understand the pre-tax costs that contributed to the invoice.

You can analyze azure spend for the customers you support and their subscriptions and resources. With this enhanced visibility into the costs of your customers, you can use spending patterns to enforce cost control mechanisms, like budgets and alerts to manage costs with continued and increased accountability.

Enable cost management at retail rates for your customers

In this update, a partner can also enable cost management features, initially at pay-as-you-go rates for your customers and resellers who have access to the subscriptions in the customer’s tenant. As a partner, if you decide to enable cost management for the users with access to the subscription, they will have the same capabilities to analyze the services they consume and set budgets to control costs that are computed at pay-as-you-go prices for Azure consumed services. This is just the first of the updates and we have features planned in the first half of 2020 to enable cost management for customers at prices that partner can set by applying a markup on the pay-as-you-go prices.

Partners can set a policy to enable cost management for users with access to an Azure subscription to view costs at retail rates for a specific customer.

If the policy is enabled for subscriptions in the customer’s tenant, users with role-based access control (RBAC) access to the subscription can now manage Azure consumption costs at retail prices.

Set up programmatic budgets and alerts to automate and notify when costs exceed threshold

As a partner, you can set up budgets and alerts to send notifications to specified email recipients when the cost threshold is exceeded. In the partner tenant, you can set up budgets for costs as invoiced to the partner. You can also set up monthly, quarterly, or annual budgets across all your customers, or for a specific customer, and filter by subscription, resource, reseller MPN ID, or resource group.

Any user with RBAC access to a subscription or resource group can also set up budgets and alerts for Azure consumption costs at retail rates in the customer tenant if the policy for cost visibility has been enabled for the customer.

When a budget is created for a subscription or resource group in the customer tenant, you can also configure it to call an action group. The action group can perform a variety of different actions when your budget threshold is met. For more information about action groups, see Create and manage action groups in the Azure portal. For more information about using budget-based automation with action groups, see Manage costs with Azure budgets.

All the experiences that we provide in Azure Cost Management natively are also available as REST APIs for enabling automated cost management experiences.

Coming soon

We will be enabling cost recommendation and optimization suggestions, for better savings and efficiency in managing Azure costs.
We will launch Azure Cost Management at retail rates for customers who are not on the Microsoft Customer Agreement and are supported by CSP partners.
Showback features that enable partners to charge a markup on consumption costs are also being planned for 2020.

Try Azure Cost Management for partners today! It is natively available in the Azure portal for all partners who have onboarded customers to the new Microsoft Customer Agreement.
Quelle: Azure

PyTorch on Azure with streamlined ML lifecycle

It's exciting to see the Pytorch Community continue to grow and regularly release updated versions of PyTorch! Recent releases improve performance, ONNX export, TorchScript, C++ frontend, JIT, and distributed training. Several new experimental features, such as quantization, have also been introduced.

At the PyTorch Developer Conference earlier this fall, we presented how our open source contributions to PyTorch make it better for everyone in the community. We also talked about how Microsoft uses PyTorch to develop machine learning models for services like Bing. Whether you are an individual, a small team, or a large enterprise, managing the machine learning lifecycle can be challenging. We'd like to show you how Azure Machine Learning can make you and your organization more productive with PyTorch.

Streamlining the research to production lifecycle with Azure Machine Learning

One of the benefits of using PyTorch 1.3 in Azure Machine Learning is Machine Learning Operations (MLOps). MLOps streamlines the end-to-end machine learning (ML) lifecycle so you can frequently update models, test new models, and continuously roll out new ML models alongside your other applications and services. MLOps provides:

Reproducible training with powerful ML pipelines that stitch together all the steps involved in training your PyTorch model, from data preparation, to feature extraction, to hyperparameter tuning, to model evaluation.
Asset tracking with dataset and model registries so you know who is publishing PyTorch models, why changes are being made, and when your PyTorch models were deployed or used in production.
Packaging, profiling, validation, and deployment of PyTorch models anywhere from the cloud to the edge.
Monitoring and management of your PyTorch models at scale in an enterprise-ready fashion with eventing and notification of business impacting issues like data drift.

Training PyTorch Models

With MLOps, data scientists write and update their code as usual and regularly push it to a GitHub repository. This triggers an Azure DevOps build pipeline that performs code quality checks, data sanity tests, unit tests, builds an Azure Machine Learning pipeline, and publishes it to your Azure Machine Learning workspace.

The Azure Machine Learning pipeline does the following tasks:

Train model task executes the PyTorch training script on Azure Machine Learning compute. It outputs a model file which is stored in the run history.
Evaluate model task evaluates the performance of the newly trained PyTorch model with the model in production. If the new model performs better than the production model, the following steps are executed. If not, they will be skipped.
Register model task takes the improved PyTorch model and registers it with the Azure Machine Learning model registry. This allows us to version control it.

You can find example code for training a PyTorch model, doing hyperparameter sweeps, and registering the model in this PyTorch MLOps example.

Deploying PyTorch models

The Machine Learning extension for DevOps helps you integrate Azure Machine Learning tasks in your Azure DevOps project to simplify and automate model deployments. Once a new model is registered in your Azure Machine Learning workspace, you can trigger a release pipeline to automate your deployment process. Models can then be automatically packaged and deployed as a web service across test and production environments such as Azure Container Instances and Azure Kubernetes Service (AKS). You can even enable gated releases so that, once the model is successfully deployed to the staging or quality assurance (QA) environment, a notification is sent to approvers to review and approve the release to production. You can see sample code for this in the PyTorch ML Ops example.

Next steps

We’re excited to support the latest version of PyTorch in Azure. With Azure Machine Learning and its MLOps capabilities, you can use PyTorch in your enterprise with a reproducible model lifecycle. Check out the MLOps example repository for an end to end example of how to enable a CI/CD workflow for PyTorch models.
Quelle: Azure

Bing delivers its largest improvement in search experience using Azure GPUs

Over the last couple of years, deep learning has become widely adopted across the Bing search stack and powers a vast number of our intelligent features. We use natural language models to improve our core search algorithm’s understanding of a user’s search intent and the related webpages so that Bing can deliver the most relevant search results to our users. We rely on deep learning computer vision techniques to enhance the discoverability of billions of images even if they don’t have accompanying text descriptions or summary metadata. We leverage machine-based reading comprehension models to retrieve captions within larger text bodies that directly answer the specific questions users have. All these enhancements lead toward more relevant, contextual results for web search queries.

Recently, there was a breakthrough in natural language understanding with a type of model called transformers (as popularized by Bidirectional Encoder Representations from Transformers, BERT). Unlike previous deep neural network (DNN) architectures that processed words individually in order, transformers understand the context and relationship between each word and all the words around it in a sentence. Starting from April of this year, we used large transformer models to deliver the largest quality improvements to our Bing customers in the past year. For example, in the query "what can aggravate a concussion", the word "aggravate" indicates the user wants to learn about actions to be taken after a concussion and not about causes or symptoms. Our search powered by these models can now understand the user intent and deliver a more useful result. More importantly, these models are now applied to every Bing search query globally making Bing results more relevant and intelligent.

Deep learning at web-search scale can be prohibitively expensive

Bing customers expect an extremely fast search experience and every millisecond of latency matters.  Transformer-based models are pre-trained with up to billions of parameters, which is a sizable increase in parameter size and computation requirement as compared to previous network architectures. A distilled three-layer BERT model serving latency on twenty CPU cores was initially benchmarked at 77ms per inference. However, since these models would need to run over millions of different queries and snippets per second to power web search, even 77ms per inference remained prohibitively expensive at web search scale, requiring tens of thousands of servers to ship just one search improvement.

Leveraging Azure Virtual Machine GPUs to achieve 800x inference throughput

One of the major differences between transformers and previous DNN architectures is that it relies on massive parallel compute instead of sequential processing. Given that graphics processing unit (GPU) architecture was designed for high throughput parallel computing, Azure’s N-series Virtual Machines (VM) with GPU accelerators built-in were a natural fit to accelerate these transformer models. We decided to start with the NV6 Virtual Machine primarily because of the lower cost and regional availability.  Just by running the three-layer BERT model on that VM with GPU, we observed a serving latency of 20ms (about 3x improvement). To further improve the serving efficiency, we partnered with NVIDIA to take full advantage of the GPU architecture and re-implemented the entire model using TensorRT C++ APIs and CUDA or CUBLAS libraries, including rewriting the embedding, transformer, and output layers.  NVIDIA also contributed efficient CUDA transformer plugins including softmax, GELU, normalization, and reduction.

We benchmarked the TensorRT-optimized GPU model on the same Azure NV6 Virtual Machine and was able to serve a batch of five inferences in 9ms, an 8x latency speedup and 43x throughput improvement compared to the model without GPU acceleration. We then leveraged Tensor Cores with mixed precision on a NC6s_v3 Virtual Machine to even further optimize the performance, benchmarking a batch size of 64 inferences at 6ms (~800x throughput improvement compared to CPU).

Transforming the Bing search experience worldwide using Azure’s global scale

With these GPU optimizations, we were able to use 2000+ Azure GPU Virtual Machines across four regions to serve over 1 million BERT inferences per second worldwide. Azure N-series GPU VMs are critical in enabling transformative AI workloads and product quality improvements for Bing with high availability, agility, and significant cost savings, especially as deep learning models continue to grow in complexity. Our takeaway was very clear, even large organizations like Bing can accelerate their AI workloads by using N-series virtual machines on Azure with built-in GPU acceleration. Delivering this kind of global-scale AI inferencing without GPUs would have required an exponentially higher number of CPU-based VMs, which ultimately would have become cost-prohibitive.  Azure also provides customers with the agility to deploy across multiple types of GPUs immediately, which would have taken months of time if we were to install GPUs on-premises.  The N-series Virtual Machines were essential to our ability to optimize and ship advanced deep learning models to improve Bing search, available globally today.

N-series general availability

Azure provides a full portfolio of Virtual Machine capabilities across the NC, ND, and NV series product lines. These Virtual Machines are designed for application scenarios for which GPU acceleration is common, such as compute-intensive, graphics-intensive, and visualization workloads.

NC-series Virtual Machines are optimized for compute-intensive and network-intensive applications.
ND-series Virtual Machines are optimized for training and inferencing scenarios for deep learning.
NV-series Virtual Machines are optimized for visualization, streaming, gaming, encoding, and VDI scenarios.

See our Supercomputing19 blog for recent product additions to the ND and NV-series Virtual Machines.

Learn more

Join us at Supercomputing19 to learn more about our Bing optimization journey, leveraging Azure GPUs.
Quelle: Azure