Monat: April 2022

Accelerate your AI applications with Azure NC A100 v4 virtual machines

Real-world AI has revolutionized and changed how people live during the past decade, including media and entertainment, healthcare and life science, retail, automotive, finance service, manufacturing, and oil and gas. Speaking to a smart home device, browsing social media with recommended content, or taking a ride with a self-driving vehicle is no longer in the future. With the ease of your smartphone, you can now deposit checks without going to the bank? All of these advances have been made possible through new AI breakthroughs in software and hardware.

At Microsoft, we host our deep learning inferencing, cognitive science, and our applied AI services on the NC series instances. The learnings and advancements made in these areas with regard to our infrastructure are helping drive the design decisions for the next generation of NC system. Because of our approach, our Azure customers are able to benefit from our internal learnings.

We are pleased to announce that the next generation of NC A100 v4 series is now available for preview. These virtual machines (VMs) come equipped with NVIDIA A100 80GB Tensor Core PCIe GPUs and 3rd Gen AMD EPYC™ Milan processors. These new offerings improve the performance and cost-effectiveness of a variety of GPU performance-bound real-world AI training and inferencing workloads. These workloads cover object detection, video processing, image classification, speech recognition, recommender, autonomous driving reinforcement learning, oil and gas reservoir simulation, finance document parsing, web inferencing, and more.

The NC A100 v4-series offers three classes of VM ranging from one to four NVIDIA A100 80GB PCIe Tensor Core GPUs. It is more cost-effective than ever before, while still giving customers the options and flexibility they need for their workloads.

Size

vCPU

Memory (GB)

GPUs (NVIDIA A100 80 GB Tensor Core)

Azure Network (Gbps)

Standard_NC24ads_A100_v4

24

220

1

20

Standard_NC48ads_A100_v4

48

440

2

40

Standard_NC96ads_A100_v4

96

880

4

80

Compared to the previous NC generation (NCv3) with NVIDIA Volta architecture-based GPUs, customers will experience between 1.5 and 2.5 times the performance boost due to:

Two times GPU to host bandwidth.
Four times vCPU cores per GPU VM.
Two times RAM per GPU VM.
Seven independent GPU instances on a single NVIDIA A100 GPU through Multi-Instance GPU (MIG) on Linux OS.

Below is a sample of what we experienced while running ResNet50 AI model training across a variety of batch sizes using the VM size NC96ads_A100_v4 compared to the existing NCv3 4 V100 GPUs VM size NC24s_v3. Tests were conducted across a range of batch sizes, from one to 256.

Figure 1: ResNet50 results were generated using NC24s_v3 and NC96ads_A100_v4 virtual machine sizes.

For additional information on how to run this on Azure and additional results please check out our performance technical community blog.

With our latest addition the NC series, you can reduce the time it takes to train your model training in around half the time and still within budget. You can seamlessly apply the trained cognitive science models to applications through batch inferencing, run multimillion atomics biochemistry simulations for next-generation medicine, host your web and media services in the cloud for tens of thousands of end-users, and so much more.

Learn more

The NC A100 v4 series are currently available in the South Central US, East US, and Southeast Asia Azure regions. They will be available in additional regions in the coming months.
For more information on the Azure NC A100 v4-series, please see:

Sign up for the preview of the NVIDIA A100 Tensor Core PCIe GPU in the Azure NC A100 v4-series. 
Performance of NC A100 v4-series.
Find out more about high-performance computing (HPC) in Azure.
Microsoft documentation for NC A100 v4-series VM.
Azure HPC optimized OS images.
Azure GPU virtual machines.

Quelle: Azure

Unlock cloud savings on the fly with autoscale on Azure

Unused cloud resources can put an unnecessary drain on your computing budget, and unlike legacy on-premises architectures, there is no need to over-provision compute resources for times of heavy usage.

Autoscaling is one of the value levers that can help unlock cost savings for your Azure workloads by automatically scaling up and down the resources in use to better align capacity to demand. This practice can greatly reduce wasted spend for those dynamic workloads with inherently “peaky” demand.

In some cases, workloads with occasionally high peak demand have extremely low average utilization, making them ill-suited for other cost optimization practices, such as rightsizing and reservations.

For periods when an app puts a heavier demand on cloud resources, autoscaling adds resources to handle the load and satisfy service-level agreements for performance and availability. And for those times when the load demand decreases (nights, weekends, holidays), autoscaling can remove idle resources to reduce costs. Autoscaling automatically scales between the minimum and maximum number of instances and will run, add, or remove VMs automatically based on a set of rules.

Autoscaling is near real-time cost optimization. Think of it this way: Rather than build an addition to your house with extra bedrooms that will go unused most of the year, you have an agreement with a nearby hotel. Your guests can check-in, at any time and at the last minute, and the hotel will automatically charge you for the days when they visit.

Not only does it utilize cloud elasticity by paying for capacity only when you need it, you can also reduce the need for an operator to continually monitor the performance of a system and make decisions about adding or removing resources.

What services can you autoscale?

Azure provides built-in autoscaling using Azure Monitor autoscale for most compute options, including:

Azure Virtual Machines Scale Sets—see How to use automatic scaling and virtual machine scale sets.
Service Fabric—see Scale a Service Fabric cluster in or out using autoscale rules.
Azure App Service—see Scale instance count manually or automatically.
Azure Cloud Services has built-in autoscaling at the role level. See How to configure autoscaling for a cloud service in the portal.

Azure Functions differs from the previous compute options because you don't need to configure any autoscale rules. The hosting plan you choose dictates how your function app is scaled:

With a consumption plan, your functions app will scale automatically, and you will only pay for compute resources when your functions are running.
With a premium plan, your app will automatically scale based on demand using pre-warmed workers that run applications with no delay after being idle.
With a dedicated plan, you will run your functions within an App Service plan at regular App Service plan rates.

Azure Monitor autoscale provides a common set of autoscaling functionality for virtual machine scale sets, Azure App Service, and Azure Cloud Service. Scaling can be performed on a schedule, or based on a runtime metric, such as CPU or memory usage.

Use the built-in autoscaling features of the platform if they meet your requirements. If not, carefully consider whether you really need more complex scaling features. Examples of additional requirements may include more granularity of control, different ways to detect trigger events for scaling, scaling across subscriptions, and scaling other types of resources.

Note that application design can impact how that app handles scale as a load increases. To review design considerations for scalable applications, including choosing the right data storage and VM size, and more, check out Design scalable Azure applications—Microsoft Azure Well-Architected Framework.

Also know that, in general, it is better to scale up than to scale down. Scaling down usually involves deprovisioning or downtime. So, choose smaller instances when a workload is highly variable and scale out to get the required level of performance.
You can set up autoscale in the Azure portal, PowerShell, Azure CLI, or Azure Monitor REST API.

Get started with autoscaling

With autoscaling, you can dynamically scale your apps to meet changing demand or anticipate loads with different schedules and set rules that trigger scaling actions. Regardless of how you set it up, the goal is to maximize the performance of your application and save money by not wasting server resources.
Quelle: Azure