Cloud Computing Köln

Azure Arc: Extending Azure management to any infrastructure

If you are like many of our customers, you run a mix of applications in your on-premises datacenters, in the cloud and at the edge. We have been on a journey over the last few years to bring you hybrid innovations to meet you where you are. We have invested in individual connected management services such as Azure Monitor and Azure Backup. We have also delivered a consistent platform through Azure Stack Hub, ensuring that investments made in Azure can be used in disconnected environments.

Many enterprises still face a sprawl of resources spread across multiple datacenters, clouds, and edge locations. Our customers tell us that they are looking for a cloud-native control plane to inventory, organize, and enforce policies for their IT resources wherever they are, from a central place.

At Microsoft Ignite this week, we're taking another major step forward with our hybrid technology. We are announcing Azure Arc, a set of technologies that extends the control plane of Azure out to on-premises, multi-cloud environments and edge. Azure Arc enables customers to have a central, unified, and self-service approach to manage their Windows and Linux Servers, Kubernetes clusters, and Azure data services wherever they are. Azure Arc also extends adoption of cloud practices like DevOps and Azure security across on-premises, multi-cloud, and edge. In addition to extending the control plane for management, Azure Arc enables customers to run Azure data services anywhere.

Extend Azure management across your environments

Hundreds of millions of Azure resources are organized, governed, and secured daily by customers using Azure Resource Manager. Azure Resource Manager is the control plane in Azure that provides robust deployment, management, and governance capabilities with Azure Cloud Shell, Azure portal, API, role-based access control (RBAC) and Azure Policy for all Azure resources.

A key aspect of Azure Arc is the work we’ve done to extend Azure Resource Manager beyond Azure so that customers have a central and unified approach to manage Windows and Linux Servers, Kubernetes clusters and Azure data services at scale across on-premises, multi-cloud, and edge.

Azure Arc extends Azure management across on-premises, multi-cloud, and edge

Using Azure Arc to govern across environments

To illustrate the above scenarios of Azure Arc, let's take a look at a large financial organization that has sprawling server-based IT systems and Kubernetes clusters deployed in datacenters, private, and public clouds. The sprawl creates difficulty to have visibility across their environment and makes it harder to manage, govern and meet compliance requirements.

With Azure Arc, they can manage servers and Kubernetes clusters to get the following benefits:

Asset organization and inventory of Windows and Linux Servers, Kubernetes clusters and Azure services with a unified view in the Azure portal and API
Universal governance of customer resources through Azure Policy
Standardized role-based access control (RBAC) across systems and different types of resources
Enable application owners to apply and audit their applications to meet compliance requirements
Ability to measure and remediate compliance at scale and down to the individual application, server, or cluster

Adopting cloud practices on-premises

Azure provides cloud DevOps and cloud-native configuration management at scale for all Azure resources. Such cloud practices are optimized for developers that need immediate and programmatic access to resources to create new cloud-native applications. Azure Arc extends these capabilities to any infrastructure across on-premises, multi-cloud, and edge environments. Developers can build containerized apps with the tools of their choice and IT teams can use configuration as code to ensure that the apps are deployed, configured, and governed uniformly using GitOps-based configuration management across on-premises, multi-cloud, and edge.

Adopt cloud practices like config management at scale

Deploy to and manage multiple locations at scale

To illustrate the above scenario of Azure Arc, let's take a look at a retailer with 100s of stores that would like to move all in-store applications to containers running on a Kubernetes clusters. They are faced with the challenge of how to uniformly deploy, configure, and manage their containerized applications across multiple locations.

With Azure Arc, IT and development teams can manage the app in existing stores, and quickly light up a new location by automating error-prone and procedural tasks. Additionally, they get the following benefits:

At scale configuration and deployment based on Azure subscriptions, resource groups, and tags
GitOps-based model for deploying configuration-as-code to one or many clusters
Application deployment and update at scale
Source control based safe deployment practices when rolling out new applications and configurations
Freedom for developers to use the tools they are familiar with

Implement Azure security anywhere

We know the importance of security and compliance to businesses, so we brought our leadership in cloud security to on-premises, multi-cloud and edge with Azure Arc. We built Azure Arc to bring capabilities and practices such as RBAC, Azure activity log for auditing actions, Azure Lighthouse for secure delegated management and enforcement of security policies through Azure Policy.

Get started

We will be sharing more updates on Azure Arc at Microsoft Ignite this week. To learn more about Azure Arc, visit the Azure Arc page.

If you're at Microsoft Ignite this week, please attend the following sessions to learn more:
BRK 2208 Introduction to Azure Arc on Tuesday, Nov 05 at 11:45 am ET
BRK 3327 Azure Arc: Extend Management and Governance on Wednesday, Nov 06 at 1:00 PM ET

You can get started right away by previewing management of Windows and Linux servers across on-premises, multi-cloud, and edge right away. Join the preview to get started with managing Windows and Linux Servers anywhere using Azure Arc.

Sign up for more information on Azure data services anywhere enabled by Azure Arc, and management of Kubernetes clusters by Azure Arc.

Azure. Invent with purpose.
Quelle: Azure

Azure Machine Learning—ML for all skill levels

Enterprises today are adopting artificial intelligence (AI) at a rapid pace to stay ahead of their competition, deliver innovation, improve customer experiences, and grow revenue. AI and machine learning applications are ushering in a new era of transformation across industries from skill sets to scale, efficiency, operations, and governance.

Microsoft Azure Machine Learning provides enterprise-grade capabilities to accelerate the machine learning lifecycle and empowers developers and data scientists of all skill levels to build, train, deploy, and manage models responsibly and at scale. At Microsoft Ignite, we’re announcing a number of major advances to Azure Machine Learning across the following areas:

New studio web experience that boosts machine learning productivity for developers and data scientists of all skill levels, with flexible authoring options from no-code drag-and-drop and automated machine learning, to code-first development.
New industry-leading Machine Learning Operations (MLOps) capabilities to manage the machine learning lifecycle, enabling data science and IT teams to deliver innovation faster.
New open and interoperable capabilities that provide choice and flexibility with support for R, Azure Synapse Analytics, Azure Open Datasets, ONNX, and other popular frameworks, languages, and tools.
New security and governance features including role-based access control (RBAC), Azure Virtual Network (VNet), capacity management, and state-of-the-art responsible AI interpretability and fairness capabilities.

Let’s dive into these announcements in detail to see how Azure Machine Learning is helping individuals, teams, and organizations meet and exceed business goals.

Access machine learning for all skill levels and boost productivity

“By improving forecasting using Azure Machine Learning automated ML, we can reduce waste and ensure pizzas are ready for our customers. This will reduce the guesswork for our operators and allow them to spend more time focusing on other aspects of store operations. Rather than guessing how many pizzas to have ready, store operators are focusing on making sure every customer experience is an excellent one.” – Anita Klopfenstein, CEO, Little Caesars Pizza.

The new studio web experience (currently in preview) enables data scientists and data engineers of all skill levels to complete end-to-end machine learning tasks, including data preparation, model training, deployment, and management in a seamless manner. Choose from three different authoring options based on your skill and preference—no-code drag-and-drop designer, automated machine learning, or a code-first notebooks experience. Access Azure Machine Learning assets (including datasets and models) and rich capabilities (including data drift, monitoring, labeling and more) all from a single location.

 

Studio web experience

Designer (currently in preview) provides drag-and-drop workflows to simplify the process of building, testing, and deploying machine learning models using a visual experience. Customers currently using the classic version of Azure Machine Learning Studio are encouraged to try Designer so they can benefit from the scale and security of Azure Machine Learning.

Automated machine learning user interface (currently in preview) helps data scientists build models without writing a single line of code. Automate the time-intensive tasks of feature engineering, algorithm selection, and hyperparameter sweeping, then operationalize your model with a few clicks of a button.

Notebooks (currently in preview) are a fully managed solution for developers and data scientists to easily get started with machine learning, with pre-configured custom environments that eliminate setup time, while providing management and enterprise readiness capabilities for IT administrators.

New data labeling (currently in preview). High quality labeled data is vital to creating high accuracy models for supervised learning. Teams can now manage data labeling projects seamlessly from within the studio web experience to get labels against data, speeding up the time-intensive process of manual labeling. Labeling tasks supported include object detection, multi-class image classification, and multi-label image classification.

Operationalize at scale with industry-leading MLOps

Azure Machine Learning features built-in MLOps capabilities for enterprise-grade machine learning lifecycle management, that enables data science and IT teams to collaborate and increase the pace of model development and deployment.

“TransLink was able to leverage MLOps in Azure Machine Learning to build and manage models and deploy them in production. This created greater efficiencies and transparency as we moved over 16,000 machine learning models from pilot to production. Ultimately, TransLink customers benefited with improvement between predicted and actual bus departure times of 74%, so they can better plan their journey on TransLink's bus network.” – Sze-Wan Ng, Director Analytics & Development, Translink.

New updates to build reproducible models and achieve machine learning governance and control

Datasets help data scientists and machine learning engineers easily access data from a number of Azure storage services, apply datasets rapidly, reuse them efficiently across tasks, and track data lineage automatically. Rich dataset and model registries help track assets and information to effectively operationalize models and simplify workflows from training to inferencing. Version control helps track and manage assets providing enhanced traceability and supporting the creation of reproducible pipelines for consistent model delivery. Audit trail capabilities ensure asset integrity and provide control logs to help meet regulatory requirements.

New updates to easily deploy models and efficiently manage the machine learning lifecycle

Batch inference helps increase productivity and decrease cost by generating predictions on terabytes of structured or unstructured data. Controlled roll-out enables the deployment of different model versions under a common scoring endpoint in order to implement a sophisticated deployment pipeline and release models with confidence. Data drift monitoring helps maintain model accuracy by detecting model performance issues from changes to model input data over time. Drift analysis includes magnitude of drift, contribution by feature, and other insights so that appropriate action can be taken, including retraining the model.

 

Data drift monitoring

Innovate using open and interoperable capabilities

With Azure Machine Learning, developers and data scientists can access built-in support for open source tools and frameworks like PyTorch, TensorFlow, and scikit-learn, or the open and interoperable ONNX format. We now support Open Neural Network Exchange (ONNX), the open standard for representing machine learning. With the new v1.0 release, ONNX Runtime offers stable Python APIs that can be used in Azure Machine Learning on both CPU and GPU.

New R-based capabilities enable data scientists to run R jobs on Azure Machine Learning and then manage and deploy R models as web services. Data scientists can choose their development environment of choice—one-click access to the browser integrated development (IDE) of RStudio Server (open source edition) or Jupyter with R.

Azure Synapse Analytics is now deeply integrated with Azure Machine Learning to greatly expand the discovery of insights from all your data and apply machine learning models to your intelligent apps.

Azure Open Datasets are now generally available and provide curated datasets, hosted on Azure, and easily accessible from Azure Machine Learning workspaces to accelerate model training. Over 25 datasets are now available, including socio-economic data, satellite imagery, and more. New datasets are continuously being added, and you can nominate additional datasets to Azure.

Build on a secure foundation

“With Azure Machine Learning our data scientist teams can work in an environment supported with industry standard trust and compliance. Enterprise readiness capabilities like RBAC VNet, Key Vault ensure that we have granular control over our resources and deliver innovation on  a secure platform that enhances productivity so that teams can focus on machine learning tasks rather than infrastructure and setup.”-  Cary Goltermann, Manager, Ignition Tax, KPMG LLP.

Security and enterprise readiness updates

Workspace capacity management (currently in preview) helps administrators review compute usage across workspaces and clusters within a subscription for efficient resource distribution. Capacity limits can be set to reallocate resources for capacity management and governance. Role Based Access Control, or RBAC, (in preview) helps define custom roles for granular access control and supports advanced security scenarios. Virtual network, or VNet, (in preview) provides a security boundary to isolate compute resources used to train and deploy models when running experiments through inferencing.

Fairness: In addition to model interpretability in Azure Machine Learning, which supports transparency and model understanding, data scientists and developers can now leverage Fairlearn, the new open source fairness assessment and mitigation tool. This tool assists organizations with uncovering insights about fairness in their model predictions through an intuitive and configurable set of visualizations.

 

Fairness feature insights

Start building today

We are excited to bring you these capabilities to help accelerate the machine learning lifecycle, from new productivity experiences that make machine learning accessible to all skill levels, to robust MLOps and enterprise-grade security, built on an open and trusted platform. We are committed to continued investments in machine learning to support your business and applications and help you drive business transformation with AI.

Get started with a free trial of Azure Machine Learning.
Learn more using new samples and tutorials.
Read all the Azure AI news from Microsoft Ignite.

Azure. Invent with purpose.
Quelle: Azure