Scale your Python service with Managed Disks

The Azure Python SDK now supports Azure Managed Disks!

 

Azure Managed Disks and 1000 VMs in a Scale Set are now generally available. Azure Managed Disks provide a simplified disk management, enhanced scalability, and better security. It takes away the notion of storage account for disks, enabling developers to scale without worrying about the limitations associated with storage accounts. This post provides a quick introduction and reference to consuming key service features from Python.

 

From a developer perspective, the Managed Disks experience in Azure CLI is idomatic to the CLI experience in other cross-platform tools. You can use the Azure Python SDK and the azure-mgmt-compute package 0.33.0 to administer Managed Disks. You can create a compute client using this tutorial. The complete API documentation is available on ReadTheDocs.

Standalone Managed Disks

Prior to Managed Disks, developers needed to maintain images for their VMs in multiple storage accounts to avoid the risk of running out of disk space. It is easy to see how this can complicate the architecture, and the dev-ops, for a service that requires a large number of VMs quickly, and has to be available across multiple regions. With Managed Disks, you do not need to worry about replicating images into new storage accounts. You can have a single image per region, and the service will make sure they are available for up to 10,000 VMs under a single subscription.

 

You can create new disks from various starting points with a lines of Python code. Here are a few specific examples:

Create an empty Managed Disk
Create a Managed Disk from Blob Storage
Create a Managed Disk from our own Image

 

Here’s a quick preview for creating an empty Managed disk in Python with a few lines of code:

from azure.mgmt.compute.models import DiskCreateOption

 

        async_creation = compute_client.disks.create_or_update(

            &;my_resource_group&039;,

            &039;my_disk_name&039;,

            {

                &039;location&039;: &039;westus&039;,

                &039;disk_size_gb&039;: 20,

                &039;creation_data&039;: {

                    &039;create_option&039;: DiskCreateOption.empty

                }

            }

        )

        disk_resource = async_creation.result()

Virtual Machine with Managed Disks

Now that you know the basics of creating managed disks, but how do you configure your service to create VMs from images stored on a Managed Disk? The service affords flexibility to create VMs from various types of Managed Disks. You can create a VM with an implicit Managed Disk for a specific disk image. Creation is simplified with implicit creation of managed disks without specifying all the disk details. You do not have to worry about creating and managing Storage Accounts.

 

A Managed Disk is also implicitly created when provisioning a VM from an OS image on the Azure Marketplace. Here’s an example for a Ubuntu VM. Notice how the storage account parameter is optional in the VM definition.

 

storage_profile = azure.mgmt.compute.models.StorageProfile(

                image_reference = azure.mgmt.compute.models.ImageReference(

                    publisher=&039;Canonical&039;,

                    offer=&039;UbuntuServer&039;,

                    sku=&039;16.04.0-LTS&039;,

                    version=&039;latest&039;

                )

            )

 

You can easily attach a previously provisioned Managed Disk as shown here. See a complete example on how to create a VM in Python (including network), and how check the full VM tutorial in Python.

Virtual Machine Scale Sets with Managed Disks

For very large scale services, Azure recommends using Virtual Machine Scale Sets (VMSS). VMSS allows developers to create a pool of VMs with identical configuration. The service allows “true autoscale” – developers do not need to pre-provision VMs. Prior to Managed Disks, the developers needed to consider the design carefully to ensure efficient Disk IO, ideally using a single storage account for up to 20 VMs. This limitation required developers to create and manage additional storage accounts to support a larger scale. With Managed Disk, you don’t have to manage any storage account at all. If you are used to the VMSS Python SDK, your storage_profile can now be exactly the same as the one used in VM creation. This feature also simplifies programming – you no longer have to manage any storage account at all.

 

The official guide to transitioning from user managed storage to Managed Disks is available in this article. Quick samples are also available for preview.

Get productive with the Azure CLI

If the CLI is your management tool of choice, there are several handy commands available for various scenarios. For example, here’s how you can create a stand alone Managed Disk from the Azure CLI with a single command:

 

az disk create -n myDisk -g myResourceGroup –size-gb 20

 

Check out Aaron Roney’s blog post to learn more CLI commands for programming Managed Disks.

Other operations

There are numerous other quick management operations you might need to get started with Managed Disks. See sample code for the following operations:

Resizing a managed disk from the Azure CLI

Updating the Storage Account type of the Managed Disks

Creating an image from Blob Storage

Creating a snapshot of a Managed Disk that is currently attached to a Virtual Machine

In summary

Managed Disks can tighten your workflow, simplify your service architecture, and offer you greater peace of mind in running a highly scalable Python cloud service. It also offers better reliability for Availability Sets by ensuring that the disks of VMs in an Availability Set are sufficiently isolated from each other to avoid single points of failure, and offers better security via granular role based access to resources. You can use the Azure CLI to create and manage your Managed Disks. Hopefully this blog post serves as a quick reference as you try Managed Disks on your own. For more information about the service, head over the Azure documentation. For feedback on the Python SDK, please send an email to azurepysdk@microsoft.com.
Quelle: Azure

Microsoft Networking Academy with the Azure Black Belt Team – Winter 2017!

Welcome to our new networking webinar series! We’ve changed the name of our bi-weekly talk to better reflect what we want to convey to you, our customers and partners! And this comes with a new name: Microsoft Networking Academy !

The Microsoft Network Academy session is taking place every other Friday this winter and spring. It is open to all customers and partners to learn more about Azure Networking, including ExpressRoute and Virtual Networking, and how to plan and design their connectivity to the Microsoft Cloud.

As a team, we’ve decided to create two formats for our Microsoft Networking Academy series: the introductory sessions, and deep dive sessions, generally presented in alternance.

In both formats, there will be an open Q&A session at the end where customers can ask the experts. Content and partner speakers will vary for each session, but the general agenda is as follows:

Introductory sessions

Azure Networking fundamentals (10 minutes)
Deep dive topic of the week (15-20 minutes)
Partner spotlight of the week (15-20 minutes)
Q&A

Deep dive sessions

Introduction (5 minutes)
Deep dive topic of the week (35-45 minutes)
Q&A (10 minutes)

We will post the agenda in advance on this blog, and to our interested viewers, you can join a distribution list by sending an email to gbb-anf@microsoft.com with the subject-line “Join Microsoft Networking Academy List”. We will email you a reminder and the agenda in advance for the upcoming sessions.

We kicked off this new Academy Series on Friday, February 17th, 2017.

Join the Skype Meeting and make sure you don’t miss out on future sessions by adding this the series to your Outlook calendar. You can also download ICS (same agenda entry for both introductory and deep dive sessions).

Here are a few links for convenience:

Session recordings for Microsoft Networking Academy will be posted on Channel 9
Previous sessions are already posted on Channel 9’s Azure Networking Fridays channel
Check list presented in our sessions
Fall 2016 season’s sessions and early winter 2017 sessions

February 3rd’s – Introductory session call recap:

Azure Networking Fundamentals
Technical overview of Azure Load Balancing with our new team member, Bryan Woodworth!
Partner Spotlight – Riverbed’s SD-WAN solution accelerates the adoption of Azure via unique automation algorithms, cloud-centric management workflows and an industry-leading focus on app performance. Come see how this really happens!
Links to the deck and to the video recording on Channel 9

February 17th’s – Deep Dive session call recap:

Quick introduction and announcement
Deep dive on the 3 ExpressRoute peerings with Eddie Villalba
Ask the Experts Q&A!
Links to the deck and to the video recording on Channel 9

March 3rd’s – Introductory session call agenda:

Azure Networking Fundamentals
Partner Spotlight – Citrix’s SD-WAN solution
The sessions will be recorded and posted on Channel 9!

Quelle: Azure

Announcing Azure Network Watcher – Network Performance Monitoring and Diagnostics Service for Azure

Have you ever felt the need to diagnose a critical problem and you needed access to packet data from a virtual machine? What if you could capture the packet data from a virtual machine in just a few clicks? How about the ability to log flow data for Network Security Groups, visualize and interpret the information with a tooling platform of your choice?

With Azure Network Watcher, you can now access a plethora of logging and diagnostic capabilities that empower you with insights to understand your network performance and health. These capabilities are accessible via Portal, Power Shell, CLI, Rest API and SDK.

What does Network Watcher enable for you?

Topology

You can now view the network topology of your deployments with just a few clicks. For example, the figure below represents the network topology of a simple web application deployed on Azure. With Network Watcher, you can now visualize the complete network topology of your application.

Sample topology view of a web application

IP flow verify

A common diagnostic need is to check whether a flow is allowed or denied to or from a virtual machine. Using “IP flow verify” you can now validate if a flow (combination of source IP, destination IP, source port, destination port and protocol) is allowed or denied. You will also be provided with the specific Network Security Group and security rule allowing or denying the flow in question.

Validate IP flow from the Portal

Next hop

Typical issues with network connectivity is misconfiguration of user defined routes. Next hop provides the ability to get the next hop type and IP address based on a specified virtual machine, allowing you to investigate any route being black-holed and conditions caused by incorrect configuration.

Get next hop from the Portal

Security Group view

Auditing your network security is vital for detecting network vulnerabilities and ensuring compliance with your IT security and regulatory governance model.

With Security Group view, you can retrieve the configured Network Security Group and security rules, as well as the effective security rules. With the list of rules applied, you can determine the ports that are open and assess network vulnerability.

In addition, your IT security and compliance governance can define prescriptive security rules that can now be programmatically audited using this feature.

As an example, PCI DSS compliance emphasizes the need to store logs and review logs that perform security functions such as firewalls. The primary intent for this is to identify anomalies and suspicious activity. With a combination of flow logs, Security Group view and Azure Automation, periodic and frequent audit can be done in a programmatic manner. You can detect and alert on suspicious and anomalous activity.

Network Security Group view for a virtual machine from the Portal

Packet capture

Capturing and accessing packet data enables you to address various needs from diagnosing a connectivity issue to network security and compliance. With Network Watcher, you can trigger packet capture on virtual machines. Applying advanced rule matching options, you can capture packets that have a specific source IP, destination IP, source port or destination port, or a byte offset from the start of the packet – even a combination of all the above. This feature is supported on both Windows and Linux virtual machines.

Configuring packet capture from the Portal

Network Subscription limits

You can now view the usage of network resources against the limits in your subscription.

View limits for network resources in your subscription in a region

NSG flow logs

Flow data is a critical component for diagnosing and validating your Network Security Group configurations. You can now enable logging of NSG flow data that is allowed or denied per Network Security Group setting to help meet these needs. The NSG flow information includes timestamp, source IP, destination IP, source port, destination port and protocol, the Network Security Group and the security rule. This data can be ingested and visualized by Microsoft tools such as Power BI, as well as security information and event management tools provided by 3rd party partners and open source tools.

Configuring NSG flow logs from the Portal

A sample Power BI dashboard with the ingested flow log

Diagnostic logs

You can now configure diagnostic logs for all the network resources in a resource group from a single pane.

Configuring Diagnostic logs for network resources in a resource group

Virtual Network Gateway Connectivity Troubleshooting

A Virtual Network Gateway provides connectivity between your on-premises site and Azure VNets. Network Watcher will enable you to troubleshoot issues due to connectivity. A comprehensive suite of built-in tests are executed to isolate over fifteen different fault conditions and the results are logged in a customer specified storage. The log contains information such as connection status, bytes sent/received, IKE errors and WFP logs.

Integration with Azure Services

Using the native capabilities offered by Network Watcher, you can build powerful end to end network monitoring scenarios using Azure services like Azure Automation, Azure Functions and Azure Log Analytics.

Proactive monitoring of VPN connection using Azure Automation and Network Watcher

Partners and ecosystem integration

We have partnered with the following 3rd party tool providers to integrate their products with Network Watcher and provide you with a holistic experience in monitoring your network in Azure.

Splunk have built an operational intelligence platform by turning data generated from Network Watcher into valuable insights.

Observable Networks have integrated the packet capture capability of Network Watcher with their ONA platform (Observable Network Appliance) to detect security issues in your virtual machine.

Bryan Doerr, CEO of Observable Networks said, “We’re excited that the results of our continuous and close collaboration with Microsoft are now reaching our mutual customers. Digital transformation and the fast-growing transition to cloud platforms, like Azure, are creating demand for new cloud native security services."

Sumo Logic provides a machine data analytics platform that can ingest flow data for Network Security Groups to help you understand network vulnerabilities.

Kalyan Ramanathan, VP of Product Marketing at Sumo Logic said, “The cloud is changing the IT landscape. New business models, rapidly changing innovation and operations are driving a new set of needs. We are pleased to be teaming with Microsoft to further enhance the cloud experience for our mutual customers. Sumo Logic Machine Data analytics solution provides real-time operational insights into today’s modern applications with deep Microsoft Azure Integration, to help customers address the volume, variety and velocity of cloud generated data.”

Open source tools

Your network monitoring needs can be augmented by open source tools such as Capanalytics, Suricata and ELK (Elastic Search, Logstash and Kibana). We hope you will be able to leverage and build on the sample integration scenarios for visualizing packet capture, network intrusion detection and visualizing flow logs.

A sample dashboard highlighting network intrusion – integrating Network Watcher, Suricata and ELK

Network Watcher availability

Azure Network Watcher is available now in preview in the following regions – US West Central, US North and US West. We are in the process of rolling out Network Watcher the rest of Azure regions around the world.

How much does it cost?

We understand the current capabilities in Network Watcher are critical to a variety of your needs from diagnostics to security and compliance. These capabilities will be available free with your subscription. Standard storage costs are applicable in certain cases.

Your requirements and requests for an integrated solution and tooling is at the center of building this advanced network monitoring capability in Azure. Your feedback from using Network Watcher is vital to help steer the product development and eco system growth.

Enjoy the preview!
Quelle: Azure

Azure Government extends lead in the Cloud with leap in FedRAMP coverage

Microsoft continues to lead the charge in realizing the power and promise of the cloud for Government customers. We are pleased to announce that Azure Government has been granted authorization for 12 additional customer-facing services to our FedRAMP High P-ATO. We now offer 32 Infrastructure and Platform services to our customers in our Azure Government compliance boundary, all of which have been authorized for use with up to High Impact level data. Our coverage of services, and the rate at which we are increasing our scope, highlight our commitment to being the most Trusted and Certified cloud, accelerating compliance for Government customers.

We are pleased to announce that the entire Operations Management Suite (OMS) is now authorized for use in Azure Government. The OMS suite empowers customers to take full advantage of the Hybrid Cloud. OMS enables customers to gain insight across their entire fleet, allowing faster response to security threats, enabling consistent control and compliance, and ensuring the availability of apps and data irrespective of where those applications or data live; in Azure, on premises, or on other cloud platforms using a federated clouds model.

Azure Log Analytics: helps you collect and analyze data generated by resources in your cloud and on-premises environments. It gives you real-time insights using integrated search and custom dashboards to readily analyze millions of records across all your workloads and servers regardless of their physical location.
Azure Automation: saves time and increases the reliability of regular administrative tasks – even schedules them to be automatically performed at regular intervals. You can automate processes using runbooks or automate configuration management using Desired State Configuration. The Automation service allows customers to maximize the value proposition of the cloud, consuming services on demand and only when required.
Azure Backup: is a unified solution to protect data on-premises and in the cloud, with 99.9% guaranteed availability! Incremental backups provide efficiency and geo-replicated storage ensures you meet availability requirements for High Impact data. And to top it off, all data is encrypted in transit and at rest using FIPS 140-2 validated encryption modules.
Azure Site Recovery: delivers the power of the cloud for disaster recovery scenarios. With Azure Site Recovery, you can automate protection and replication of your virtual machines, remotely monitor the health of your fleet, orchestrate recovery as needed using customizable plans, and test your recovery capabilities without impacting your system availability.

Authorization of the Azure Resource Manager (ARM) service enables our government customers to deploy complex architectures in Azure Government automatically and consistently. With ARM, you define the infrastructure and dependencies for your app in a single declarative template. ARM templates are flexible enough to use for all your environments; test, staging, or production.  If you create a solution from the Azure Marketplace, the solution will automatically include a template that you can use for your app.

Corresponding with the addition of ARM, we have also added our Resource Providers for Compute, Storage, and Networking. These Resource Providers enable seamless, automated deployment of Compute, Storage, and Networking resources as needed and on demand using the ARM template architecture; including ARM templates that meet certification requirements which is coming soon as part of our Azure Blueprint program.

Azure CRP: is the Compute Resource Provider, used in creating and managing virtual machine resources and extensions in simple to use Azure Resource Manager templates.
Azure SRP: is the Storage Resource Provider, used in creating and managing blob, table, queue, and storage account management resources in simple to use Azure Resource Manager templates.
Azure NRP: is the Network Resource Provider, which delivers a series of Software-defined Networking (SDN) and Network Function Virtualization features for the Azure Government environment.  NRP gives you more granular network control, metadata tags, faster configuration, rapid and repeatable customization, and multiple control interfaces. You can use the NRP to create software load balancers, public IPs, network security groups, virtual networks, among others.

Microsoft has collaborated with our regulators to dramatically decrease the time required to take a service from available to certified. In fact, we have a roadmap that adds all services currently available in Azure Government to our FedRAMP High boundary. We are committed to ensuring that Azure Government provides the best the cloud has to offer and that all of our offerings are certified at the highest levels of compliance. Please visit the Microsoft Trust Center for additional details and reach out to AzureBlueprint@Microsoft.com for support on how these compliant services can be included in your cloud ATO efforts.

Product Group
Azure Government Service
Availability

Compute
 
 

 
Batch
Newly Authorized

 
Cloud Services
Authorized

 
Compute Resource Provider
Newly Authorized

 
Virtual Machines
Authorized

 
Service Fabric
In Progress

 
Virtual Machine Scale Sets
In Progress

Storage
 
 

 
Storage
Authorized

 
Storage Resource Provider
Newly Authorized

Networking
 
 

 
Application Gateway
Authorized

 
Express Route
Authorized

 
Load Balancer
Authorized

 
Network Resource Provider
Newly Authorized

 
Traffic Manager
Authorized

 
Virtual Network
Authorized

 
VPN Gateway
Authorized

Databases
 
 

 
Redis Cache
Newly Authorized

 
SQL Database
Authorized

 
SQL Data Warehouse
In Progress

 
SQL Server Stretch Database
In Progress

Intelligence + Analytics
 
 

 
Power BI
Newly Authorized

 
HDInsight
In Progress

Monitoring + Management
 
 

 
Automation
Newly Authorized

 
Azure Government Portal
Newly Authorized

 
Azure Resource Manager
Newly Authorized

 
Azure Runtime
Authorized

 
Backup
Authorized

 
Log Analytics
Newly Authorized

 
Scheduler
Newly Authorized

 
Site Recovery
Authorized

Security + Identity
 
 

 
Azure Active Directory
Authorized

 
Key Vault
Authorized

 
Azure MFA
In Progress

Web + Mobile
 
 

 
Media Services
Newly Authorized

 
Notification Hubs
Authorized

 
Web Apps
Authorized

 
API Apps
In Progress

 
Mobile Apps
In Progress

Enterprise Integration
 
 

 
Service Bus
Authorized

 
Store Simple
Authorized

IoT
 
 

 
Event Hubs
Authorized

 
 
 

Quelle: Azure

Azure Analysis Services now available in Canada Central and Australia Southeast

Last October we released the preview of Azure Analysis Services, which is built on the proven analytics engine in Microsoft SQL Server Analysis Services. With Azure Analysis Services you can host semantic data models in the cloud. Users in your organization can then connect to your data models using tools like Excel, Power BI, and many others to create reports and perform ad-hoc data analysis.

We are excited to share with you that the preview of Azure Analysis Services is now available in 2 additional regions: Canada Central and Australia Southeast.  This means that Azure Analysis Services is available in the following regions: Australia Southeast, Canada Central, Brazil South, Southeast Asia, North Europe, West Europe, West US, South Central US, North Central US, East US 2 and West Central US.

New to Azure Analysis Services? Find out how you can try Azure Analysis Services or learn how to create your first data model.
Quelle: Azure

Kubernetes now Generally Available on Azure Container Service

There is a common thread in advancements in – they enable a focus on applications rather than the machines running them. Containers, one of the most topical areas in cloud computing, are the next evolutionary step in virtualization.

Companies of every size and from all industries are embracing containers to deliver highly available applications with greater agility in the development, test and deployment cycle. Azure Container Service (ACS) is a service optimized for container applications. Today we are pleased to announce a number of improvements to ACS, most notably Kubernetes is now Generally Available as one of three choices of orchestrator.

Azure is the only public cloud platform that provides a container service with the choice of the three most popular open source orchestrators available today. ACS’s approach of openness has been pivotal in driving the adoption of containers on Azure. Enterprises and startups alike recognize the momentum around ACS and the benefit it brings to their applications, which includes agile deployment, portability and scalability.

With today’s news, we again deliver on our goal of providing our customers the choice of open-source orchestrators and tooling that simplifies the deployment of container based applications in the cloud. The ACS team are announcing today our next wave of features that includes:

Kubernetes now generally available (GA) – We announced preview support for Kubernetes in November 2016. Since then, we have received a lot of valuable feedback from customers. Based on this feedback we have improved Kubernetes support and now move it to GA. For more details, check out Brendan’s blog titled, "Containers as a Service: The foundation for next generation PaaS".
Preview of Windows Server Containers with Kubernetes – Coinciding with latest Kubernetes release, adding support for Windows Server Containers and coupled with enterprise customers expressing strong interest in adopting and going into production with Windows Server Containers, this is a great time to provide additional choice in orchestrator for Windows Server customers using ACS. Customers can now preview both Docker Swarm (launched in preview last year) as well as Kubernetes though ACS, providing choice as well as consistency with two of the top three Linux container orchestration platforms.
DC/OS 1.8.8 update – We are updating our DC/OS support to version 1.8.8. DC/OS is a production-proven platform that elastically powers both containers and big data services. ACS delivers the open source DC/OS while our partnership with Mesosphere ensures customers requiring additional enterprise features are catered for. Key features of 1.8.8 include a new orchestration framework, called Metronome, to run scheduled jobs. This has been added to a new Jobs tab in the DC/OS UI, along with a number of other UI improvements; and the addition of GPU and CNI support in the Universal container runtime. Based on Apache Mesos, DC/OS is trusted by Esri, BioCatch and many other Fortune 1000 companies. We have worked with Mesosphere to produce an e-book called "Deploying Microservices and Containers with ACS and DC/OS."

We love hearing from our customers about how they are using containers on Azure and the benefits it brings to their application development lifecycle. BioCatch, a startup based out of Israel, builds real time fraud prevention software that went from a PoC into production on ACS in a matter of weeks. Stories like this show the power of container-based applications and get us excited about the possibilities – we hope to hear from you, too.

You can easily get started deploying an Azure Container Service cluster using the Azure portal or the recently released Azure CLI 2.0 by using the az acs command. For example, this tutorial shows you how to deploy an ACS DC/OS cluster with a few simple Azure CLI 2.0 commands.
Quelle: Azure

Best Practices: How to deploy Azure Site Recovery Mobility Service

With large enterprises deploying Azure Site Recovery (ASR) as their trusted Disaster Recovery (DR) solution for application-aware DR, their DR architects have asked us about the best practices to be followed while deploying ASR in production environments. Given ASR’s multi-VM consistency promise to provide full application recovery on Microsoft Azure, the mobility service is a critical piece in the VMware to Azure scenario. In this blog, we take a look at the various options to deploy the ASR mobility service during different stages of a production ASR rollout.

Deployment Considerations

At a high level the challenges that we hear about day to day can be summarized as shown in the below table.

Firewall and Network Security

My organization has tight security policies. It does not allow me to change servers’ firewall settings to allow push install of ASR mobility service on the servers we want to protect.

Credential Management

My organization’s password expiry policy forces application owners to change the administrator password periodically. This causes ASR workflows that install and upgrade the mobility service to fail. Can I manage ASR mobility service deployment using software deployment tools (like System Center Configuration Manager) so that I don’t have to worry about these credentials?
As a hosting service provider, I want to provide DR as a Service to my customers, and I don’t like providing the customer’s virtual machine’s credentials to ASR, for it to push the mobility service. Can I manage the ASR mobility service initial deployment and upgrades using software deployment tools?

At Scale Deployment

My ASR proof of concept is done, and now we are starting a full-fledged production rollout. I have thousands of servers to protect. Is there a solution other than the push install service that we can use to deploy the ASR mobility service to all our production servers?
I want to pre-install the ASR mobility service during our planned software maintenance window, but replication should not start immediately. I want to start replicating virtual machines in batches to ensure that the initial replication traffic does not clog our network, and also finishes in a predictable desired timeframe.

Deployment Best Practices

Our goal here at Microsoft is to make Azure Site Recovery easy to deploy and use. We know that each enterprise environment is different and needs a customized solution to suite its security and audit needs. Therefore, we have support for multiple different ways in which you can install the ASR mobility service on the servers you want to protect. 

Note: All the ASR mobility service installation methods listed below can be used to deploy the mobility service on supported Microsoft Windows and Linux operating systems.

Push install mobility service during Enable Protection

Push install is the easiest method to deploy the ASR mobility service on the virtual machines you want to protect. This method is best suited for a proof of concept demonstration and deployment in production environments where firewall and network security rules are less stringent. To perform push install, your environment needs to meet the pre-requisites mentioned in our Prepare for push install documentation.

Install mobility service using software deployment tools

Enterprises use software deployment tools like System Center Configuration Manager (SCCM), Windows Server Update Service (WSUS), or other third party software deployment tools to push software on servers in their environment. ASR allows out-of-band installation of the mobility service via these software deployment tools. The documentation page Automate Mobility Service installation using software deployment tools, provides you instructions and scripts that allows you to use your favorite software deployment tool to install the ASR mobility service in your production environment – the documentation uses SCCM as an example.

This method is best suited for a production rollout of Azure Site Recovery and gives you the following advantages:

No need to add firewall exceptions
Deploy at enterprise scale
No need to manage guest (protected virtual machine) credentials

Install mobility service using Azure Automation Desired State Configuration (DSC)

In organizations that heavily use Azure services in their production environment, Azure Automation Desired State Configuration can be used to deploy and manage the deployment of ASR mobility service. The documentation page Deploy the Mobility Service with Azure Automation DSC for replication of VM talks in detail about how to use Azure Automation DSC to install and manage the lifecycle of the ASR mobility service.

This method is best suited for a production rollout of Azure Site Recovery assuming you use Microsoft Azure Services to manage your IT infrastructure, and gives you the following advantages:

No need to add firewall exceptions
Deploy at enterprise scale
No need to manage guest (protected virtual machine) credentials
Enforces software configuration on your protected servers

Manual install (command line and GUI Based)

The ASR mobility service can be installed manually via command line or GUI. If you plan to protect 5-10 servers, and don’t have a software deployment tool being used in your organization, then you can use the manual install method. The manual install method can also be used for proof of concept deployments. The command line install method can be used to create scripts to automate installations in your production environment. You can find both of these methods documented at Install Mobility Service using command line and Install Mobility Service using GUI.

Closing Notes

The below decision tree helps to summarize how to choose the best deployment option that suites your environment.

You can check out additional product information and start replicating your workloads to Microsoft Azure using Azure Site Recovery today. You can use the powerful replication capabilities of Site Recovery for 31 days at no charge for every new physical server or virtual machine that you replicate. Visit the Azure Site Recovery forum on MSDN for additional information and to engage with other customers, or use the ASR UserVoice to let us know what features you want us to enable next.

Azure Site Recovery, as part of Microsoft Operations Management Suite, enables you to gain control and manage your workloads no matter where they run (Azure, AWS, Windows Server, Linux, VMware, or OpenStack) with a cost-effective, all-in-one cloud IT management solution. Existing System Center customers can take advantage of the Microsoft Operations Management Suite add-on, empowering them to do more by leveraging their current investments. Get access to all the new services that OMS offers, with a convenient step-up price for all existing System Center customers. You can also access only the IT management services that you need, enabling you to on-board quickly and have immediate value, paying only for the features that you use.
Quelle: Azure

Backup Managed Disk VMs using Azure Backup

Last week we announced the general availability of Managed Disks. Managed Disks are Azure Resource Manager (ARM) resources, that can be deployed via templates to create thousands of Managed Disks without worrying about creating storage accounts or specifying disk details. Backup of Managed disk VMs against accidental deletions and corruptions resulting from human errors is a critical capability for customers of all sizes. With Azure Backup service, you get key enterprise features like backup, restore, policy based management, backup alerts, job monitoring, instant data recovery without deploying any infrastructure in your tenant environment. You get the ability to backup Managed Disk VMs directly from VM management blade and the user experience is consistent with backup of VMs attached to Standard or Premium Unmanaged Disks. 

Value Proposition

Azure Backup’s cloud-first approach provides:

Freedom from infrastructure: No need to deploy any infrastructure to backup VMs
Eliminate backup storage management with bottomless Recovery Services vault.
Pay as you go model with no egress costs for restores.
Self-service backup and restore

Key features

Application Consistent backups for Windows Azure VMs and File-system consistent backup for Linux Azure VMs without the need to shutdown VM.
Policy Based Management:  Azure Backup allows you to specify the backup schedule as well as retention policy of backups.  The service handles periodic backups as well as pruning of recovery points beyond the configured retention period. 
Long Term Retention of backup data for years even beyond the lifecycle of the VM.
Full VM and Disk restore:  In case your VM is corrupted and needs replacement or want to simply make a copy of the VM you can do so with full VM or disk restore. 
Instant Data Recovery:  With Instant Data Recovery, you can restore individual files and folders within the VM instantly without provisioning any additional infrastructure, and at no additional cost. Instant Restore provides a writeable snapshot of a recovery point that you can quickly mount, browse, recover files/folders by simply copying them to a destination of your choice. These snapshots even allow you to open application files such as SQL, MySQL directly from cloud recovery point snapshots as if they are present locally and attach them to live application instances, without having to copy them.
Role Based Access:  You can limit the access to backup data in the Recovery Services vault using Role Based Access controls. Azure Backup supports Backup Contributor, Backup Operator and Backup Reader roles at a vault level.
Monitoring and Alerting: You can monitor your backup and restore jobs from the Recovery Services Vault dashboard.   In addition, they can also configure email alerts for job failures.

Customers can backup data to Recovery Services Vault in all public Azure regions, including Canada, UK, and West US2.

Getting started

To get started, enable backup with a few steps:

Select a virtual machine from the Virtual machines list view. Select Backup in the Settings menu.
Create or select a Recovery Services Vault:  The vault maintains backups in a separate storage account with its own lifecycle management. 
Create or select a Backup Policy

Watch the video below to instantly recover files from an Azure VM (Windows) backup.

Watch the video below to instantly recover files from an Azure VM (Linux) backup.

The instant restore capability will be available soon for users who are protecting their Linux VMs using Azure VM backup. If you are interested in being an early adopter and want to provide valuable feedback, please let us know at linuxazurebackupteam@service.microsoft.com. Watch the video below to know more.

Related links and additional content

Want more details? Check out Azure Backup documentation and Managed Disk Blog
Learn more about Azure Backup
Need help? Reach out to Azure Backup forum for support
Sign up for a free Azure trial subscription
Tell us how we can improve Azure Backup by contributing new ideas and voting up existing ones.
Follow us on Twitter @AzureBackup for the latest news and updates

 

 
Quelle: Azure

Microsoft Enterprise Services Tips to using SQL Data Warehouse effectively

Azure SQL Data Warehouse is a SQL-based fully managed, petabyte-scale cloud solution for data warehousing. With tight integration between other Azure products, SQL Data Warehouse represents the backbone of any major cloud data warehousing solution in Azure. With decoupled compute and storage, elastic scale-out functionality, and the ability to pause and resume to meet demand, SQL Data Warehouse gives customers the flexibility, cost-savings, and performance they need, right when they need it.

Microsoft Enterprise Services (ES) is one of the largest consulting and support businesses in the world. Microsoft ES operates across more than 100+ subsidiaries around the globe operating in spaces such as cloud productivity, mobility solutions, adoption services, and more. With the speed and scale at which clients’ operations transform and grow, it is paramount that ES stays ahead of the curve to meet future demand.

Right data at the right time allows ES to make the best decision about where to fund resources and how to best serve their customers. Traditional data warehouse reporting stacks took far too long to deliver reports and were far too inflexible to change models. Adding to the costs of complexity, maintenance, and scaling to match growing data, their traditional on-premise data warehouse was producing less value than the cost of upkeep and development, distracting from the core business value of delivering insights.

The move to a modern data warehouse solution was becoming readily apparent. ES analytics split their workload into data processing and distribution. Core requirements for ES included easy scalability, high IO, multi-terabyte storage, row level data security, and support of more than 200 concurrent users. Working through the host of Azure service offerings, ES landed on a solution with Azure SQL Data Warehouse as their data processing and ad-hoc analysis workload with IaaS SQL Server 2016 Always On instances as their data distribution layer.

Implementation

The implementation for ES at a high level takes audit and historical data from a variety of sources on-premise which first land into Azure Storage Blobs. Polybase is then used to load data in parallel quickly into the Azure SQL Data Warehouse where it is then processed and transformed into dimension and fact tables. Afterwards, these dimension and fact tables are moved into Analysis Services and SQL Server IaaS instances to support quick and highly concurrent access a variety of business users. Across this solution, Azure Data Factory acts as the orchestrating ELT framework, allowing for a single interface to control the data flow for the majority of the pipeline.

8 Tips for SQL Data Warehouse

Putting together a solution like this, while performant, is not a trivial task. Listed below is some guidance straight from the ES team on designing solutions with Azure SQL Data Warehouse:

1. Stage the data in Azure SQL DW:

One of the guiding principles of our warehouse has been to stage the data in its native form, i.e. the way it is available in source. There are various reasons as to why we persist a copy of the source like performance, data quality, data persistence for validation, etc… because of staging the data, we are able to distribute data as per our needs and ensure we have minimal data skew. Rarely, if at all, have we used round robin mechanism to store data.

2. Arriving at a common distribution key:

The data staged into the Azure SQL Data Warehouse instance was ingested from a 3NF data source. This helped us to slightly change the schema from source and include the base table’s foreign key as part of all tables, which are distributed on the same key. During our fact load we join the tables on these set of keys thereby minimizing all DMS operations and in some cases, no DMS operations occur. Hence this gives us an edge in terms of performance. As an example, the data in source systems have one to many relationships between tables. However, in our SQL DW we have inserted a common distribution key across all tables, based on business logic and because this key gets loaded when ELT runs.

However, we would recommend checking the data skew before going ahead with this approach as the distribution key must be chosen completely based on the skew that one may see. When data skew is high or the joins are not compatible we create an interim table, which is distributed on the same key as the other join table. We us CTAS to accomplish this which incurs one DMS operation to get all keys but improves performance when there are complex joins.

3. Vertical Partitioning of Wide Tables:

We had a row size limitation of 32KB in Azure SQL DW. Since we had several wide tables with 150+ columns and many with varchar(4000) we came up with an approach to vertically partition the table on the same distribution key. This helped us to overcome the challenge of 32KB and at the same time provide the required performance while joining the two tables as the distribution key was the same.

Note: SQL Data Warehouse now supports 1MB wide rows

4. Use the right resource class:

In several cases we had complex facts that would need more resources (memory and CPU) to speed-up the process of fact loads. Not just facts, we also had dimensions which had complex business rules and type 2 kind of implementations. We designed our ETL in such a way that lesser complex facts and dimensions would run on smallrc resource class providing for more parallelism, whereas the more complex facts which would need more resources would run using largerc resource class.

5. Use the primary key as distribution column for master tables:

In the source from where we ingest data into SQL DW, we have many master tables that we use in SQL DW to look up these tables for building our facts. In such a case, we have made these tables with reasonable amount of data (>1 million rows) being distributed on the primary key, which is a unique integer. This has given us the advantage having even data distribution (minimal to no data skew), thereby making our look up queries really fast.

6. Using Dynamic Scale up and Scale down for saving costs:

Our ELTs using ADF are designed in such a way that prior to the scheduled ELT kick off, we scale up our instance from 100DWU to 600DWU. This has led to huge cost savings. Our ELT runs for nearly 4-5 hours during this time and the DWU usage is capped at 600 DWU. During month end when there is a faster need for processing and businesses need data faster, we have the option of scaling to 1000 DWU. All this is done as part of our ELT’s and no manual intervention is needed.

7. Regular Maintenance:

In our case, we included updating statistics and index rebuilds as part of the ELT process. No sooner the dimension and fact load is completed, we check for all the CCI’s where the fragmentation is > 5% and rebuild the index. Similarly, for the key tables, we are updating statistics to ensure best performance.

8. Leverage SQL Server 2016 for data marts:

Azure SQL DW is the primary data processing engine, whereas we chose to have SQL Server 2016 running on DS 14 IAAS VM’s are our primary source of data distribution. This has enabled us to leverage high concurrency provided by SQL Server and use the power of Azure SQL DW for processing. We have close to 1000+ users who would be using the data provisioned from Azure SQL DW.

Using Azure SQL Data Warehouse as part of their solution, Microsoft Enterprise Services was able to reduce run times by up to 40% and provide insights to business users with reduced latency.

If you have not already explored this fully managed, petabyte scale cloud data warehouse service, learn more at the links below.

Learn more

What is Azure SQL Data Warehouse?
SQL Data Warehouse best practices
Video library
MSDN forum
Stack Overflow forum
Quelle: Azure

Register today for SMB Live

This year’s SMB Live events, coming to 22 cities across the United States, include a day of new in-depth and hands-on marketing content, and a day of 200 & 300-level technical training to modernize your organization’s technical skills and marketing capabilities. You’ll leave SMB Live with the tools you need for cloud business growth and to be prepared to help modernize your SMB customers to Azure Hybrid IT, Enterprise Mobility + Security, Microsoft Dynamics 365, and more!

Space at these SMB Live events is limited. Register today to secure your spot at the event in the city nearest you.

Register Today!

Day 1

Are you taking full advantage of the hybrid cloud opportunity with your customers? Is Azure coming up more and more in your customer conversations? Are your customers looking for a secure and efficient cloud solution? Join our experts at this free, one-day SMB Live session and see how you too can be that market leader.

What you will take away from this training:

Simple ways you can create a differentiated Cloud offering, including developing repeatable solutions to grow your profitability by learning how successful partners are:

building a Cloud Infrastructure or Azure Hybrid IT practice to protect customers, while increasing recurring revenues. 
leading with security solutions leveraging Enterprise Mobility + Security (EMS)

How to align your sales process to the new Cloud customer buying behaviors and proven ways to maximize your revenue from your existing customer base. 
See how new Microsoft business productivity tools can help you drive new and recurring profits. Hear what’s new with Windows 10 Enterprise Subscription E3 for CSP, Microsoft Dynamics 365 for Financials, SQL & Windows Server 2016 and Office 365 E5 (Skype Telephony).

What Organizations should attend:

Experienced Cloud partners who are interested in developing a new or current cloud practice beyond Office 365 with Microsoft Azure Hybrid IT solutions.

Who from the organization should attend Day 1?

Business leads, sales, marketing, and/or technical business decision-makers.

Register Today! 

Day 2

Learn how to differentiate yourself from your competitors using Azure Hybrid cloud solutions and scalable workloads. This one-day technical hands on training will provide you with the tools and skills necessary to develop and manage your cloud practice; easily spec and estimate monthly Azure costs and learn how powerful Azure Migration offerings from other partners can help move your customers to the cloud.

What you will take away from this training:

Technical Readiness – How to develop and deploy core Azure workloads with hands-on training configuring Business Continuity solutions and Business Operations
Accurate Azure Price Forecasting – Learn how to present an AZURE solution with the use of detailed pricing structures and scenarios.
Understanding the Azure Solution Profitability – Review and understand the profitability potential with a view to near and long term customer relationships.
New and Existing Customer Assessment and Migration Resources – Resources to help you analyze your current customer base and begin migrations immediately to Azure.

What Organizations should attend:

Experienced Cloud partners who are interested in developing a new or current cloud practice beyond Office 365 with Microsoft Azure Hybrid IT solutions

Who from the organization should attend day 2?

Business / Technical Decision Makers, Pre / Post sales Technical Roles. 

Register Today!
Quelle: Azure