Now available for preview: Workload importance for Azure SQL Data Warehouse

Azure SQL Data Warehouse is a fast, flexible and secure analytics platform for enterprises of all sizes. Today we are announcing the preview availability of workload importance on the Gen2 platform to help customers manage resources more efficiently. Workload importance gives data engineers the ability to use importance to classify requests. Requests with higher importance are guaranteed quicker access to resources which helps meet SLAs.

“More with less” is often the motto when it comes to operating data warehousing solutions. The ability to easily scale up compute resources gives data engineers tremendous flexibility. However, when there is budget pressure and scaling down is required, problems can arise.  Workload importance allows high business value work to meet SLAs in a shared environment with fewer resources.

An example of workload importance is shown below. The CEO’s request was submitted last and classified with high importance. Because the CEO’s request has high importance, it is granted access to resources before the Analyst requests allowing it to complete sooner.

Get started now classifying requests with importance

Classifying requests is done with the new CREATE WORKLOAD CLASSIFIER syntax. Below is an example that maps the login for the ExecutiveReports role to ABOVE_NORMAL importance and the AdhocUsers role to BELOW_NORMAL importance. With this configuration, members of the ExecutiveReports role have their queries complete sooner because they get access to resources before members of the AdhocUsers role.

CREATE WORKLOAD CLASSIFIER ExecReportsClassifier
WITH (WORKLOAD_GROUP = 'mediumrc'
,MEMBERNAME = 'ExecutiveReports'
,IMPORTANCE = above_normal);

CREATE WORKLOAD CLASSIFIER AdhocClassifier
WITH (WORKLOAD_GROUP = 'smallrc'
,MEMBERNAME = 'AdhocUsers'
,IMPORTANCE = below_normal);

For more information on workload importance refer to the Classification and Importance overview topics in the documentation. Check out the CREATE WORKLOAD CLASSIFIER doc as well.

See workload importance in action in the below videos:

Workload Importance concepts
Workload Importance scenarios

Next Steps

To get started today, create an Azure SQL Data Warehouse.
For feature requests, please vote on our UserVoice.
To stay up-to-date on the latest Azure SQL Data Warehouse news and features, follow us on Twitter @AzureSQLDW.

Quelle: Azure

Achieve more with Microsoft Game Stack

This blog post was authored by Kareem Choudhry, Corporate Vice President, Microsoft Gaming Cloud.

Microsoft is built on the belief of empowering people and organizations to achieve more – it is the DNA of our company. Today we are announcing a new initiative, Microsoft Game Stack, in which we commit to bringing together Microsoft tools and services that will empower game developers like yourself, whether you’re an indie developer just starting out or a AAA studio, to achieve more.

This is the start of a new journey, and today we are only taking the first steps. We believe Microsoft is uniquely suited to deliver on that commitment. Our company has a long legacy in games – and in building developer-focused platforms.

There are 2 billion gamers in the world today, playing a broad range of games, on a broad range of devices. There is as much focus on video streaming, watching, and sharing within a community as there is on playing or competing. As game creators, you strive every day to continuously engage your players, to spark their imaginations, and inspire them, regardless of where they are, or what device they’re using. Today, we’re introducing Microsoft Game Stack, to help you do exactly that.

What exactly is Microsoft Game Stack?

Game Stack brings together all of our game-development platforms, tools, and services—such as Azure, PlayFab, DirectX, Visual Studio, Xbox Live, App Center, and Havok—into a robust ecosystem that any game developer can use. The goal of Game Stack is to help you easily discover the tools and services you need to create and operate your game.

The cloud plays a critical role in Game Stack, and Azure fills this vital need. Azure provides the building blocks like compute and storage, as well as cloud-native services from machine learning and AI, to push notifications and mixed reality spatial anchors. Azure is already available in 54 regions globally, including China, and continues to invest in building highly secure and sustainable cloud infrastructure and additional services for game developers. Azure’s global scale is what will give Project xCloud streaming technology the scale to deliver a great gaming experience for players worldwide, regardless of their device and location.

Already with Azure, companies like Rare, Ubisoft, and Wizards of the Coast are hosting multiplayer game servers, safely and securely storing player data, analyzing game telemetry, protecting their games from DDOS attacks, and training AI to create more immersive gameplay.

While Azure is part of Game Stack, it’s important to call out that Game Stack is cloud, network, and device agnostic. And we’re not stopping here.

What’s new?

The next piece of Game Stack is PlayFab, a complete backend service for building and operating live games. A year ago, we welcomed PlayFab into Microsoft through an acquisition. Today we’re excited to announce we are bringing PlayFab into the Azure family. Together, Azure and PlayFab are a powerful combination: Azure brings reliability, global scale, and enterprise-level security; PlayFab provides Game Stack with managed game-development services, real-time analytics, and LiveOps capabilities. Last fall, we saw what these two platforms can do together with PlayFab Multiplayer Servers, which allows you to safely launch and scale up multiplayer games by dynamically hosting your servers with Azure cloud compute.

To quote PlayFab’s co-founder James Gwertzman, “Modern game creators are less like movie directors, and more like cruise directors. Long-term success requires engaging players in a continuous cycle of creation, experimentation, and operation. It’s no longer possible to just ship your game and move on.” This is why a year ago, we welcomed PlayFab into Microsoft through an acquisition. PlayFab supports all major devices, from iOS and Android, to PC and Web, to Xbox, Sony PlayStation, and Nintendo Switch; and all major game engines, including Unity and Unreal. PlayFab will also continue to support all major clouds going forward.

Today we’re also excited to announce five new PlayFab services in preview.

In public preview today:

PlayFab Matchmaking: Powerful matchmaking for multiplayer games, adapted from Xbox Live matchmaking, but now available to all games and all devices.

In private preview today (contact us to join the preview):

PlayFab Party: Voice and chat services, adapted from Xbox Party Chat, but now available to all games and for all devices. Party leverages Azure Cognitive Services for real-time translation and transcription to make games accessible to more players.
PlayFab Game Insights: Combines robust real-time game telemetry with game data from multiple other sources to measure your game’s performance and create actionable insights. Powered by Azure Data Explorer, Game Insights will offer connectors to existing first- and third-party data sources including Xbox Live.
PlayFab Pub Sub: Subscribe your game client to messages pushed from PlayFab’s servers via a persistent connection, powered by Azure SignalR. This enables scenarios such as real-time content updates, matchmaking notifications, and simple multiplayer gameplay.
PlayFab User Generated Content: Engage your community by allowing players to create and safely share user generated content with other players. This technology was originally built to support the Minecraft marketplace.

Growing the Xbox Live community

Another major component of Game Stack is Xbox Live. Over the past 16 years, Xbox Live has become one of the most vibrant and engaged gaming communities in the world. It is also a safe and inclusive network that has broken down boundaries in how gamers connect across devices.

Today, we’re excited for Xbox Live to become part of Microsoft Game Stack, providing identity and community services. Under Game Stack, Xbox Live will expand its cross-platform capabilities, as we introduce a new SDK that brings this community to iOS and Android devices.

Mobile developers will now be able to reach some of the most highly engaged and passionate gamers on the planet with Xbox Live. These are just a few of the benefits for mobile developers:

Trusted Game Identity: With the new Xbox Live SDK, developers can focus on creating great games and leverage Microsoft‘s trusted identity network to support log-in, privacy, online safety, and child accounts. 
Frictionless Integration: New a la carte service offerings and no Xbox Live certification pass give mobile developers flexibility in how they build and update their games. Developers just use the services that best fit their needs.
Vibrant Gaming Community: Reach Xbox Live’s growing community and connect gamers across a multitude of platforms. Find creative ways to enable achievements, Gamerscore, and “hero” stats, which have their own out-of-game experience, to keep gamers engaged.

Other Game Stack components

Other components of Game Stack include Visual Studio, Mixer, DirectX, Azure App Center, Visual Studio, Visual Studio Code, and Havok. In the coming months, as we work to improve and grow Game Stack, you’ll see deeper connections between these services as we unify them to work more seamlessly together.

As an example of how this integration is already underway, today we’re bringing together PlayFab and these Game Stack components:

App Center: Crash log data from App Center is now connected to PlayFab, allowing you to better understand and respond to problems in your game in real-time by tying crash logs back to individual player profiles.
Visual Studio Code: With PlayFab’s new plug-in for Visual Studio Code, editing and updating Cloud Script just got a lot easier.

Create your world today and achieve more

As we expand our focus to the cloud, the nature of the platform may be changing, but our commitment to empower game developers like yourself is unwavering, and we’re looking forward to the journey ahead with Microsoft Game Stack. Our teams are inspired and excited by the possibilities as we start to pull together all these great services and technologies. Please be sure to share your feedback with us as we go, so we can help you achieve more. If you’re at GDC, stop by the Microsoft booth in the South Hall of the Moscone Center to try out many of the new services, and to learn more about the exciting opportunities ahead.
Quelle: Azure

Simplifying your environment setup while meeting compliance needs with built-in Azure Blueprints

I’m excited to announce the release of our first Azure Blueprint built specifically for a compliance standard, the ISO 27001 Shared Services blueprint sample which maps a set of foundational Azure infrastructure, such as virtual networks and policies, to specific ISO controls.

Microsoft Azure leads the industry with over 90 compliance offerings. Azure meets a broad set of international and industry-specific compliance standards, such as General Data Protection Regulation (GDPR), ISO 27001, HIPAA, PCI, SOC 1 and SOC 2, as well as country-specific standards, including FedRAMP and other NIST 800-53 derived standards, Australia IRAP, UK G-Cloud, and Singapore MTCS. Many of our customers have expressed their interest in being able to leverage and build upon our internal compliance practices for their environments with a service that maps compliance settings automatically.

To help our customers simplify the creation of their environments in Azure while successfully interpreting US and international governance requirements, we are announcing a series of built-in Blueprints Architectures that can be leveraged during your cloud-adoption journey. Azure Blueprints is a free service that helps customers deploy and update cloud environments in a repeatable manner using composable artifacts such as policies, deployment templates, and role-based access controls. This service is built to help customers set up governed Azure environments and can scale to support production implementations for large-scale migrations.

The ISO 27001 Shared Services Blueprint is already available to your Azure tenant. Simply navigate to the Blueprints page, click “Create blueprint”, and choose the ISO27001 Shared Services blueprint from the list.

The ISO 27001 blueprint is designed to help you deploy production ready, secure end-to-end solutions in one click and includes:

Hardened infrastructure resources: Azure Resource Manager templates are used to automatically deploy the components of the architecture into Azure by specifying configuration parameters during setup. The infrastructure components include Azure Firewall, Active Directory, Key Vault, Azure Monitor, Log Analytics, Virtual Networks with subnets, Network Security Groups, and Role Based Access Control definitions. Additionally, these resources can be locked by Blueprints as a security measure to protect the consistency of the defined blueprint and the environment it was designed to create.
Policy controls: Set of Azure policies that help provide real-time enforcement, compliance assessment, and remediation.
Proven virtual datacenter architectures: The infrastructure resources provided are based on the Microsoft approved virtual datacenter (VDC) architectures which take into consideration scale, performance, security, and governance.
Security and compliance controls: You still benefit from all the controls for which Microsoft is responsible as your cloud provider, and now this blueprint helps you configure a number of the remaining controls to meet ISO 27001 requirements.
Documentation: Step by step deployment guide outlining the shared services infrastructure and the policy control mapping matrix.
Migration runway: Provides a prescriptive set of instructions for deploying an Azure recommended foundation to accelerate migrations via the Azure migration center.

At Microsoft, we are committed to helping our customers leverage Azure in a secure and compliant manner. Over the next few months you will continue to see new built-in blueprints released for HITRUST, PCI DSS, UK National Health Service (NHS) Information Governance (IG) Toolkit, FedRAMP, and Center for Internet Security (CIS) Benchmark. If you would like to participate in any early previews please sign up, or if have a suggestion for a compliance blueprint, please share it via the Azure Governance Feedback Forum.

Learn more about the Azure ISO 27001 Blueprints.
Quelle: Azure

Monitoring on HDInsight Part 1: An Overview

Azure HDInsight offers several ways to monitor your Hadoop, Spark, or Kafka clusters. Monitoring on HDInsight can be broken down into three main categories:

Cluster health and availability
Resource utilization and performance
Job status and logs

Two main monitoring tools are offered on Azure HDInsight, Apache Ambari which is included with all HDInsight clusters and optional integration with Azure Monitor logs, which can be enabled on all HDInsight clusters. While these tools contain some of the same information, each has advantages in certain scenarios. Read on for an overview of the best way to monitor various aspects of your HDInsight clusters using these tools.

Cluster health and availability

Azure HDInsight is a high-availability service that has redundant gateway nodes, head nodes, and ZooKeeper nodes to keep your HDInsight clusters running smoothly. While this ensures that a single failure will not affect the functionality of a cluster, you may still want to monitor cluster health so you are alerted when an issue does arise. Monitoring cluster health refers to monitoring whether all nodes in your cluster and the components that run on them are available and functioning correctly. Ambari is the recommended way to monitor the health for any given HDInsight cluster. You can learn more about monitoring cluster availability using Ambari in our documentation, “Availability and reliability of Apache Hadoop clusters in HDInsight.”

Ambari portal view showing the status of all components on a head node

Cluster resource utilization and performance

To maintain optimal performance on your cluster, it is essential to monitor resource utilization. This can be accomplished using Ambari and Azure Monitor logs.

With Ambari

Ambari is the recommended way to monitor utilization across the whole cluster. The Ambari dashboard shows easily glanceable widgets that display metrics such as CPU, network, YARN memory, and HDFS disk usage. The “Hosts” tab shows metrics for individual nodes so you can ensure the load on your cluster is evenly distributed. The “YARN Queue Manager” is also accessible through Ambari. This allows you to manage the capacity of each of your job queues to see how jobs are distributed between them and whether any jobs are resource constrained. Read more about using Ambari to monitor cluster performance in our documentation, “Monitor cluster performance.”

The Ambari Portal dashboard that shows the utilization of your entire cluster at a glance

With Azure Monitor logs

You can monitor resource utilization at the virtual machine (VM) level using Azure Monitor logs. All VMs in an HDInsight cluster push performance counters into the Perf table in your Log Analytics workspace, including CPU, memory, and disk usage. Like any other Log Analytics table, you can query the Perf table, create visualizations with view designer, and configure alerts. One of the key benefits of Log Analytics is that you can push metrics and logs from multiple HDInsight clusters to the same Log Analytics workspace, allowing you to monitor multiple clusters in one place. You can read more about working with performance data in Azure Monitor logs by visiting our documentation, “View or analyze data collected with Log Analytics log search.”

Job status and logs

Another key part of monitoring HDInsight clusters is monitoring the status of submitted jobs and viewing relevant logs to assist with debugging. You may want to know how many jobs are currently running or when a job fails.

With Azure Monitor logs

The recommended way to do this on Azure HDInsight is through Azure Monitor logs. HDInsight clusters emit workload-specific logs from the OSS components and metrics with each line being a record. An example of this would be the number of apps pending, failed, and killed for Spark/Hadoop clusters and incoming messages for Kafka clusters. You can query the tables and set up alerts when certain metrics meet your defined thresholds. For example, you could set up an alert that fires and sends you an email or takes some other action whenever a Spark job fails.

HDInsight monitoring solutions

Workload-specific HDInsight monitoring solutions that build on top of the Azure Monitor logs integration are also available. These solutions are premade dashboards that contain visualizations for the aforementioned workload metrics. For example, the Spark solution shows graphs of metrics like pending, failed, and killed apps over time. Because these solutions are backed by a Log Analytics workspace, the visualizations show data for all clusters that emit metrics to the workspace. In result, you can see visualizations of these workload metrics from multiple clusters of the same type and all in one place.

The HDInsight Spark monitoring solution

With Ambari

You can also view workload information from Spark/Hadoop clusters in the YARN ResourceManager UI, which is accessible via the Ambari portal.  The YARN UI shows detailed information about all job submissions and provide a link to the capacity scheduler, where you can view information about your job queues. You can also access raw ResourceManager log files through the Ambari portal if you need to further debug jobs.

Try HDInsight now

Between Apache Ambari and Azure Log Analytics integration, HDInight offers comprehensive tools for monitoring all aspects of your HDInsight cluster. We hope you will take full advantage of monitoring on HDInsight and we are excited to see what you will build with Azure HDInsight. Read this developer guide and follow the quick start guide to learn more about implementing these pipelines and architectures on Azure HDInsight. Stay up-to-date on the latest Azure HDInsight news and features by following us on Twitter #AzureHDInsight and @AzureHDInsight. For questions and feedback, reach out to AskHDInsight@microsoft.com.

About HDInsight

Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics that enables customers to easily run popular open source frameworks including Apache Hadoop, Spark, Kafka, and others. The service is available in 36 public regions and Azure Government and National Clouds. Azure HDInsight powers mission-critical applications in a wide variety of sectors and enables a wide range of use cases including ETL, streaming, and interactive querying.
Quelle: Azure

Simplify disaster recovery with Managed Disks for VMware and physical servers

Azure Site Recovery (ASR) now supports disaster recovery of VMware virtual machines and physical servers by directly replicating to Managed Disks. Beginning in March 2019 and moving forward, all new protections have this capability available on the Azure portal. In order to enable replication for a machine, you no longer need to create storage accounts. You can now write replication data directly to a type of Managed Disk. The choice of Managed Disk type should be based on the data change rate on your source disks. Available options are Standard HDD, Standard SSD and Premium SSD.

Please note, this change will not impact the machines which are already in a protected state. They will continue to replicate to storage accounts. However, you can still choose to use Managed Disks at the time of failover by updating the settings in compute and network blade.

There are benefits in writing to Managed Disks:

Hassle free management of capacity on Microsoft Azure: You don’t need to track and manage multiple target storage accounts anymore. ASR will create the replica disks at the time of enabling replication. An Azure Managed Disk is created for every virtual machine (VM) disk at on-premises. This is managed by Azure.
Seamless movement between different types of Managed Disks: After enabling protection, if the data change rate or churn pattern on your source disk changes, you do not need to disable and enable the replication with Managed Disks. You can simply choose to switch the type of Managed disk in order to handle the new data change rate. However, once you change the Managed Disk type, please be sure that you wait for fresh recovery points to be generated if you need to do test failover or failover post this activity.

The replication architecture for ASR is refined with replication logs first being uploaded to a cache storage account in Azure. These logs are processed by ASR and then pushed into the replica Managed Disk in Azure. Snapshots are created on these Managed Disks at a frequency which is applied by replication policy at the time of enabling replication. You can find the name of replica and target Managed Disks on the disks blade of the replicated item. At the time of failover, one of the recovery points on replica Managed Disk is chosen by the customer. This recovery point is used to create the target Managed Disk in Azure which is attached to the VM when it is brought up.

It is recommended to use the replication option LRS for cache storage. Since cache account is standard storage and only holds temporary data, it is not required to have multiple cache storage accounts in a recovery services vault.

Get started with ASR today. Support for writing to Managed Disks is available in all Azure regions. It will be released on national clouds soon!

Related links and additional content

Tag Managed Disks in Azure for billing
Learn more about pricing of Managed Disks
Learn more about Azure Site Recovery churn limits
Set up disaster recovery for VMware or physical machines to Azure

Quelle: Azure

Spinning up cloud-scale analytics is even more compelling with Talend and Microsoft

Special thanks to Lee Schlesinger and the Talend team for their contribution to this blog post. Following the significant announcement around the continued price-performance leadership of Azure Data Warehouse in February 2019, Talend announced support of Stitch Data Loader for Azure SQL Data Warehouse. Stich Data Loader is Talend’s recent addition to its offering portfolio small and mid-market customers. With Stitch Data Loader, customers can load 5 million rows/month into Azure SQL Data Warehouse for free or scale up to an unlimited number of rows with a subscription.

All across the industry, there is a rapid shift to the cloud. Utilizing fast, flexible, and secure cloud data warehouse is an important first step in that journey. With Microsoft Azure SQL Data Warehouse and Stitch Data Loader companies can get started faster than ever. The fact that ADW can be up to 14x faster, and 94 percent less expensive than similar options in the marketplace, should only help further accelerate adoption of cloud scale analytics by customers of all sizes.

Building pipelines to the cloud with Stitch Data Loader

The Stitch team built the Azure SQL Data Warehouse integration with the help of Microsoft engineers. The solution leverages Azure Blob Storage and PolyBase to get data into the Azure cloud and ultimately loaded to SQL Data Warehouse. We take care of all issues with data type transformation between source and destination, schema changes, and bulk loading.

To start moving data, just specify your host address and database name and provide authentication credentials. Stitch will then start loading data from all of your sources in minutes.

Stitch Data Loader enables Azure SQL Data Warehouse users to analyze data from more than 90 data sources, including databases, SaaS tools, and ad networks. We also sponsor and integrate with the Singer open source ETL project, which makes it easy to get additional or custom data sources into Azure SQL Data Warehouse.

Stitch’s destination switching feature also makes it easy for existing Stitch users to take their existing integrations and start loading them into Azure SQL Data Warehouse right away.

Going further with Talend Cloud and Azure SQL Data Warehouse

What if you’re ready to scale out your data warehousing efforts and layer on data transformation, profiling, and quality? Talend Cloud offers many more sources as well as more advanced data processing and data quality features that are available within the ADW and the Azure Platform. With over 900 connectors available, you’ll be able to move all your data, no matter the format or source. With data preparation and additional security features built-in, you can get Azure-ready in no time.

Take Uniper for instance. Using Azure and Talend Cloud, they built a cloud-based data analytics platform to integrate over 100 data sources including temperature and IoT sensors, from various external and internal sources. They constructed the full flow of business transactions — spanning market analytics, trading, asset management, and post-trading — while enabling data governance and self-service, resulting in reduced integration costs by 80 percent and achieving ROI in 6 months.

What’s next?

Start your free trial of Stitch today and load data into Azure SQL Data Warehouse in minutes
Find out more about the Azure SQL Data Warehouse’s unmatched price-performance and related announcements from Microsoft.

Quelle: Azure

Azure Databricks – VNet injection, DevOps Version Control and Delta availability

Azure Databricks provides a fast, easy, and collaborative Apache® Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions that drive the business forward, all backed by industry-leading SLAs.

With Azure Databricks, you can set up your Spark environment in minutes and autoscale quickly and easily. You can also apply your existing skills and collaborate on shared projects in an interactive workspace with support for Python, Scala, R, and SQL, as well as data science frameworks and libraries like TensorFlow and PyTorch.

We’re continuously listening to customers and answering questions as we evolve this service. This blog outlines important service announcements that we are proud to deliver for our customers.

Azure Databricks Delta available in Standard and Premium SKUs

Azure Databricks Delta brings new levels of reliability and performance for production workloads based on a number of improvements including transaction support, schema validation, indexing, and data versioning.

Since the preview of Delta was announced, we have received overwhelmingly positive feedback on how it has helped customers build complex pipelines for both batch and streaming data, and simplified ETL pipelines. We are excited to announce that Delta is now available in our Standard SKU offering in addition to Premium SKU offering so you can leverage its capabilities to the fullest and build pipelines more efficiently. Now everyone can get the benefits of Databricks Delta‘s reliability and performance.

You can read more about Azure Databricks Delta in our guide, “Introduction to Databricks Delta,” and import our quickstart notebook.

Azure DevOps Services Version Control

Azure DevOps is a collection of services that provide an end-to-end solution for the five core practices of DevOps: planning and tracking, development, build and test, delivery, and monitoring and operations.

Initially, we started with GitHub integration for Azure Databricks notebooks. On popular demand, we have introduced the ability to set your Git provider to Azure DevOps Services.

Authentication with Azure DevOps Services is done automatically when you authenticate using Azure Active Directory (Azure AD). The Azure DevOps Services organization must be linked to the same Azure AD tenant as Databricks. You can easily select your Git provider to Azure DevOps Services as shown in the documentation, “Azure DevOps Services Version Control.”

Deploy Azure Databricks in your own Azure virtual network (VNet injection) preview

By default, we deploy and manage your clusters for you in managed VNETs, with peering enabled. We create and manage these VNETs, but they reside in your subscription. We also manage the accompanying network security group rules.

Some customers, however, require network customization. I am pleased to announce that if you need to, now you can deploy Azure Databricks in your own existing virtual network (VNet injection). Connect Azure Databricks to other Azure services, such as Azure Storage, in a secure manner using service endpoints or to on-premises data sources for use with Azure Databricks, taking advantage of user-defined routes. You can also connect Azure Databricks to a network virtual appliance to inspect all outbound traffic and take actions according to allow and deny rules. Configure Azure Databricks to use custom DNS and configure network security group (NSG) rules to specify egress traffic restrictions.

Deploying Azure Databricks to your own virtual network also lets you take advantage of flexible CIDR ranges. See the documentation to quickly and easily configure Azure Databricks in your Vnet using the Azure Portal UI.

Get started today!

Try Azure Databricks and let us know your feedback!

Try Azure Databricks through a 14-day premium trial with free Databricks Units.
Sign up for the webinar on Machine Learning on Azure.
Watch the video on how to get started with the Apache Spark on Azure Databricks.
Visit the repository of Azure Databricks resources to continue learning.

Quelle: Azure

Azure Marketplace new offers – Volume 33

We continue to expand the Azure Marketplace ecosystem. From February 1 to February 15, 2019, 50 new offers successfully met the onboarding criteria and went live. See details of the new offers below:

Virtual machines

Attunity Replicate: Attunity Replicate integrates data in real time to Azure targets, including Azure SQL Data Warehouse, Azure SQL Database, and Azure Event Hubs, and it helps load, ingest, migrate, distribute, consolidate, and synchronize data.

Cyber Security Assessment Tool (CSAT): The Cyber Security Assessment Tool (CSAT) from QS solutions provides insight into security vulnerabilities through automated scans and analyses.

Fortinet FortiSandbox Advanced Threat Protection: FortiSandbox for Azure enables organizations to defend against advanced threats natively in the cloud, alongside third-party security solutions, or as an extension to their on-premises security architectures.

InterSystems IRIS for Health Community Edition: InterSystems IRIS for Health provides the capabilities for building complex, data-intensive applications. It’s a comprehensive platform spanning data management, interoperability, transaction processing, and analytics.

KNIME Server: KNIME Server offers shared repositories, advanced access management, flexible execution, web enablement, and commercial support. Share data, nodes, metanodes, and workflows across your team and throughout your company.

ME PasswordManagerPro 20 admins,25 keys: ManageEngine Password Manager Pro is a web-based, privileged identity management solution that lets you manage privileged identity passwords, SSH keys, and SSL certificates.

MODX on Windows Server 2016: MODX is an agile and user-friendly content management system that offers unlimited scalability and high flexibility.

MODX on Windows Server 2019: MODX is an agile and user-friendly content management system that offers unlimited scalability and high flexibility.

Panzura Freedom NAS 7.1.8.0: Panzura Freedom Filer is a hybrid cloud data management solution that enables global enterprise customers to consolidate their data islands into a single source of truth in Azure.

Puppet Enterprise 2018.1.7: Puppet Enterprise lets you automate the entire lifecycle of your Azure infrastructure, simply and securely, from initial provisioning through application deployment.

Vemn Digital Folder: Digital Folder is a web application that facilitates the management of your organization’s digital documents, creating sustainable digital transformation.

WALLIX Bastion: With an unobtrusive architecture, full multitenancy, and virtual appliance packaging, WALLIX Bastion (WAB Suite) provides an effective route to security and compliance.

Web applications

Discovery Hub and Azure SQL DB: Discovery Hub supports core analytics, the modern data warehouse, IoT, and AI. Developed with a cloud-first mindset, Discovery Hub provides a cohesive data fabric across Microsoft on-premises technology and Azure Data Services.

Discovery Hub and Azure SQL DB and AAS: Discovery Hub Application Server for Azure is a high-performance data management platform that accelerates your time to data insights.

Forcepoint Email Security V8.5.3: Forcepoint Email Security is an enterprise email and data loss prevention solution offering inbound and outbound protection against malware, blended threats, and spam.

MultiChain on Azure: Save time configuring servers and installing MultiChain, a leading enterprise blockchain platform, using these templates.

NetGovern: NetGovern enables legal supervisors, attorneys, paralegals, and case administrators to perform e-discovery on file systems, email archives, SharePoint, and file-sharing solutions such as Box.com and Citrix ShareFile.

NetGovern Multitenant: Enable anyone to rapidly respond to e-discovery requests. This application will deploy a shared infrastructure layer with one tenant preloaded for cloud service providers to deliver service to their customers.

Radware Alteon VA Application Cluster: This Alteon virtual appliance on Azure provides a simple and agile way to consume and deploy all standard ADC functionality as well as advanced services like WAF, acceleration, and application performance monitoring.

S2IX – Secure Search and Information Exchange: Secure by design, S2IX provides a protected business process automation solution. This gives users the ability to collaborate and manage documents in an environment they can depend on, even in remote or challenging locations.

Starburst Presto (v 0.213-e) for Azure HDInsight: Architected for the separation of storage and compute, Presto is perfect for querying data in Azure Blob storage, Azure Data Lake storage, relational databases, Cassandra, MongoDB, Kafka, and many others.

Veritas Resiliency Platform Repository Server: Veritas Resiliency Platform (VRP) provides single-click disaster recovery and migration for any source workload into Azure. This version of Veritas Resiliency Platform Repository Server will upgrade your installation.

Container solutions

Drupal with NGINX Container Image: Drupal with NGINX enhances the popular open-source content management system with the performance and security of NGINX. Drupal's modular architecture lets you create many different types of websites and applications.

Consulting services

4-Week Azure Assessment: This cloud migration assessment by Quisitive is designed to help organizations assess cloud readiness, evaluate best paths for their application environment, and build a clear road map and ROI view for potential Azure migration.

AgileAscend-M365 Migration: 3 week Implementation: Working alongside your project manager and IT staff, Agile IT's award-winning professional team will assist in planning, preparing, and migrating on-premises infrastructure to Microsoft Azure.

AgileProtect: Azure Data Backup – 3-Week Imp.: AgileProtect Standard is designed for business-critical systems. AgileProtect Standard backs up everything on the system, enabling a complete replica to be spun up on suitable hardware at will.

AgileSecurity: Intune – 3 week implementation: With Agile IT, define a mobile device management strategy that fits the needs of your organization. Set granular app policies to containerize data access while preserving the familiar Office 365 user experience.

Azure 5-Day Proof of Concept (POC): Chorus IT's proof of concept allows you to evaluate Azure with a small-scale partial implementation or focus on a particular area you want to evaluate.

Azure Accelerate: 2-week Proof of Concept: iLink Systems Inc.'s proof of concept is designed to streamline an organization’s journey to the cloud through a combination of training, workshops, comparative analysis, and rapid prototyping.

Azure Analytics services – 2 Hour Briefing: This briefing by Incremental Group will take you through the capabilities available from Azure Analytics and discuss how these could help your organization.

Azure Cost Optimization: 3 Week Assessment: This assessment by DXC focuses on analyzing current Azure consumption to identify opportunities to right-size the environment, inclusive of storage, networking, and virtual machines.

Azure Managed CI/CD Pipeline: 8-Wk Implementation: 2nd Watch will identify your environment requirements and implement CI/CD pipeline tools to your specifications, accelerating your adoption of agile methodologies.

Azure Migration Quickstart: 4 Week POC: The Azure Migration Quickstart by DXC works to test an initial workload of O/S, application, and/or database to migrate into Azure as a proof of concept.

Azure Performance Optimization: 3 Week Assessment: DXC's Azure Performance assessment provides a data-driven review of your existing Azure environment to help identify and resolve performance challenges.

Azure Security Managed Services: 2 Wk Assessment: In this assessment, DXC consultants will implement one or more tools to assist in the security review of your Azure configuration and architecture.

Cloud Cost Optimisation – 10 day implementation: risual's implementation allows organizations to track and monitor their costs on Azure to ensure they are getting the most value possible.

Cloud Readiness Assessment: 1-Day: Incremental Group’s Cloud Readiness Assessment reviews an organization's IT infrastructure and supports the organization's future business plans while assessing the impact they could have on the IT infrastructure.

DevOps Quality Services: 8 Wk Implementation: Sogeti USA offers assistance in designing and implementing DevOps quality programs, including establishing an automation testing framework, building a quality pipeline, and providing recommendations on enterprise metrics.

Identity and Access Management – 2 Hour Briefing: One of Incremental Group’s expert consultants will talk you through the wide range of services available from Microsoft Azure to help you manage your identity and access management.

Microsoft Azure AI Chatbot: 1-Hr Assessment: In this assessment, Cynoteck Technology Solutions will discuss AI chatbot development. Learn how chatbots and Azure Bot Service can benefit your business.

Microsoft Azure Health Check: 1 Week Assessment: This health check by DXC will involve a review of cloud architecture, cost optimization, Azure security best practices, and configuration best practices.

Migrate Dynamics GP to Azure: 1 Hr Assessment: Syvantis Technologies will walk you through the process of migrating your Microsoft Dynamics GP system to Microsoft Azure.

Modernize Your Apps – 2 Week Assessment: Modernize your legacy systems and build new applications in Azure with SPR Consulting's two-week assessment to help you take advantage of the flexibility of Azure.

PCI Azure Implementation Services: 4 Week POC: DXC will build a detailed proof of concept to offload the bulk of deploying and managing PCI-compliant workloads in Microsoft Azure.

SAP to Azure Migration: 1-Day Workshop: This SAP on Azure workshop by Infopulse will cover best practices for architecting, developing, and managing SAP services and apps in Azure. Customers should have good architectural knowledge in SAP Basis and Azure services.

SAP to Azure Migration: 1-Hour Briefing: Infopulse's briefing will help you understand the key benefits and challenges of a SAP-to-Azure migration and choose the best cloud solution for your business.

SAP to Azure Migration: 2-Week PoC: Infopulse will help you develop a proof of concept to validate the feasibility of your ideas and identify the benefits of migrating your on-premises SAP solution to Microsoft Azure.

SAP to Azure Migration Readiness: 1-Day Assessment: Infopulse will help you identify business drivers and the potential challenges of a SAP migration to Azure, then gather all requirements and create a suitable migration strategy.

Small Systems Mainframe Migration: 3-Wk Assessment: This assessment by Asysco Inc. will investigate smaller mainframe systems in order to develop a plan to migrate to Azure.

TCO & Cloud Readiness Assessment – 6 Wk Assessment: Ensono's assessment will involve installing a console server (built by the customer), gathering data, creating an HCP tenant, ingesting the initial server list, and conducting analysis.

Quelle: Azure

Create a transit VNet using VNet peering

Azure Virtual Network (VNet) is the fundamental building block for any customer network. VNet lets you create your own private space in Azure, or as I call it your own network bubble. VNets are crucial to your cloud network as they offer isolation, segmentation, and other key benefits. Read more about VNet’s key benefits in our documentation, “What is Azure Virtual Network?”

With VNets, you can connect your network in multiple ways. You can connect to on-premises using Point-to-Site (P2S), Site-to-Site (S2S) gateways or ExpressRoute gateways. You can also connect to other VNets directly using VNet peering.

Customer network can be expanded by peering Virtual Networks to one another. Traffic sent over VNet peering is completely private and stays on the Microsoft Backbone. No extra hops or public Internet involved. Customers typically leverage VNet peering in the hub-and-spoke topology. The hub consists of shared services and gateways, and the spokes comprise business units or applications.

Gateway transit

Today I’d like to do a refresh of a unique and powerful functionality we’ve supported from day one with VNet peering. Gateway transit enables you to use a peered VNet’s gateway for connecting to on-premises instead of creating a new gateway for connectivity. As you increase your workloads in Azure, you need to scale your networks across regions and VNets to keep up with the growth. Gateway transit allows you to share an ExpressRoute or VPN gateway with all peered VNets and lets you manage the connectivity in one place. Sharing enables cost-savings and reduction in management overhead.

With Gateway transit enabled on VNet peering, you can create a transit VNet that contains your VPN gateway, Network Virtual Appliance, and other shared services. As your organization grows with new applications or business units and as you spin up new VNets, you can connect to your transit VNet with VNet peering. This prevents adding complexity to your network and reduces management overhead of managing multiple gateways and other appliances.

VNet peering works across regions, across subscriptions, across deployment models (classic to ARM), and across subscriptions belonging to different Azure Active Directory tenants.

You can create a Transit VNet like one shown below.

Easy to set up – just check a box!

To use this powerful capability, simply check a box.

Create or update the virtual network peering from Hub-RM to Spoke-RM inside the Azure portal. Navigate to the Hub-RM VNet or the VNet with the gateway you’d like to use for gateway transit, and select Peerings, then Add:

Set the Allow gateway transit option
Select OK

Create or update the virtual network peering from Spoke-RM to Hub-RM from the Azure portal. Navigate to the Spoke-RM VNet, select on Peerings, then Add:

Select the Hub-RM virtual network in the corresponding subscription
Set the Use remote gateways option
Select OK

You can learn more in this detailed step by step guide on how to configure VPN gateway transit for virtual network peering.

Security

You can use Network Security Groups and security rules to control the traffic between on-premises and your Azure VNets. Security policies can be centralized in the Hub or transit VNet. A network virtual appliance (NVA) can inspect all traffic going on-premises as well as into Azure. Since there policy is set in a central VNet, you can set it up just once.

Routing

We plumb the routes, so you don’t have to. Each Azure Virtual Machine (VM) you deploy will benefit with the routes being plumbed automatically. To confirm a virtual network peering, you can check effective routes for a network interface in any subnet in a virtual network. If a virtual network peering exists, all subnets within the virtual network have routes with next hop type VNet peering, for each address space in each peered virtual network.

Monitoring

You can check the health status of your VNet Peering connection in the Azure portal. Connected means you are all peered and good to go. Initiated means a second link needs to be created. Disconnected means the peering has been deleted from one side.

You can also troubleshoot connectivity to a virtual machine in a peered virtual network using Network Watcher's connectivity check. Connectivity check lets you see how traffic is routed from a source virtual machine's network interface to a destination virtual machine's network interface as seen below.

Limits

You can peer with 100 other VNets. We’ve increased limits by four times and as our customers scale in Azure we will continue to increase these limits. Stay updated with limits by visiting our documentation, “Azure subscription and service limits, quotas, and constraints.”

Pricing

You pay only on traffic that goes through the gateway. No double charge. Traffic passing through a remote gateway in this scenario is subject to VPN gateway charges and does not incur VNet peering charges. For example, If VNetA has a VPN gateway for on-premise connectivity and VNetB is peered to VNetA with appropriate properties configured, traffic from VNetB to on-premises is only charged egress per VPN gateway pricing. VNet peering charges do not apply.

Availability

VNet peering with gateway transit works across classic Azure Service Management (ASM) and Azure Resource Manager (ARM) deployment models. Your gateway should be in your ARM VNet. It also works across subscriptions and subscriptions belonging to different Azure Active Directory tenants.

Gateway transit has been available since September 2016 for VNet peering in all regions and will be available for global VNet peering shortly.

Gateway Transit with VNet peering enables you to create a transit VNet to keep your shared services in a central location. To keep up with your growing scale, you can scale your VNets and use your existing VPN gateway saving management overhead, cost, and time. We developed a template you can use to get started. Try it out today!
Quelle: Azure

Stay informed about service issues with Azure Service Health

When your Azure resources go down, one of your first questions is probably, “Is it me or is it Azure?” Azure Service Health helps you stay informed and take action when Azure service issues like incidents and planned maintenance affect you by providing a personalized health dashboard, customizable alerts, and expert guidance.

In this blog, we’ll cover how you can use Azure Service Health’s personalized dashboard to stay informed about issues that could affect you now or in the future.

Monitor Azure service issues and take action to mitigate downtime

You may already be familiar with the Azure status page, a global view of the health of all Azure services across all Azure regions. It’s a good reference for major incidents with widespread impact, but we recommend using Azure Service Health to stay informed about Azure incidents and maintenance. Azure Service Health only shows issues that affect you, provides information about all incidents and maintenance, and has richer capabilities like alerting, shareable updates and RCAs, and other guidance and support.

Azure Service Health tracks three types of health events that may impact you:

Service issues: Problems in Azure services that affect you right now.
Planned maintenance: Upcoming maintenance that can affect the availability of your services in the future. Typically communicated at least seven days prior to the event.
Health advisories: Health-related issues that may require you to act to avoid service disruption. Examples include service retirements, misconfigurations, exceeding a usage quota, and more. Usually communicated at least 90 days prior, with notable exceptions including service retirements, which are announced at least 12 months in advance, and misconfigurations, which are immediately surfaced.

Learn more about your personalized health dashboard.

Get started with Azure Service Health

Azure Service Health’s dashboard provides a large amount of information about incidents, planned maintenance, and other health advisories that could affect you. While you can always visit the dashboard in the portal, the best way to stay informed and take action is to set up Azure Service Health alerts. With alerts, as soon as we publish any health-related information, you’ll get notified on whichever channels you prefer, including email, SMS, push notification, webhook into ServiceNow, and more.

Below are a few resources to help you get started:

Review your Azure Service Health dashboard and set up alerts.
If you need help getting started, check our Azure Service Health documentation.
We always welcome feedback. Submit your ideas or email us with any questions or comments at servicehealth@microsoft.com.

Quelle: Azure