The Value of IoT-Enabled Intelligent Manufacturing

As the manufacturing industry tackles some significant challenges including an aging workforce, compliance issues, and declining revenue, the Internet of Things (IoT) is helping reinvent factories and key processes. At the heart of this transformation journey is the design and use of IoT-enabled machines that help lead to reduced downtime, increased productivity, and optimized equipment performance.

Learn how you can apply insights from real-world use cases of IoT-enabled intelligent manufacturing when you attend the Manufacturing IoT webinar on March 28th. For additional hands-on, actionable insights around intelligent edge and intelligent cloud IoT solutions, join us on April 19th for the Houston Solution Builder Conference.

Using IoT solutions to move from a reactive to predictive model

In the past, factory managers often had no way of knowing when a machine might begin to perform poorly or completely shut down. When something went wrong, getting the equipment back up and running was often time consuming and based on trial-and-error troubleshooting. And for the company, any unplanned downtime meant slowed or halted production, resulting in lower productivity and higher costs.

The development of IoT-enabled machines with sensors allows companies to improve overall efficiency, performance, and profitability. Rockwell Automation found it time consuming and challenging to monitor its equipment in remote locations. Using Microsoft Azure to connect them, Rockwell Automation now sees real-time performance information and can proactively maintain equipment before an incident occurs.

Kontron S&T, a Microsoft partner, also recently developed the SUSiEtec platform, an end-to-end IoT solution that enables companies to build scalable edge computing solutions using Microsoft Azure IoT Edge integration and customization services. With SUSiEtec, companies can dynamically decide where data analysis will take place and manage distributed IoT devices regardless of where they’re located or how many devices are used. Join the Manufacturing IoT webinar to learn more about SUSiEtec and how to develop secure, manageable IoT solutions for manufacturing.

Keeping IoT data secure with Azure Sphere

Using IoT to create the factory of the future also means additional access points into the factory network and systems, so creating a secure network is top priority. Factory managers typically access IoT data using mobile devices, which creates even more access points. For a true connected IoT experience and factory, security is foundational.

Azure Sphere provides a foundation of security and connectivity that starts in the silicon and extends to the cloud. Together, Azure Sphere microcontrollers (MCUs), secured OS, and turnkey cloud security service guard every Azure Sphere device accessing IoT data, IoT sensors, and IoT-enabled machines. By adding useful software to Edge hardware, factories are protected with IT-proven standards as well as new Operational Technology (OT) network security.

Getting ready to develop IoT solutions

Moving to a factory of the future starts with determining what you want to achieve through the IoT-enabled machine. If predictive maintenance is the end goal, start by conducting an inventory of data sources. Identify all potential sources and types of relevant data to determine what is most essential. Then you’ll need to lay the groundwork for a robust predictive model by pulling in data that includes both expected behavior and failure logs.

With the initial logistics determined, the next step is to create a model and test and iterate to figure out which model is best at forecasting the timing of unit failures. By moving to a live operational setting, you can apply the model to live, streaming data to observe how it works in real-world conditions. After adjusting your maintenance processes, systems, and resources to act on the new insights, the final step is to integrate the model with Azure IoT Central into operations.

Of course, not all companies have the skillset or resources to develop an IoT solution from scratch. To accelerate the design, development, and implementation process, partners can utilize the Microsoft Accelerator program. By using open-source code or leveraging proven architectures, companies can create a fully customizable solution and quickly connect devices to existing systems in minutes. For instance, the Predictive Maintenance solution accelerator combines key Azure IoT services like IoT Hub and Stream analytics to proactively optimize maintenance and create automatic alerts and actions for remote diagnostics, maintenance requests, and other workflows.

Digitally transforming your own business and building or deploying IoT solutions that are highly scalable and economical to manage takes partnerships. Join Microsoft and Kontron S&T on March 28th for the webinar, Go from Reaction to Prediction – IoT in Manufacturing, and discover new approaches for achieving your business goals.
Quelle: Azure

Microsoft and NVIDIA bring GPU-accelerated machine learning to more developers

With ever-increasing data volume and latency requirements, GPUs have become an indispensable tool for doing machine learning (ML) at scale. This week, we are excited to announce two integrations that Microsoft and NVIDIA have built together to unlock industry-leading GPU acceleration for more developers and data scientists.

Azure Machine Learning service is the first major cloud ML service to integrate RAPIDS, an open source software library from NVIDIA that allows traditional machine learning practitioners to easily accelerate their pipelines with NVIDIA GPUs
ONNX Runtime has integrated the NVIDIA TensorRT acceleration library, enabling deep learning practitioners to achieve lightning-fast inferencing regardless of their choice of framework.

These integrations build on an already-rich infusion of NVIDIA GPU technology on Azure to speed up the entire ML pipeline.

“NVIDIA and Microsoft are committed to accelerating the end-to-end data science pipeline for developers and data scientists regardless of their choice of framework,” says Kari Briski, Senior Director of Product Management for Accelerated Computing Software at NVIDIA. “By integrating NVIDIA TensorRT with ONNX Runtime and RAPIDS with Azure Machine Learning service, we’ve made it easier for machine learning practitioners to leverage NVIDIA GPUs across their data science workflows.”

Azure Machine Learning service integration with NVIDIA RAPIDS

Azure Machine Learning service is the first major cloud ML service to integrate RAPIDS, providing up to 20x speedup for traditional machine learning pipelines. RAPIDS is a suite of libraries built on NVIDIA CUDA for doing GPU-accelerated machine learning, enabling faster data preparation and model training. RAPIDS dramatically accelerates common data science tasks by leveraging the power of NVIDIA GPUs.

Exposed on Azure Machine Learning service as a simple Jupyter Notebook, RAPIDS uses NVIDIA CUDA for high-performance GPU execution, exposing GPU parallelism and high memory bandwidth through a user-friendly Python interface. It includes a dataframe library called cuDF which will be familiar to Pandas users, as well as an ML library called cuML that provides GPU versions of all machine learning algorithms available in Scikit-learn. And with DASK, RAPIDS can take advantage of multi-node, multi-GPU configurations on Azure.

Learn more about RAPIDS on Azure Machine Learning service or attend the RAPIDS on Azure session at NVIDIA GTC.

ONNX Runtime integration with NVIDIA TensorRT in preview

We are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework. Developers can now tap into the power of TensorRT through ONNX Runtime to accelerate inferencing of ONNX models, which can be exported or converted from PyTorch, TensorFlow, MXNet and many other popular frameworks. Today, ONNX Runtime powers core scenarios that serve billions of users in Bing, Office, and more.

With the TensorRT execution provider, ONNX Runtime delivers better inferencing performance on the same hardware compared to generic GPU acceleration. We have seen up to 2X improved performance using the TensorRT execution provider on internal workloads from Bing MultiMedia services.

To learn more, check out our in-depth blog on the ONNX Runtime and TensorRT integration or attend the ONNX session at NVIDIA GTC.

Accelerating machine learning for all

Our collaboration with NVIDIA marks another milestone in our venture to help developers and data scientists deliver innovation faster. We are committed to accelerating the productivity of all machine learning practitioners regardless of their choice of framework, tool, and application. We hope these new integrations make it easier to drive AI innovation and strongly encourage the community to try it out. Looking forward to your feedback!
Quelle: Azure

Microsoft Azure for the Gaming Industry

This blog post was co-authored by Patrick Mendenall, Principal Program Manager, Azure. 

We are excited to join the Game Developers Conference (GDC) this week to learn what’s new and share our work in Azure focused on enabling modern, global games via cloud and cloud-native technologies.

Cloud computing is increasingly important for today’s global gaming ecosystem, empowering developers of any size to reach gamers in any part of the world. Azure’s 54 datacenter regions, and its robust global network, provides globally available, high performance services, as well as a platform that is secure, reliable, and scalable to meet current and emerging infrastructure needs. For example, earlier this month we announced the availability of Azure South Africa regions. Azure services enable every phase of the game development lifecycle from designing, building, testing, publishing, monetizing, measurement, engagement, and growth, providing:

Compute: Gaming services rely on a robust, reliable, and scalable compute platform. Azure customers can choose from a range of compute- and memory-optimized Linux and Windows VMs to run their workloads, services, and servers, including auto-scaling, microservices, and functions for modern, cloud-native games.
Data: The cloud is changing the way applications are designed, including how data is processed and stored. Azure provides high availability, global data, and analytics solutions based on both relational databases as well as big data solutions.
Networking: Azure operates one of the largest dedicated long-haul network infrastructures worldwide, with over 70,000 miles of fiber and sub-sea cable, and over 130+ edge sites. Azure offers customizable networking options to allow for fast, scalable, and secure network connectivity between customer premises and global Azure regions.
Scalability: Azure offers nearly unlimited scalability. Given the cyclical usage patterns of many games, using Azure enables organizations to rapidly increase and/or decrease the number of cores needed, while only having to pay for the resources that are used.
Security: Azure offers a wide array of security tools and capabilities, to enable customers to secure their platform, maintain privacy and controls, meet compliance requirements (including GDPR), and ensure transparency.
Global presence: Azure has more regions globally than any other cloud provider, offering the scale needed to bring games and data closer to users around the world, preserving data residency, and providing comprehensive compliance and resiliency options for customers. Using Azure’s footprint, the cost, the time, and the complexity of operating a game at global scale can be reduced.
Open: with Azure you can use the software you choose whether it be operating systems, engines, database solutions, or open source – run it on Azure.

We’re also excited to bring PlayFab into the Azure family. Together, Azure and PlayFab are a powerful combination for game developers. Azure brings reliability, global scale, and enterprise-level security, while PlayFab provides Game Stack with managed game services, real-time analytics, and comprehensive LiveOps capabilities.

We look forward to meeting many of you at GDC 2019 to learn about your ideas in gaming, discussing where cloud and cloud-native technologies can enable your vision, and sharing more details on Azure for gaming. Join us at the conference or contact our gaming industry team at azuregaming@microsoft.com.

Details on all of these are available via links below.

Learn more about Microsoft Game Stack.
Talks at GDC:

Thursday, March 21, 2019 at 11:30 AM: Best Practices for Building Resilient, Scalable, Game Services in Microsoft Azure
Thursday, March 21, 2019 at 12:45 PM: Save Time for Creativity: Unlocking the Potential for Your Game's Data with Microsoft Azure

Azure Gaming Reference Architectures: Landing Page

Multiplayer/Game Servers
Analytics
Leaderboards
Cognitive Services

GDC Booth demos for Azure:

AI Training with Containers – Use Azure and Kubernetes to power Unity ML Agents
Game Telemetry – Build better game balance and design
Build NoSQL Data Platforms – Azure Cosmos DB: a globally distributed, massively scalable NoSQL database service
Cross Realms with SQL – Build powerful databases with Azure SQL

Quelle: Azure

March 2019 changes to Azure Monitor Availability Testing

Azure Monitor Availability Testing allows you to monitor the availability and responsiveness of any HTTP or HTTPS endpoint that is accessible from the public internet. You don't have to add anything to the web site you're testing. It doesn't even have to be your site, you could test a REST API service you depend on. This service sends web requests to your application at regular intervals from points around the world. It alerts you if your application doesn't respond, or if it responds slowly.

At the end of this month we are deploying some major changes to this service, these changes will improve performance and reliability, as well as allow us to make more improvements to the service in the future. This post will highlight some of the changes, as well as describe some of the changes you should be aware of to ensure that your tests continue running without any interruption.

Reliability improvements

We are deploying a new version of the availability testing service. This new version should improve the reliability of the service, resulting in fewer false alarms. This change also increases the capacity for the creation of new availability tests, which is greatly needed as Application Insights usage continues to grow. Additionally, the architecture of this new design enables us to add new regions much more easily. Expect to see additional regions from which you can test your app’s availability in the future!

New UI

Along with the new backend architecture, we are updating the availability testing UI with a brand new design. See the image below for a sneak peek of the UI that we will be rolling out for all customers in the next few weeks. 

The new design is more consistent with other experiences in Application Insights. It reduces the number of clicks needed to see highly requested information, and surfaces insights about your availability tests to the right side of the availability scatter plot. The new chart supports time brushing, you can click and drag over a section of the chart to zoom into just that time period. Additionally, this design loads faster than the previous one!

IP address changes

If you have whitelisted certain IP addresses because you are running web tests on your app, but your web server is restricted to serving specific clients, then you should be aware that we are deploying our service on new IP ranges. We are increasing the capacity of our service, and this requires adding additional test agents.

Effective March 20, 2019, we will begin running tests from our new test agents, and this will require you to update your whitelist. The list containing all of the necessary whitelisted IPs, including our previous IP ranges and the new IP ranges is published in our documentation, “IP addresses used by Application Insights and Log Analytics.”

France South changes

France South will no longer be offered as a region from which you can perform availability tests. All existing tests in France South will be moved to a duplicate service running in France Central which will appear in the portal as “France Central (formerly France South).”  If you already have a test running in France Central, this means that your test will run from France Central twice per time period. Your existing alert rules will not be affected.

New testing region

We will be adding an additional region within Europe from which to run availability tests. An announcement will be made when this region is available.

Next steps

Log into your Azure account today to get started with the new Application Insights Availability UX. You can also learn more about how to get started by visiting our “Azure Monitor Documentation.”
Quelle: Azure

Securely monitoring your Azure Database for PostgreSQL Query Store

A few months ago, I shared best practices for alerting on metrics with Azure Database for PostgreSQL. Though I was able to cover how to monitor certain key metrics on Azure Database for PostgreSQL, I did not cover how to monitor and alert on the performance of queries that your application is heavily relying on. As a PostgreSQL database, from time to time you will need to investigate if there are any queries running indefinitely on a PostgreSQL database. These long running queries may interfere with the overall database performance and likely get stuck on some background process. This blog post covers how you can set up alerting on query performance related metrics using Azure Functions and Azure Key Vault.

What is Query Store?

Query Store was a feature in Azure Database for PostgreSQL announced in early Fall 2018 that seamlessly enables tracking query performance over time. This simplifies performance troubleshooting by helping you quickly find the longest running and most resource-intensive queries. Learn how you can use Query Store on a wide variety of scenarios by visiting our documentation, “Usage scenarios for Query Store.” Query Store, when enabled, automatically captures a history of query runtime and wait statistics. It tracks this data over time so that you can see database usage patterns. Data for all users, databases, and queries is stored in a database named azure_sys in the Azure Database for PostgreSQL instance.

Query Store is not enabled on a server by default. However, it is very straightforward to opt-in on your server by following the simple steps detailed in our documentation, “Monitor performance with the Query Store.” After you have enabled Query Store to monitor your application performance, you can set alerts on various metrics such as long running queries, regressed queries, and more that you want to monitor.

How to set up alerting on Query Store metrics

You can achieve near real-time alerting on Query Store metrics monitoring using Azure Functions and Azure Key Vault. This GitHub repo provides you with an Azure Function and a PowerShell script to deploy a simple monitoring solution, which gives you some flexibility to change what and when to alert.

Alternatively, you can clone the repo to use this as a starting point and make code changes to better fit your scenario. The Visual Studio solution, when built with your changes, will automatically package the zip file you need to complete your deployment in the same fashion that is described here.

In this repo, the script DeployFunction creates an Azure function to serve as a monitor for Azure Database for PostgreSQL Query Store. Understanding the data collected by query performance insights will help you identify the metrics that you can alert on.

If you don't make any changes to the script or the function code itself and only provide the required parameters to DeployFunction script, here is what you will get:

A function app.
A function called PingMyDatabase that is time triggered every one minute.
An alert condition that looks for any query that has a mean execution time of longer than five seconds since the last time query store data is flushed to the disk.
An email when an alert condition is met with an attached list of all of the processes that was running on the instance, as well as the list of long running queries.
A key vault that contains two secrets named pgConnectionString and senderSecret that hold the connection string to your database and password to your sender email account respectively.
An identity for your function app with access to a Get policy on your secrets for this key vault.

You simply need to run DeployFunction on Windows PowerShell command prompt. It is important to run this script from Windows PowerShell. Using Windows PowerShell ISE will likely result in errors as some of the macros may not resolve as expected.

The script then creates the resource group and Key Vault deploys a monitoring function app, updates app configuration settings, and sets up the required Key Vault secrets. At any point during the deployment, you can view the logs available in the .logs folder.

After the deployment is complete, you can validate the secrets by going to the resource group in the Azure portal. As shown in the following diagram, two secrets keys are created, pgConnString and senderSecret. You can select the individual secrets if you want to update the value.

Depending on the condition set in the SENDMAILIF_QUERYRETURNSRESULTS app settings, you will receive an email alert when the condition is met.

How can I customize alert condition or supporting data in email?

After the default deployment goes through, using Azure portal you can update settings by selecting Platform features and then Application settings.

You can change the run interval, mail to, if condition, or supporting data to be attached by making changes to the below settings and saving them on your exit.

Alternatively, you can simply use az cli to update these settings like the following.

$cronIntervalSetting="CronTimerInterval=0 */1 * * * *"

az functionapp config appsettings set –resource-group yourResourceGroupName –name yourFunctionAppName –settings $cronIntervalSetting

Or

az functionapp config appsettings set –resource-group $resourceGroupName –name $functionAppName –settings "SENDMAILIF_QUERYRETURNSRESULTS=select * from query_store.qs_view where mean_time > 5000 and start_time >= now() – interval '15 minutes'"

Below are common cases on conditions that you can monitor and alert by either updating the function app settings after your deployment goes through or updating the corresponding value in DeployFunction.ps1 prior to your deployment:

Case

Function app setting name

Sample value

Query 3589441560 takes more than x milliseconds on average in the last fifteen minutes

SENDMAILIF_QUERYRETURNSRESULTS

select * from query_store.qs_view where query_id = 3589441560 and mean_time > x and start_time >= now() – interval '15 minutes'

Queries with cache hit less than 90 percent

SENDMAILIF_QUERYRETURNSRESULTS

select * , shared_blks_hit / nullif(shared_blks_hit + shared_blks_read, 0) AS as cache_hit from query_store.qs_view where shared_blks_hit / nullif(shared_blks_hit + shared_blks_read, 0) < 0.90

Queries with a mean execution time that is more than x milliseconds

SENDMAILIF_QUERYRETURNSRESULTS

select * from query_store.qs_view where mean_time > x and start_time >= now() – interval '15 minutes'

If an alert condition is met, check if there is an ongoing autovacuum operation, list the processes running and attach the results to email

LIST_OF_QUERIESWITHSUPPORTINGDATA

{“count_of_active_autovacuum”:” select count(*) from pg_stat_activity where position('autovacuum:' IN query) = 1 “,"list_of_processes_at_the_time_of_alert":"select now()-query_start as Running_Since,pid,client_hostname,client_addr, usename, state, left(query,60) as query_text from pg_stat_activity"}

How secure is this?

The script provides you with the mechanism to store your secrets in a Key Vault. Your secrets are secured as they are encrypted in-transit and at rest. However, the function app accesses the Key Vault over the network. If you want to avoid this and access your secrets over your virtual network (VNet) through the backbone, you will need to configure a VNet for both your function app and your Key Vault. Note, that VNet support of function apps is in preview and is currently available in selected Azure regions. When the proper deployment scenarios are supported, we may revisit this script to accommodate the changes. Until then, you will need to configure a VNet manually to accomplish the setup below.

We are always looking to hear feedback from you. If you have any feedback for the Query Store on PostgreSQL, or monitoring and alerting on query performance, please don’t hesitate to contact the Azure Database for PostgreSQL team.

Acknowledgments

Special thanks to Korhan Ileri, Senior Data Scientist, for developing the script and contributing to this post. As well as Tosin Adewale, Software Engineer from the Azure CLI team for closely partnering with us.
Quelle: Azure

Reducing security alert fatigue using machine learning in Azure Sentinel

Last week we launched Azure Sentinel, a cloud native SIEM tool. Machine learning (ML) in Azure Sentinel is built-in right from the beginning. We have thoughtfully designed the system with ML innovations aimed to make security analysts, security data scientists, and engineers productive. The focus is to reduce alert fatigue and offer ML toolkits tailored to the security community. The three ML pillars in Azure Sentinel include Fusion, built-in ML, build your own ML.

Fusion

Alert fatigue is real. Security analysts face a huge burden of triage as they not only have to sift through a sea of alerts, but also correlate alerts from different products manually or using a traditional correlation engine.

Our Fusion technology, currently in public preview, uses state of the art scalable learning algorithms to correlate millions of lower fidelity anomalous activities into tens of high fidelity cases. Azure Sentinel integrates with Microsoft 365 solution and correlates millions of signals from different products such as Azure Identity Protection, Microsoft Cloud App Security, and soon Azure Advanced Threat Protection, Windows Advanced Threat Protection, O365 Advanced Threat Protection, Intune, and Azure Information Protection. You can learn how to turn Fusion on by visiting our documentation, “Enable Fusion.”

Fusion combines yellow alerts, which themselves may not be actionable, into high fidelity security interesting red cases. We look at disparate products to produce actionable incidents so as to reduce the false positive rate. From our measurement with external customers and internal evaluation, we have a median 90 percent reduction in alert fatigue. This is possible because Fusion can detect complex, multi-stage attacks and differs from traditional correlation engines in the following ways:

Traditional correlation engines

Fusion

Assume that the attacker takes only one path to attain their goal.

Iterative attack simulation – Fusion encodes uncertainty with paths/stages by simulating different attack paths using an iterative arkov chain Monte Carlo simulations.

Assumes the attacker follows a static kill chain, as the attack path is executed.

Probabilistic cloud kill chain – Fusion constantly updates the probability of moving to the next step in kill chain through a custom defined prior probability function.

Assumes that all the information is present in the logs to catch the attacker.

Using advances in graphical methods – we encode uncertainty in completeness/connectivity of information in the kill chain helping us to detect novel attacks.

In the above screenshot, one can see that the Fusion case, and the two composite alerts that went into it.

Organizations are currently using Fusion for the following scenarios to compound anomalies from Identity Protection and Microsoft Cloud App Security products.

Anomalous login leading to O365 mailbox exfiltration
Anomalous login leading to suspicious cloud app administrative activity
Anomalous login leading to mass file deletion
Anomalous login leading to mass file download
Anomalous login leading to O365 impersonation
Anomalous login leading to mass file sharing
Anomalous login leading to ransomware in cloud app

Built-in ML

Machine learning is now an essential toolkit in security analytics to detect novel types of attacks that escape the traditional rules based system. However, a scarce ML talent pool makes it difficult for security organizations to staff applied security data scientists. To democratize the ML toolkit tailored to the needs of the security community, we introduce built-in ML which is currently in limited public preview.

Built-in ML is designed for security analysts and engineers, with no prior ML knowledge to reuse ML systems designed by Microsoft’s fleet of security machine learning engineers. The benefits of built-inML systems are that organizations dont have to worry about traditional investments like ML training cross validation, or deployment and quickly identify threats that wouldnt be found with a traditional approach.

Behind the cover, built-in ML uses principles of model compression and elements of transfer learning to make the model developed by Microsoft’s ML engineers ready to use for any organization’s needs. Our models are trained on diverse datasets, and periodically retrained to take concept drift into account.

We are opening our flagship geo login anomaly model for any security analyst to use to detect unusual logins in SSH logs. No ML expertise is necessary, customers bring in their logs to Azure Sentinel and use built-in ML systems to gain analysis instantly.

Build-your-own ML

We recognize that organizations have different levels of investments in machine learning for security use cases. Some organizations may have data scientists who need to go deeper and customize the analysis further. For these organizations, we offer the option of Build-you-own ML to author security analytics.

Azure Sentinel will offer Databricks, Spark, and Jupyter Notebook detection’s authoring environment, in order to take care of data plumbing, provide ML algorithm in templates, code snippets for model training and scheduling, and soon introduce seamless model management, model deployment, workflow scheduler, data versioning capabilities and specialized security analytics libraries. This will free up security data scientists from tedious pipeline and platform work, and focus on productive analytics on a hyper scale ML-security platform.

Additional resources

We will be updating this space with the technical details behind these innovations! If you have questions about turning on built-in ML or using build-your-own ML infrastructure, please reach out to askepd@microsoft.com. We also strongly recommend customers enable Fusion when they use Azure Sentinel. You can learn how to turn Fusion on by visiting our documentation, “Enable Fusion.”
Quelle: Azure

Microsoft and NVIDIA extend video analytics to the intelligent edge

Artificial Intelligence (AI) algorithms are becoming more intelligent and sophisticated every day, allowing IoT devices like cameras to bridge the physical and digital worlds. The algorithms can trigger alerts and take actions automatically — from finding available parking spots and missing items in a retail store to detecting anomalies on solar panels or workers approaching hazardous zones.

Processing these state-of-the-art AI algorithms in a datacenter requires a stable high-bandwidth connection to deliver videos feeds to the cloud. However, these cameras are often located in remote areas with unreliable connectivity or it may not be sensible given bandwidth, security, and regulatory needs.

Microsoft and NVIDIA are partnering on a new approach for intelligent video analytics at the edge to transform raw, high-bandwidth videos into lightweight telemetry. This delivers real-time performance and reduces compute costs for users. The “cameras-as-sensors” and edge workloads are managed locally by Azure IoT Edge and the camera stream processing is powered by NVIDIA DeepStream. Once the videos are converted, the data can be ingested to the cloud using Azure IoT Hub.

The companies plan to offer customers enterprise-ready devices running DeepStream in the Azure IoT device catalog, and the NVIDIA DeepStream module will soon be made available in the Azure IoT Edge marketplace.

Over the years, Microsoft and NVIDIA have helped customers run demanding applications on GPUs in the cloud. With this latest collaboration, NVIDIA DeepStream and Azure IoT Edge extend the AI-enhanced video analytics pipeline to where footage is captured, securely and at scale. Now, our customers can get the best of both worlds—accelerated video analytics at the edge with NVIDIA GPUs and secure connectivity and powerful device management with Azure IoT Edge and Azure IoT Hub.

To learn more, visit the Azure IoT Edge and NVIDIA DeepStream product pages. If you are attending GTC in person, join us Tuesday, March 19, 2019 from 9:00 – 10:00 AM at session S9545 – “Using the DeepStream SDK for AI-Based Video Analytics” or visit Microsoft at Booth 1122.
Quelle: Azure

Azure Machine Learning service now supports NVIDIA’s RAPIDS

Azure Machine Learning service is the first major cloud ML service to support NVIDIA’s RAPIDS, a suite of software libraries for accelerating traditional machine learning pipelines with NVIDIA GPUs.

Just as GPUs revolutionized deep learning through unprecedented training and inferencing performance, RAPIDS enables traditional machine learning practitioners to unlock game-changing performance with GPUs. With RAPIDS on Azure Machine Learning service, users can accelerate the entire machine learning pipeline, including data processing, training and inferencing, with GPUs from the NC_v3,  NC_v2, ND or ND_v2 families. Users can unlock performance gains of more than 20X (with 4 GPUs), slashing training times from hours to minutes and dramatically reducing time-to-insight.

The following figure compares training times on CPU and GPUs (Azure NC24s_v3) for a gradient boosted decision tree model using XGBoost. As shown below, performance gains increase with the number of GPUs. In the Jupyter notebook linked below, we’ll walk through how to reproduce these results step by step using RAPIDS on Azure Machine Learning service.

How to use RAPIDS on Azure Machine Learning service

Everything you need to use RAPIDS on Azure Machine Learning service can be found on GitHub.

The above repository consists of a master Jupyter Notebook that uses the Azure Machine Learning service SDK to automatically create a resource group, workspace, compute cluster, and preconfigured environment for using RAPIDS. The notebook also demonstrates a typical ETL and machine learning workflow to train a gradient boosted decision tree model. Users are also free to experiment with different data sizes and the number of GPUs to verify RAPIDS multi-GPU support.

About RAPIDS

RAPIDS uses NVIDIA CUDA for high-performance GPU execution, exposing GPU parallelism and high memory bandwidth through a user-friendly Python interface. It includes a dataframe library called cuDF which will be familiar to Pandas users, as well as an ML library called cuML that provides GPU versions of all machine learning algorithms available in Scikit-learn. And with DASK, RAPIDS can take advantage of multi-node, multi-GPU configurations on Azure.

Accelerating machine learning for all

With the support for RAPIDS on Azure Machine Learning service, we are continuing our commitment to an open and interoperable ecosystem where developers and data scientists can use the tools and frameworks of their choice. Azure Machine Learning service users will be able to use RAPIDS in the same way they currently use other machine learning frameworks, and they will be able to use RAPIDS in conjunction with Pandas, Scikit-learn, PyTorch, TensorFlow, etc. We strongly encourage the community to try it out and look forward to your feedback!
Quelle: Azure

Azure Container Registry virtual network and Firewall rules preview support

While Azure Container Registry (ACR) supports user and headless-service account authentication, customers have expressed their requirements for limiting public endpoint access. Customers can now limit registry access within an Azure Virtual Network (VNet), as well as whitelist IP addresses and ranges for on-premises services.

VNet and Firewall rules are supported with virtual machines (VM) and Azure Kubernetes Services (AKS).

Choosing between private and PaaS registries

As customers move into production, their security teams have a checklist they apply to production workloads, one of which is limiting all public endpoints. Without VNet support, customers had to choose between standalone products, or OSS projects they could run and manage themselves. This puts a larger burden on the customers to manage the storage, security, scalability, and reliability a production registry requires.

With VNet and Firewall rules, customers can achieve their security requirements, while benefiting from integrated security, secured at rest, geo-redundant, and geo-replicated PaaS Container Registry. Thus, freeing up their resources to focus on the unique business problems they face.

Azure Container Registry PaaS, enabling registry products

The newest VNet and Firewall rule capabilities of ACR are just the latest set of capabilities in container lifecycle management. ACR provides core primitives that other registry or CI/CD products may build upon. Our goal with ACR isn’t to compete with our partners, rather enable them with core cloud capabilities, allow them to focus on the higher level, unique capabilities each offer.

Getting started

Using the Azure CLI, or the Azure portal, customers can follow our documentation for configuring VNet and Firewall rules.

VNet and Firewall rules preview pricing

During preview, VNet and Firewall rules will be included in the Azure Container Registry’s Premium Tier.

Preview and general availability dates

As of March 18, 2019, VNet and Firewall rules are available for public preview in all 25 public cloud regions. General availability (GA) will be based on a curve of usage and feedback.

More information

Azure Container Registry
Geo-replicating registries
OS & Framework Patching
ACR Tasks

Quelle: Azure

Power IoT and time-series workloads with TimescaleDB for Azure Database for PostgreSQL

We’re excited to announce a partnership with Timescale that introduces support for TimescaleDB on Azure Database for PostgreSQL for customers building IoT and time-series workloads. TimescaleDB has a proven track record of being deployed in production in a variety of industries including oil & gas, financial services, and manufacturing. The partnership reinforces our commitment to supporting the open-source community to provide our users with the most innovative technologies PostgreSQL has to offer.

TimescaleDB allows you to scale for fast ingest and complex queries while natively supporting full SQL. It leverages PostgreSQL as an essential building block, which means that users get the familiarity and reliability of PostgreSQL, along with the scalability and performance of TimescaleDB. Enabling TimescaleDB on your new or existing Azure Database for PostgreSQL server will eliminate the need to run two databases to collect relational and time-series data.

How to get started

If you don’t already have an Azure Database for PostgreSQL server, you can create one with the Azure CLI command az postgres up. Next, run the following command to add TimescaleDB to your Postgres libraries:

az postgres server configuration set –resource-group mygroup –server-name myserver –name shared_preload_libraries –value timescaledb

Restart the server to load the new library. Then, connect to your Postgres database and run:

CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;

You can now create a TimescaleDB hypertable from scratch or migrate your existing time-series data.

Postgres with TimescaleDB as a foundation for IoT applications

PostgreSQL is enabling many IoT scenarios. To learn more, refer to the blog post, “Creating IoT applications with Azure Database for PostgreSQL.” With TimescaleDB, this experience is even better. IoT organizations can now also leverage the insights hidden in machine generated data to build new features, automate processes, and drive efficiency.

Challenge
Solution

IoT devices generate a lot of data which needs to be stored efficiently.
TimescaleDB automatically partitions data into chunks to scale for these types of workloads.

IoT data is complex (i.e. marrying device metadata, geospatial data, and time-series data).
TimescaleDB combines relational capabilities with time-series specific functions and is compatible with other PostgreSQL extensions including PostGIS.

IoT data needs to be accessed by multiple users (i.e. internal users for analytics or external users to expose data in real-time).
TimescaleDB speaks full SQL, a query language that is familiar across entire organizations.

IoT data requires diverse, customizable ingest pipelines that require a database with a broad ecosystem.
TimescaleDB inherits PostgreSQL’s entire ecosystem of tools and extensions.

IoT applications are made up of data at their core, and need to be stored in a reliable database.
TimescaleDB inherits PostgreSQL’s 20+ years of reliability and stability.

TimescaleDB offers valuable performance characteristics on top of PostgreSQL. For IoT use cases that highly leverage time-series data, TimescaleDB implements automatic chunk partitioning to support high insert rates. Below is a comparison on Azure PostgreSQL with and without TimescaleDB and observed degradation in insert performance over time. You can imagine that with IoT use cases with large amounts of time-series data, using TimescaleDB can provide significant value for IoT applications that need both relational features and scalability.

Note: General Purpose Compute Gen 5 with 4 vCores, 20GB RAM with Premium Storage

Although IoT is an obvious use case for a time-series database, time-series data actually exists everywhere. Time-series data is essentially collected over time with an associated timestamp. With TimescaleDB, developers can continue to use PostgreSQL, while leveraging TimescaleDB to scale for time-series workloads.

Next steps

Learn more Azure Database for PostgreSQL and get started using the Azure portal or command line.
Learn more about TimescaleDB by visiting their website, docs, or join their Slack community and post any questions you may have there.

As always, we encourage you to leave feedback below. You can also engage with the Azure Database for PostgreSQL through our feedback page and our forums if you have questions or feature suggestions.
Quelle: Azure