Kategorie: Microsoft Azure

Rewrite HTTP headers with Azure Application Gateway

We are pleased to share the capability to rewrite HTTP headers in Azure Application Gateway. With this, you can add, remove, or update HTTP request and response headers while the request and response packets move between the client and backend application. You can also add conditions to ensure that the headers you specify are rewritten only when the conditions are met. The capability also supports several server variables which help store additional information about the requests and responses, thereby enabling you to make powerful rewrite rules.

Figure 1: Application Gateway removing the port information from the X-Forwarded-For header in the request and modifying the Location header in the response.

Rewriting the headers helps you accomplish several important scenarios. Some of the common use cases are mentioned below.

Remove port information from the X-Forwarded-For header

Application gateway inserts X-Forwarded-For header to all requests before it forwards the requests to the backend. The format of this header is a comma-separated list of IP:Port. However, there may be scenarios where the backend applications require the header to contain only the IP addresses. One such scenario is when the backend application is a Content Management System (CMS) because most CMS are not able to parse the additional port information in the header. For accomplishing such scenarios, you can set the header to the add_x_forwarded_for_proxy server variable which contains the X-Forwarded-For client request header without the port information.

Figure 2: Application Gateway configuration for removing the port information from the X-Forwarded-For header.

Better integration with App service and other multi-tenant backends

When a backend application sends a redirection response, you may want to redirect the client to a different URL than the one specified by the backend application. One such scenario is when an app service is hosted behind an application gateway.

Since app service is a multi-tenant service, it uses the host header in the request to route to the correct endpoint. App services have a default domain name of *.azurewebsites.net (say contoso.azurewebsites.net) which is different from the application gateway's domain name (say contoso.com). Since the original request from the client has application gateway's domain name contoso.com as the host name, the application gateway changes the hostname to contoso.azurewebsites.net, so that the app service in the backend can route it to the correct endpoint. But when the app service sends a redirection response, it uses the same hostname in the location header of its response as the one in the request it receives from the application gateway. Therefore, when the app service performs a redirection to its relative path (redirect from /path1 to /path2), the client will make the request directly to contoso.azurewebsites.net/path2, instead of going through the application gateway (contoso.com/path2). This will bypass the application gateway which is not desirable.

This issue can be resolved by setting the hostname in the location header to the application gateway's domain name. To do this, you can create a rewrite rule with a condition that evaluates if the location header in the response contains azurewebsites.net and performs an action to rewrite the location header to have application gateway's hostname.

Figure 3: Application Gateway configuration for modifying the location header.

Implement security-related HTTP headers to prevent vulnerabilities

Several security vulnerabilities can be fixed by implementing necessary headers in the application response. Some of these security headers are X-XSS-Protection, Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, etc. You can use application gateway to set these headers for all responses.

Get started

You can learn more about HTTP header rewrite with Application Gateway and find detailed instructions about how to configure HTTP header rewrite in Application Gateway.

Please send your feedback

There are a few different routes to provide feedback:

UserVoice: Post new ideas for Application Gateway on our UserVoice page.
Join our cohort: We’re always interested in having new customers join our cohorts to get early access to new features and help us improve Application Gateway going forward. If you are interested in joining our cohorts, simply fill out this quick form.

Quelle: Azure

Machine Learning powered detections with Kusto query language in Azure Sentinel

This post is co-authored by Tim Burrell, Principal Security Engineering Manager and Dotan Patrich, Principal Software Engineer.

As cyberattacks become more complex and harder to detect. The traditional correlation rules of a SIEM are not enough, they are lacking the full context of the attack and can only detect attacks that were seen before. This can result in false negatives and gaps in the environment. In addition, correlation rules require significant maintenance and customization since they may provide different results based on the customer environment.

Advanced Machine Learning capabilities that are built in into Azure Sentinel can detect indicative behaviors of a threat and helps security analysts to learn the expected behavior in their enterprise. In addition, Azure Sentinel provides out-of-the-box detection queries that leverage the Machine Learning capabilities of Azure Monitor Logs query language that can detect suspicious behaviors in such as abnormal traffic in firewall data, suspicious authentication patterns, and resource creation anomalies. The queries can be found in the Azure Sentinel GitHub community.

Below you can find three examples for detections leveraging built in Machine Learning capabilities to protect your environment.

Time series analysis of authentication of user accounts from unusual large number of locations

A typical organization may have many users and many applications using Azure Active Directory for authentication. Some applications (for example Office365 Exchange Online) may have many more authentications than others (say Visual Studio) and thus dominate the data. Users may also have a different location profile depending on the application. For example high location variability for email access may be expected, but less so for development activity associated with Visual Studio authentications. The ability to track location variability for every user/application combination and then investigate just some of the most unusual cases can be achieved by leveraging the built in query capabilities using the operators make-series and series_fit_line.

SigninLogs
| where TimeGenerated >= ago(30d)
| extend locationString= strcat(tostring(LocationDetails["countryOrRegion"]), "/", tostring(LocationDetails["state"]), "/", tostring(LocationDetails["city"]), ";")
| project TimeGenerated, AppDisplayName , UserPrincipalName, locationString
| make-series dLocationCount = dcount(locationString) on TimeGenerated in range(startofday(ago(30d)),now(), 1d)
by UserPrincipalName, AppDisplayName
| extend (RSquare,Slope,Variance,RVariance,Interception,LineFit)=series_fit_line(dLocationCount)
| where Slope >0.3

Creation of an anomalous number of resources

Resource creation in Azure is a normal operation in the environment. Operations and IT teams frequently spin up environments and resources based on the organizational needs and requirements. However, an anomalous creation of resource by users that don’t have permissions or aren’t supposed to create these resources is extremely interesting. Tracking anomalous resources creation or suspicious deployment activities in azure activity log can provide a lead to spot an execution technique done by an attacker.

AzureActivity
| where TimeGenerated >= ago(30d)
| where OperationName == "Create or Update Virtual Machine" or OperationName == "Create Deployment"
| where ActivityStatus == "Succeeded"
| make-series num = dcount(ResourceId) default=0 on EventSubmissionTimestamp in range(ago(30d), now(), 1d) by Caller
| extend outliers=series_outliers(num, "ctukey", 0, 10, 90)
| project-away num
| mvexpand outliers
| where outliers > 0.9
| summarize by Caller

Firewall traffic anomalies

Firewall traffic can be an additional indicator of a potential attack in the organization. The ability to establish a baseline that represents the usual firewall traffic behavior on a weekly or an hourly basis can help point out the anomalous increase in traffic. Using the built-in capabilities in the Log Analytics query language can point directly to the traffic anomaly and be investigated.

CommonSecurityLog
| summarize count() by bin(TimeGenerated, 1h)

With Azure Sentinel, you can create the above advanced detection rules to detect anomalies and suspicious activities in your environment, create your own detection rules or leverage the rich GitHub library that contains detections written by Microsoft security researchers.
Quelle: Azure

Azure.Source – Volume 78

Preview | News & updates | Technical content | Azure shows | Events | Customers, partners, and industries

Now in preview

Hybrid storage performance comes to Azure

When it comes to adding a performance tier between compute and file storage, Avere Systems has led the way with its high-performance caching appliance known as the Avere FXT Edge Filer. Last week at NAB, attendees will got a first look at the new Azure FXT Edge Filer, now with even more performance, memory, SSD, and support for Azure Blob. Since Microsoft’s acquisition of Avere last March, we’ve been working to provide an exciting combination of performance and efficiency to support hybrid storage architectures with the Avere appliance technology. We are currently previewing the FXT 6600 model at customer sites, with a second FXT 6400 model becoming available with general availability.

News and updates

Want to evaluate your cloud analytics provider? Here are the three questions to ask.

In February, an independent study by GigaOm compared Azure SQL Data Warehouse, Amazon Redshift, and Google BigQuery using the highly recognized TPC-H benchmark. They found that Azure SQL Data Warehouse is up to 14 times faster and costs 94 percent less than other cloud providers. And today, we are pleased to announce that in GigaOm’s second benchmark report, this time with the equally important TPC-DS benchmark, Azure SQL Data Warehouse is again the industry leader. Not Amazon Redshift. Not Google BigQuery. These results prove that Azure is the best place for all your analytics.

Introducing the App Service Migration Assistant for ASP.NET applications

In June 2018, we released the App Service Migration Assessment Tool. The Assessment Tool was designed to help customers quickly and easily assess whether a site could be moved to Azure App Service by scanning an externally accessible (HTTP) endpoint. Today we’re pleased to announce the release of an updated version, the App Service Migration Assistant! The new version helps customers and partners move sites identified by the assessment tool by quickly and easily migrating ASP.Net sites to App Service. Read this blog to learn more about the tool and begin your migration.

Expanding Azure IoT certification service to support Azure IoT Edge devices

In December 2018, Microsoft launched the Azure IoT certification service, a web-based test automation workflow to streamline the certification process through self-serve tools. Now we are taking steps to expand the service to  also support Azure IoT Edge Device certification. An Azure IoT Edge device is a device comprised of three key components: IoT Edge modules, IoT Edge runtime, and a cloud-based interface. Learn more about these three components in this blog explaining IoT Edge.

Azure Updates

Learn about important Azure product updates, roadmap, and announcements. Subscribe to notifications to stay informed.

Technical content

Smarter, faster, safer: Azure SQL Data Warehouse is simply unmatched

We want to call attention to the exciting news that Azure SQL Data Warehouse has again outperformed other cloud providers in the most recent GigaOm benchmark report. This is the result of relentless innovation and laser-focused execution on providing new features our customers need, all while reducing prices so customers get industry-leading performance at the best possible value. In this blog, we take a closer look at the technical capabilities of these new features and, most importantly, how you can start using them today.

Azure Security Center exposes crypto miner campaign

Azure Security Center discovered a new cryptocurrency mining operation on Azure customer resources. The operation took advantage of an old version of known open-source CMS, with a known RCE vulnerability as the entry point, and then after using the CRON utility for persistency, it mines “Monero” cryptocurrency using a new compiled binary of the “XMRig” open-source crypto mining tool. Check out our blog for details.

You gotta keep privileges separated

When writing scripts for automation or building out a service, don't run under your own credentials. This creates a single point of failure on you for the service. It's also good practice to separate out concerns between environments. This way even if someone accidentally runs a test command against production, it won't have disastrous results. One recommended approach is to use service principals. An Azure service principal is an identity for use with applications, services, and tools to access Azure resources. Using service principals allows us to assign specific permissions that are limited in scope to precisely what is required so we can minimize the impact if it's compromised! This blog explains how.

How do teams work together on an automated machine learning project?

When it comes to executing a machine learning project in an organization, data scientists, project managers, and business leads need to work together to deploy the best models to meet specific business objectives. A central objective of this step is to identify the key business variables that the analysis needs to predict. We refer to these variables as the model targets, and we use the metrics associated with them to determine the success of the project. In this use case, we look at how a data scientist, project manager, and business lead at a retail grocer can leverage automated machine learning and Azure Machine Learning service to reduce product overstock.

How to Use Azure Pipeline Task and Job Conditions

An Azure Pipeline Job is a grouping of tasks that run sequentially on the same target. In many cases, you will want to only execute a task or a job if a specific condition has been met. Azure Pipeline conditions allow us to define conditions under which a task or job will execute. In this blog, we will detail a common situation in which pipeline conditions are helpful, the configuration of this condition, and what documentation links offer more information.

Moving your database to Azure

In this session we show you how we migrated an on-premises MongoDB database to Azure Cosmos DB and SQL Server database to an Azure SQL Server Managed Instance. You’ll learn about data preparation decisions, performing the migration, and ensuring your application has zero downtime while switching over to the cloud hosted database providers.

Azure Stack IaaS – part seven of a series

Most apps get delivered by a team. When your team delivers the app through virtual machine (VMs), it is important to coordinate efforts. Born in the cloud to serve teams from all over the world, Azure and Azure Stack have some handy capabilities to help you coordinate VM operations across your team. In this blog, we look at features such as single sign-in, role-based access, and collaborating with people outside your organization.

How to accelerate DevOps with Machine Learning lifecycle management

DevOps is the union of people, processes, and products to enable the continuous delivery of value to end users. DevOps for machine learning is about bringing the lifecycle management of DevOps to Machine Learning. Utilizing Machine Learning, DevOps can easily manage, monitor, and version models while simplifying workflows and the collaboration process. Effectively managing the Machine Learning lifecycle is critical for DevOps’ success. And the first piece to machine learning lifecycle management is building your machine learning pipeline or pipelines. We explain how.

How do teams work together on an automated machine learning project?

When it comes to executing a machine learning project in an organization, data scientists, project managers, and business leads need to work together to deploy the best models to meet specific business objectives. A central objective of this step is to identify the key business variables that the analysis needs to predict. We refer to these variables as the model targets, and we use the metrics associated with them to determine the success of the project.

How to stay informed about Azure service issues

Azure Service Health helps you stay informed and take action when Azure service issues like outages and planned maintenance affect you. It provides you with a personalized dashboard that can help you understand issues that may be impacting resources in your Azure subscriptions. For any event, you can get guidance and support, share details with your colleagues, and receive issue updates. We’ve posted a new video series to help you learn how to use Azure Service Health and ensure you stay on top of service issues.

How to stay on top of Azure best practices

Optimizing your cloud workloads can seem like a complex and daunting task. We created Azure Advisor, a personalized guide to Azure best practices, to make it easier to get the most out of Azure.

How Skype modernized its backend infrastructure using Azure Cosmos DB

Founded in 2003, Skype has grown to become one of the world’s premier communication services, making it simple to share experiences with others wherever they are. Since its acquisition by Microsoft in 2010, Skype has grown to more than four billion total users, more than 300 million monthly active users, and more than 40 million concurrent users. In a three-part series, we discuss how Skype used Azure Cosmos DB to solve real-world challenges.

Azure shows

Episode 274 – Reliability Engineering | The Azure Podcast

David Blank-Edelman, a Senior Cloud Advocate at Microsoft, gives us some great insight into what customers should be thinking about when it comes to the reliability of their cloud applications.

HTML5 audio not supported

Using the new Basic Process in Azure DevOps | DevOps Lab

In this episode, Abel chats with Dan Hellem to walk through the details of the new Basic process in Azure DevOps and learn how it works.

Redis Edge on Azure IoT Edge | Internet of Things Show

RedisEdge from Redis Labs is a purpose-built database for the demanding conditions at the IoT edge. It has the ability to ingest millions of writes per second with <1ms latency, has a 5MB footprint, and is available on ARM32, ARM64, and x64 architectures.

Azure Monitor action groups | Azure Friday

Azure Monitor action groups enable you to define a list of actions to execute when an alert is triggered. In this episode, we demonstrate how to configure a Service Health alert to use an action group.

How to test Azure Functions | Azure Tips & Tricks

In this edition of Azure Tips and Tricks, learn how to test Azure Functions with unit and integration test methods.

Management Groups, Policy, and Blueprints in Azure Governance | Microsoft Mechanics – Azure

The latest on governing Azure subscriptions for Cloud Architects or Ops Managers. Satya Vel, from the Azure Governance Team, demonstrates Microsoft's approach to Azure Governance overall, which now includes more granular control of policy across different apps and departments in your organization with management groups. You'll also see the new Azure Blueprint templates that simplify setting up your environment to meet specific compliance requirements such as ISO, as well as easier tracking of policy changes and their impact. We'll show you how you can now apply governance capabilities across your Azure Kubernetes workloads.

Party with Palermo at the Microsoft MVP Summit | Azure DevOps Podcast

This week Jeffrey Palermo has a special episode for you all! It is recorded live, from the night before the Microsoft MVP Summit, at Jeffrey’s annual “Party with Palermo!” get-together for MVPs.

HTML5 audio not supported

Episode 6 – AI Forensics and Pharaoh Hounds | AzureABILITY Podcast

AI/Machine Learning pioneer Andre Magni visits the pod to talk computer intelligence; from Microsoft's AI mission (to amplify human ingenuity with intelligent technology) to data-curation gotchas and modelling pitfalls to identifying dead bodies using AI.

HTML5 audio not supported

Events

Countdown for Microsoft Build: Things to Do Part 1

Get ready to see the awesome sights of Seattle while you're at Microsoft Build this May, including the Museum of Pop Culture and Wings over Washington.

Microsoft at SAP Sapphire NOW 2019: A trusted path to cloud innovation

In a few weeks, more than 22,000 people from around the globe will converge in Orlando, Florida May 7-9  for the SAP Sapphire NOW and ASUG Annual Conference. Each year, the event brings together thought leaders across industries to find innovative ways to solve common challenges, unlock new opportunities, and take advantage of emerging technologies that are changing the business landscape as we know it. This year, Microsoft has elevated its presence with engaging in-booth experiences and informative sessions that will educate, intrigue, and inspire attendees as they take the next step in their digital transformation journey.

Customers, partners, and industries

Bitnami Apache Airflow Multi-Tier now available in Azure Marketplace

A few months ago, we released a blog post that provided guidance on how to deploy Apache Airflow on Azure. The template in the blog provided a good quick start solution for anyone looking to quickly run and deploy Apache Airflow on Azure in sequential executor mode for testing and proof of concept study.

Leveraging AI and digital twins to transform manufacturing with Sight Machine

Azure has mastered ingesting and storing manufacturing data with services such as Azure IoT Hub and Azure Data Lake, and now our partner Sight Machine has solved for the other huge challenge: data variety. Sight Machine on Azure is a leading AI-enabled analytics platform that enables manufacturers to normalize and contextualize plant floor data in real-time. The creation of these digital twins allows them to find new insights, transform operations, and unlock new value.

Azure AI does that?

Whether you’re just starting off in tech, building, managing, or deploying apps, gathering and analyzing data, or solving global issues —anyone can benefit from using cloud technology. In this post we’ve gathered five cool examples of innovative artificial intelligence (AI) to showcase how you can be a catalyst for real change.

Azure Front Door gets WAF support, a new Premium plan for Azure Functions & changes to Azure alerts | Azure This Week – A Cloud Guru

This time on Azure This Week, Lars covers Azure Front Door which gets Web Application Firewall support, Azure Functions get a new Premium plan for more serverless action, Azure alerts get an overhaul, and a new series – "Azure Fireside Chats" launches on A Cloud Guru.

Quelle: Azure

Deploying Grafana for production deployments on Azure

This blog is co-authored by Nick Lopez, Technical Advisor at Microsoft.

Grafana is one of the popular and leading open source tools for visualizing time series metrics. Grafana has quickly become the preferred visualization tool of choice for developers and operations teams for monitoring server and application metrics. Grafana dashboards enable operation teams to quickly monitor and react to performance, availability, and overall health of the service. You can now also use it to monitor Azure services and applications by leveraging the Azure Monitor data source plugin, built by Grafana Labs. This plugin enables you to include all metrics from Azure Monitor and Application Insights in your Grafana dashboards. If you would like to quickly setup and test Grafana with Azure Monitor and Application Insights metrics, we recommend you refer to the Azure Monitor Documentation.

 

Grafana server image in Azure Marketplace provides a great QuickStart deployment experience. The image provisions a virtual machine (VM) with a pre-installed Grafana dashboard server, SQLite database  and the Azure plugin. The default setup with a single VM deployment is great for a proof of concept study and testing. For high availability of monitoring dashboards for your critical applications and services, it’s essential to think of high availability of Grafana deployments on Azure. The following is the proposed and proven architecture to setup Grafana for high availability and security on Azure.

Setting up Grafana for production deployments

Grafana Labs recommends setting up a separate highly available shared MySQL server for setting up Grafana for high availability. The Azure Database for MySQL and MariaDB are managed relational database services based on the community edition of MySQL and the MariaDB database engine. The service provides high availability at no additional cost, predictable performance, elastic scalability, automated backups and enterprise grade security with secure sockets layer (SSL) support, encryption at rest, advanced threat protection, and VNet service endpoint support. Utilizing a remote configuration database with Azure Database for MySQL or Azure Database for MariaDB service allows for horizontal scalability and high availability of Grafana instances required for enterprise production deployments.

Leveraging Bitnami Multi-Tier Grafana templates for production deployments

Bitnami lets you deploy a multi-node, production ready Grafana solution from the Azure Marketplace with just a few clicks. This solution uses several Grafana nodes with a pre-configured load balancer and Azure Database for MariaDB for data storage. The number of nodes can be chosen at deployment time depending on your requirements. Communication between the nodes and the Azure Database for MariaDB service is also encrypted with SSL to ensure security.

A key feature of Bitnami's Grafana solution is that it comes pre-configured to provide a fault-tolerant deployment. Requests are handled by the load balancer, which continuously tests nodes to check if they are alive and automatically reroutes requests if a node fails. Data (including session data) is stored in the Azure Database for MariaDB and not on the individual nodes. This approach improves performance and protects against data loss due to node failure.

For new deployments, you can launch Bitnami Grafana Multi-Tier through the Azure Marketplace!

Configuring existing installations of Grafana to use Azure Database for MySQL service

If you have an existing installation of Grafana that you would like to configure for high availability, you can use the following steps that demonstrate configuring Grafana instance to use Azure Database for MySQL server as the backend configuration database. In this walkthrough, we will be using an example of Ubuntu with Grafana installed and configure Azure Database for MySQL as a remote database for Grafana setup.

Create an Azure Database for MySQL server with the General Purpose tier which is recommended for production deployments. If you are not familiar with the database server creation, you can read the QuickStart tutorial to familiarize yourself with the workflow. If you are using Azure CLI, you can simply set it up using az mysql up.
If you have already installed Grafana on the Ubuntu server, you’ll need to edit the grafana.ini file to add the Azure Database for MySQL parameters. As per the Grafana documentation on the Database settings, we will focus on the database parameters noted in the documentation. Please note: The username must be in the format user@server due to the server identification method of Azure Database for MySQL. Other formats will cause connections to fail.
Azure Database for MySQL supports SSL connections. For enterprise production deployments, it is recommended to always enforce SSL. Additional information around setting up SSL with Azure Database for MySQL can be found in the Azure Database for MySQL documentation. Most modern installations of Ubuntu will have the necessary Baltimore Cyber Trust CA certificate already installed in your /etc/ssl/certs location. If needed, you can download the SSL Certificate CA used for Azure Database for MySQL from  this location. The SSL mode can be provided in two forms, skip-verify and true. With skip-verify we will not validate the certificate provided but the connection is still encrypted. With true we are going to ensure that the certificate provided is validated   by the Baltimore CA. This is useful for preventing “man in the middle” attacks. Note that in both situations, Grafana expects the certificate authority (CA) path to be provided.
Next, you have the option to store the sessions of users in the Azure DB for MySQL in the table session. This is configured in the same grafana.ini under the session section. This is beneficial for instance in situations where you have load balanced environments to maintain sessions for users accessing Grafana. In the provider_config parameter, we need to include the user@server, password, full server and the TLS/SSL method. In this manner, this can be true or ssl-verify. Note that this is the go-sql-driver/mysql driver where more documentation is available.
After this is all set, you should be able to start Grafana and verify the status with the commands below:

systemctl start grafana-server
systemctl status grafana-server

If you see any errors or issues, the default path for logging is /var/log/grafana/ where you can confirm what is preventing the startup. The following is a sample error where the username was not provided as user@server but rather just user.

lvl=eror msg="Server shutdown" logger=server reason="Service init failed: Migration failed err: Error 9999: An internal error has occurred. Please retry or report your issues.

Otherwise you should see the service in an Ok status and the initial startup will build all the necessary tables in the Azure DB for MySQL database.

Key takeaways

The single VM setup for Grafana is great for quick start, testing and a proof of concept study but it may not be suitable for production deployments.
For enterprise production deployments of Grafana, separating the configuration database to the dedicated server enables high availability and scalability.
The Bitnami Grafana Multi-Tier template provides production ready template leveraging the scale out design and security to provision Grafana with a few clicks with no extra cost.
Using managed database services like Azure Database for MySQL for production deployments provides built-in high availability, scalability, and enterprise security for the database repository.

Additional resources

Get started with Bitnami Multi-Tier Solutions on Microsoft Azure

Monitor Azure services and applications using Grafana

Monitor your Azure services in Grafana

Setting up Grafana for high availability

Azure Database for MySQL documentation

Acknowledgments

Special thanks to Shau Phang, Diana Putnam, Anitah Cantele and Bitnami team for their contributions to the blog post.
Quelle: Azure