Faster and cheaper: SQL on Azure continues to outshine AWS

Over a million on-premises SQL Server databases have moved to Azure, representing a massive shift in where customers are collecting, storing, and analyzing their data.

Modernizing your databases provides the opportunity to transform your data architecture. SQL Server on Azure Virtual Machines allows you to maintain control over your database and operating system while still benefiting from cloud flexibility and scale. For some, this represents a step in the journey to a fully-managed database, while others choose this deployment option for compatibility with on-premises workloads such as SQL Server Reporting Services.

Whatever the reason, migrating SQL workloads to Azure Virtual Machines is a popular option. Azure customers benefit from our unique built-in security and manageability capabilities, which automate tasks like patching and backups. In addition to providing these unparalleled innovations, it is important to provide customers with the best price-performance possible. Once again, SQL Server on Azure Virtual Machines comes out on top.

SQL Server on Azure leads in price-performance

GigaOm, an independent research firm, recently published a study comparing throughput performance between SQL Server on Azure Virtual Machines and SQL Server on AWS EC2. Azure emerged as the clear leader across both Windows and Linux for mission-critical workloads, up to 3.4 times faster and up to 87 percent less expensive than AWS EC2.1

The images above are performance and price-performance comparisons from the GigaOm report. The performance metric is throughput (transactions per second, tps); higher performance is better. The price-performance metric is three-year pricing divided by throughput (transactions per second, tps), lower price-performance is better.

With Azure Ultra Disk, GigaOm was able to achieve 80,000 input or output per second (IOPS) per single disk, maxing out the virtual machine’s throughput limit, and well exceeding the capabilities of AWS provisioned IOPS.2

A key reason why Azure price-performance is superior to AWS is Azure BlobCache, which provides free reads. Given that most online transaction processing (OLTP) workloads today come with a ten-to-one read-to-write ratio, this provides customers with significant savings.

Unmatched innovation from the team that brought SQL Server to the world

With a proven track record over 25 years, the engineering team behind SQL Server continues to drive security and innovation to meet our customers’ changing needs. Whether executing on-premises, in the cloud, or on the edge, the result is the most comprehensive, consistent, and secure solution for your data.

Azure SQL Virtual Machines offer unique built-in security and manageability, including automatic security patching and automated high-availability, and database recovery to a specific point in time. Azure’s unique security capabilities include advanced data security for SQL Server on Azure Virtual Machines, which enables both vulnerability assessments and advanced threat protection. Customers self-installing SQL Server on virtual machines in the cloud can now register with our resource provider to enable this same functionality.

Get started with SQL in Azure today

Migrate from SQL Server on-premises to SQL Server 2019 in Azure Virtual Machines today. Get started with preconfigured Azure SQL Virtual Machine images on Red Hat Enterprise Linux, SUSE Linux Enterprise Server, Ubuntu, and Windows in minutes. Take advantage of the Azure Hybrid Benefit to reuse your existing on-premises Windows server and SQL Server licenses in Azure for significant savings.

When you add it up, SQL databases are simply best on Azure. Learn more about why SQL Server is best on Azure, and use a $200 in Azure credits with a free account3 or Azure Dev or Test credits4 for additional cost savings.

 

1Price-performance claims based on data from a study commissioned by Microsoft and conducted by GigaOm in October 2019. The study compared price performance between SQL Server 2017 Enterprise Edition on Windows Server 2016 Datacenter edition in Azure E64s_v3 instance type with 4x P30 1TB Storage Pool data (Read-Only Cache) + 1x P20 0.5TB log (No Cache) and the SQL Server 2017 Enterprise Edition on Windows Server 2016 Datacenter edition in AWS EC2 r4.16xlarge instance type with 1x 4TB gp2 data + 1x 1TB gp2 log. Benchmark data is taken from a GigaOm Analytic Field Test derived from a recognized industry standard, TPC Benchmark™ E (TPC-E). The Field Test does not implement the full TPC-E benchmark and as such is not comparable to any published TPC-E benchmarks. The Field Test is based on a mixture of read-only and update intensive transactions that simulate activities found in complex OLTP application environments. Price-performance is calculated by GigaOm as the cost of running the cloud platform continuously for three years divided by transactions per second throughput. Prices are based on publicly available US pricing in West US for SQL Server on Azure Virtual Machines and Northern California for AWS EC2 as of October 2019. The pricing incorporates three-year reservations for Azure and AWS compute pricing, and Azure Hybrid Benefit for SQL Server and Azure Hybrid Benefit for Windows Server and License Mobility for SQL Server in AWS, excluding Software Assurance costs.  Price-performance results are based upon the configurations detailed in the GigaOm Analytic Field Test.  Actual results and prices may vary based on configuration and region.

2Claims based on data from a study commissioned by Microsoft and conducted by GigaOm in October 2019. The study compared price-performance between SQL Server 2017 Enterprise Edition on Windows Server 2016 Datacenter edition in Azure E64s_v3 instance type with 1x Ultra 1.5TB with 650MB per sec throughput and the SQL Server 2017 Enterprise Edition on Windows Server 2016 Datacenter edition in AWS EC2 r4.16xlarge instance type with 1x 1.5TB io1 provisioned log + data. Benchmark data is taken from a GigaOm Analytic Field Test derived from a recognized industry standard, TPC Benchmark™ E (TPC-E). The Field Test does not implement the full TPC-E benchmark and as such is not comparable to any published TPC-E benchmarks. The Field Test is based on a mixture of read-only and update intensive transactions that simulate activities found in complex OLTP application environments. Price-performance is calculated by GigaOm as the cost of running the cloud platform continuously for three years divided by transactions per second throughput. Prices are based on publicly available US pricing in north Europe for SQL Server on Azure Virtual Machines and Ireland for AWS EC2 as of October 2019. Price-performance results are based upon the configurations detailed in the GigaOm Analytic Field Test.  Actual results and prices may vary based on configuration and region.

3Additional information about $200 Azure free account available at https://azure.microsoft.com/en-us/free/.

4Dev or Test Azure credits and pricing available for paid Visual Studio subscribers only.
Quelle: Azure

SAP HANA backup using Azure Backup is now generally available

Today, we are sharing the general availability of Microsoft Azure Backup’s solution for SAP HANA databases in the UK South region.

Azure Backup is Azure's native backup solution, which is BackInt certified by SAP. This offering aligns with Azure Backup’s mantra of zero-infrastructure backups, eliminating the need to deploy and manage backup infrastructure. You can now seamlessly backup and restore SAP HANA databases running on Microsoft Azure Virtual Machines (VM) — M series Virtual Machine is also supported, and leverage enterprise management capabilities that Azure Backup provides.

Benefits

15-minute Recovery Point Objective (RPO): Recovery of critical data of up to 15 minutes is possible.
One-click, point-in-time restores: Easy to restore production data on SAP HANA databases to alternate servers. Chaining of backups and catalogs to perform restores is all managed by Azure behind the scenes.
Long-term retention: For rigorous compliance and audit needs, you can retain your backups for years, based on the retention duration, beyond which the recovery points will be pruned automatically by the built-in lifecycle management capability.
Backup Management from Azure: Use Azure Backup’s management and monitoring capabilities for improved management experience.

Watch this space for more updates on GA rollout to other regions. We are currently in preview in these Azure geos.

Getting started

We are working on making the SAP HANA backup experience even better. Find out the scenarios we support today.
See how to backup and restore SAP HANA databases.
Manage and Monitor the backed up SAP HANA databases.
Need help? Read the troubleshooting documentation. Or reach out to the Azure Backup forum for support.
Learn more about Azure backup.
Follow us on twitter @AzureBackup for more updates.

Quelle: Azure

Azure Cost Management updates – November 2019

Whether you're a new student, thriving startup, or the largest enterprise, you have financial constraints and you need to know what you're spending, where, and how to plan for the future. Nobody wants a surprise when it comes to the bill, and this is where Microsoft Azure Cost Management comes in.

We're always looking for ways to learn more about your challenges and how Cost Management can help you better understand where you're accruing costs in the cloud, identify and prevent bad spending patterns, and optimize costs to empower you to do more with less. Here are a few of the latest improvements and updates based on your feedback:

Cost Management now available for Cloud Solution Providers
What's new in Cost Management Labs
Customizing the name on dashboard tiles
Upcoming changes to Azure usage data
Save up to 72% with Azure reservations–now available for 16 services
New videos
Documentation updates

Let's dig into the details.

Cost Management now available for Cloud Solution Providers

In case you missed it, as of November 1, Cloud Solution Provider (CSP) partners can now see and manage costs for their customers using Azure Cost Management in the Azure portal by transitioning them to Azure plan subscriptions via Microsoft Customer Agreement. Partners can also enable Azure Cost Management for customers to allow them to see and manage the cost of their subscriptions.

If you're working with a CSP partner to manage your Azure subscriptions, talk to them about getting you onboarded and your subscriptions switched over to the new Azure plan using Microsoft Customer Agreement. Not only will this allow you to see and manage costs in the Azure portal, but you'll also be able to use some Azure services that aren't currently available to your classic CSP subscriptions. As an example, some organizations have dependencies on external solutions that still require classic services, including virtual machines. To work around this, organizations are creating separate pay-as-you-go subscriptions for those resources. This adds additional overhead to manage separate billing accounts with Microsoft and your partner. Once you've switched over to Azure plan subscriptions, you may be able to consolidate any existing CSP and non-CSP subscriptions into a single billing account, managed by your partner. In general, you'll have the same benefits and offerings at the same time as everyone else using Microsoft Customer Agreement. Make sure you talk to your partner today!

If you're a CSP provider, enabling Cost Management for your customers involves three steps:

Confirm acceptance of the Microsoft Customer Agreement on behalf of your customers
Present the Microsoft Customer Agreement to your customers and, once they've agreed, confirm the customer's official acceptance in Partner Center or via the API/SDK.
Transition your customers to Azure plan
The last step for you, as the partner, to see and manage cost in the Azure portal is to transition existing CSP offers to an Azure plan. You'll need to do this once for each reseller and direct customer.
Enable Azure Cost Management for your customers
In order for your customers to see and manage costs in Azure Cost Management, they need to have access to view charges for their subscriptions. This can be enabled from the Azure portal for each customer and shows them their cost based on pay-as-you-go prices and do not include partner discounts or any discounts you may offer. Please ensure your customers understand the cost will not match your invoice if you offer additional discounts or use custom prices.

To learn more about what you'll see after enabling Azure Cost Management for your customers, read Get started with Azure Cost Management for partners.

What's new in Cost Management Labs

With Cost Management Labs, you get a sneak peek at what's coming in Azure Cost Management and can engage directly with us to share feedback and help us better understand how you use the service so we can deliver more tuned and optimized experiences. Here are a few features you can see in Cost Management Labs:

Get started quicker with the cost analysis Home view
Cost Management offers five built-in views to get started with understanding and drilling into your costs. The Home view gives you quick access to those views so you get to what you need faster.
Performance optimizations in cost analysis and dashboard tiles—Now available in the public portal
Whether you're using tiles pinned to the dashboard or the full experience, you'll find cost analysis loads faster than ever.
NEW: Show views name on pinned cost analysis tiles—Now available in the public portal
When you pin cost analysis to the dashboard, it now shows the name of the view you pinned. To change it, simply save the view with the desired name and pin cost analysis again!
NEW: Quick access to cost analysis help and support—Now available in the public portal
Have a question? Need help? The quickstart tutorial is now one click away in cost analysis. And if you run into an issue, create a support request from cost analysis to send additional context to help you submit and resolve your issue quicker than ever.

Of course, that's not all. Every change in Cost Management is available in Cost Management Labs a week before it's in the full Azure portal. We're eager to hear your thoughts and understand what you'd like to see next. What are you waiting for? Try Cost Management Labs today.

Customizing the name on dashboard tiles

You already know you can save and share views in cost analysis. You'll typically start by saving a customized view in cost analysis so others can use it. You might share a link so they can jump directly into the view from outside the portal or share an image of the view to include in an email or presentation. But if you really want to keep an eye on specific perspectives of your cost every time you sign in to the portal, the best option is to pin your view to the dashboard.

Pinning is easy: Just click the pin icon in the top-right corner of cost analysis and you're done. When you pin your view, the tile shows the name of your view, the scope it represents, and the main chart or table from cost analysis. If you have an older tile you need to rename, open it in cost analysis, click Save as to change the name of the view, then pin it again.

Enjoy and let us know what you'd like to see next!

Upcoming changes to Azure usage data

Many organizations use the full Azure usage and charges to understand what's being used, identify what charges should be internally billed to which teams, and to look for opportunities to optimize costs with Azure reservations and Azure Hybrid Benefit. If you're doing any analysis or setup integration based on product details in the usage data, please update your logic for the following services. All of the following changes will start effective December 1:

VNet Gateway service will become VPN Gateway.
Process Automation Watcher will have a new meter ID.
Azure Monitor custom metrics will become Network watcher perf monitor connection metrics.

Also, remember the key-based EA billing APIs have been replaced by new Azure Resource Manager APIs. The key-based APIs will still work through the end of your enrollment, but will no longer be available when you renew and transition into Microsoft Customer Agreement. Please plan your migration to the latest version of the UsageDetails API to ease your transition to Microsoft Customer Agreement at your next renewal.

Save up to 72 percent with Azure reservations – now available for 16 services

Azure reservations help you save up to 72% compared to pay-as-you-go rates when you commit to one or three years of usage. You may know Azure Advisor tells you when you can save money with virtual machine reservations, but did you know with the addition of six new services, you can now purchase reservations for a total of 16 services? Here's the full list as of today:

Virtual machines and managed disks
Blob storage
App Service
SQL database and data warehouse
Azure Database for MySQL, MariaDB, and PostgreSQL
Cosmos DB
Data Explorer
Databricks
SUSE and Red Hat Linux
Azure Red Hat OpenShift
Azure VMWare solution by CloudSimple

What services would you like to see next? Learn more about Azure reservations and start saving today!

New videos

If you weren't able to make it to Microsoft Ignite 2019 or didn't catch the Azure Cost Management sessions, they're now available online and open for everyone:

Analyze, manage, and optimize your cloud cost with Azure Cost Management (46 minutes)
Learn how Azure Cost Management can help you gain visibility, drive accountability, and optimize your cloud costs. Special guest, Mars Inc, will show how they use Azure Cost Management to get the most value out of Azure.
 
Manage and optimize your cloud cost with Azure Cost Management (21 minutes)
Just getting started with Azure? Get a quick view of how you can use Azure Cost Management as you use Can't make the full hour? Join us for a quick overview of Azure Cost Management in this short, theater session.

If you're looking for something a little shorter, you can also check out these videos:

Azure Cost Management at a glance (6 minutes)
Azure Cost Management overview (14 minutes)

Subscribe to the Azure Cost Management YouTube channel to stay in the loop with new videos as they're released and let us know what you'd like to see next.

Documentation updates

There were many documentation updates. Here are a few you might be interested in:

Added information about Data Explorer reservations
Outlined a set of account management tasks available in the Azure portal for account admins
Introducing the Microsoft Cloud Adoption Framework for Azure

Want to keep an eye on all of the documentation updates? Check out the Cost Management doc change history in the azure-docs repository on GitHub. If you see something missing, select Edit at the top of the document and submit a quick pull request.

What's next?

These are just a few of the big updates from last month. We're always listening and making constant improvements based on your feedback, so please keep the feedback coming.

Follow @AzureCostMgmt on Twitter and subscribe to the YouTube channel for updates, tips, and tricks. And, as always, share your ideas and vote up others in the Cost Management feedback forum.
Quelle: Azure

Application Gateway Ingress Controller for Azure Kubernetes Service

Today we are excited to offer a new solution to bind Azure Kubernetes Service (AKS) and Application Gateway. The new solution provides an open source Application Gateway Ingress Controller (AGIC) for Kubernetes, which makes it possible for AKS customers to leverage Application Gateway to expose their cloud software to the Internet.

Bringing together the benefits of the Azure Kubernetes Service, our managed Kubernetes service, which makes it easy to operate advanced Kubernetes environments and Azure Application Gateway, our native, scalable, and highly available, L7 load balancer has been highly requested by our customers.

How does it work?

Application Gateway Ingress Controller runs in its own pod on the customer’s AKS. Ingress Controller monitors a subset of Kubernetes’ resources for changes. The state of the AKS cluster is translated to Application Gateway specific configuration and applied to the Azure Resource Manager. The continuous re-configuration of Application Gateway ensures uninterrupted flow of traffic to AKS’ services. The diagram below illustrates the flow of state and configuration changes from the Kubernetes API, via Application Gateway Ingress Controller, to Resource Manager and then Application Gateway.

Much like the most popular Kubernetes Ingress Controllers, the Application Gateway Ingress Controller provides several features, leveraging Azure’s native Application Gateway L7 load balancer. To name a few:

URL routing
Cookie-based affinity
Secure Sockets Layer (SSL) termination
End-to-end SSL
Support for public, private, and hybrid web sites
Integrated web application firewall

The architecture of the Application Gateway Ingress Controller differs from that of a traditional in-cluster L7 load balancer. The architectural differences are shown in this diagram:

An in-cluster load balancer performs all data path operations leveraging the Kubernetes cluster’s compute resources. It competes for resources with the business apps it is fronting. In-cluster ingress controllers create Kubernetes Service Resources and leverage kubenet for network traffic. In comparison to Ingress Controller, traffic flows through an extra hop.
Ingress Controller leverages the AKS’ advanced networking, which allocates an IP address for each pod from the subnet shared with Application Gateway. Application Gateway has direct access to all Kubernetes pods. This eliminates the need for data to pass through kubenet. For more information on this topic see our “Network concepts for applications in Azure Kubernetes Service” article, specifically “Comparing network models” section.

Solution performance

As a result of Application Gateway having direct connectivity to the Kubernetes pods, the Application Gateway Ingress Controller can achieve up to 50 percent lower network latency vs in-cluster ingress controllers. Application Gateway is a managed service, backed by Azure virtual machine scale sets. As a result, Application Gateway does not use AKS compute resources for data path processing. It does not share or interfere with the resources allocated to the Kubernetes deployment. Autoscaling Application Gateway at peak times, unlike an in-cluster ingress, will not impede the ability to quickly scale up the apps’ pods. And of course, switching from in-cluster L7 ingress to Application Gateway will immediately decrease the compute load used by AKS.

We compared the performance of an in-cluster ingress controller and Application Gateway Ingress Controller on a three node AKS cluster with a simple web app running 22 pods per node. A total of 66 web app pods shared resources with three in-cluster ingresses – one per node. We configured Application Gateway with an instance count of two. We used Apache Bench to create a total of 100K requests with concurrency set at 3K requests. We launched Apache Bench twice: once pointing it to the SLB fronting the in-cluster ingress controller, and a second time connecting to the public IP of Application Gateway. On this very busy AKS cluster we recorded the mean latency across all requests:

Application Gateway: 480ms per request
In-cluster Ingress: 710ms per request

As proven by the data gathered above, under heavy load, the in-cluster ingress controller has approximately 48 percent higher latency per request compared to Application Gateway ingress. Running the same benchmark on the same cluster but with two web app pods per node, a total of six pods, we observed the in-cluster ingress controller performing with approximately 17 percent higher latency than Application Gateway.

What’s next?

Application Gateway Ingress Controller is now stable and available for use in production environments. The project is maturing quickly, and we are working actively to add new capabilities. We are working on enhancing the product with features that customers have been asking for, such as using certificates stored on Application Gateway, mutual TLS authentication, gRPC, and HTTP/2. We invite you to try the new Application Gateway Ingress Controller for AKS, follow our progress, and most importantly – give us feedback on GitHub.
Quelle: Azure

Disaster Recovery with GitOps

Whether your OpenShift cluster(s) are hosted on-premise or in the cloud downtime happens. This could be a temporary outage or it can be an extended outage with no resolution in sight. This article will explain how GitOps can be used for the rapid redeployment of your Kubernetes objects. One important thing to note is that GitOps can only restore Kubernetes objects so that means any persistent data required for an application to correctly function must be restored for stateful applications, such as databases, to be back in service.
Continuing with our usage of Argo CD we will discuss two different ways to start the process in restoring these objects. For both of these procedures we will assume that a new cluster has been deployed and that Argo CD has also been deployed. We also assume that the same OpenShift routes and DNS zones will be used because the OpenShift routes should be stored within git as well.
Using the Argo CD binary you will manually need to define the repositories and Argo CD applications. This process will require you to have a list of repositories and command to define them within Argo CD. For example, we could run the following to restore our simple-app project.
argocd repo add https://github.com/cooktheryan/blogpost
argocd app create –project default
–name simple-app –repo https://github.com/cooktheryan/blogpost.git
–path . –dest-server https://kubernetes.default.svc
-dest-namespace simple-app –revision master

This process works as long as you have a list of all repositories, git branches, and namespaces documented. Once these items are all defined and loaded into Argo CD, the objects will begin to deploy within the cluster and sync with Argo CD.
With some planning we can make this process better though by using git to manage our GitOps resources. Storing a copy of the configmap and the various ArgoCD applications within Git or even something simple as a file or object share that exists outside of the data center hosting the OpenShift cluster will allow us to rapidly redefine the objects managed by Argo CD.
First, let’s take a look at the configmap in YAML format. We will see the repositories currently defined within Argo CD.
oc get configmap -n argocd argocd-cm -o yaml
apiVersion: v1
data:
repositories: |
– url: https://github.com/cooktheryan/blogpost
– url: http://github.com/openshift/federation-dev.git
– sshPrivateKeySecret:
key: sshPrivateKey
name: repo-federation-dev-3296805493
url: git@github.com:openshift/federation-dev.git
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:”v1″,”kind”:”ConfigMap”,”metadata”:{“annotations”:{},”labels”:{“app.kubernetes.io/name”:”argocd-cm”,”app.kubernetes.io/part-of”:”argocd”},”name”:”argocd-cm”,”namespace”:”argocd”}}
creationTimestamp: “2019-09-26T18:46:47Z”
labels:
app.kubernetes.io/name: argocd-cm
app.kubernetes.io/part-of: argocd
name: argocd-cm
namespace: argocd
resourceVersion: “474704”
selfLink: /api/v1/namespaces/argocd/configmaps/argocd-cm
uid: fe331084-e08d-11e9-a49a-52fdfc072182

We will next save the configmap in YAML format.
oc get configmap -n argocd argocd-cm -o yaml –export > argocd-cm.yaml

But what if my repository requires a ssh key? If that is the case then we will need to export
the secret as well. If your repositories do not require a ssh key or authentication then ignore this step.
The configmap identifies the name of the secret that is used by the repository.
oc get secrets -n argocd repo-federation-dev-3296805493 -o yaml > repo-federation-dev-secret.yaml

We will now need to backup any Argo CD applications. This can be done per individual application or by just exporting
all of the applications to a YAML file. For this example since we only have one application within Argo CD.
It would be suggested to store the applications individually and within the same git repository that the Kubernetes
objects are defined so that they will be under revision control and available in the event of a disaster.
oc get applications -o yaml –export > simple-app-backup.yaml

Now that we have all of required Argo CD objects we will now import them into our new server that has been deployed when
the new environment was brought online.
First, we will update the configmap to include our previously defined repositories.
oc apply -f argocd-repos.yaml -n argocd-cm.yaml

OPTIONAL: If credentials were used for any of the repositories then the credentials in the secret must be imported before running a repo list.
oc apply -f repo-federation-dev-secret.yaml -n argocd

Next, we will restore our Argo CD applications which will cause the our Kubernetes objects like namespaces, services, deployments, and routes to deploy
onto the cluster.
oc create -f backup.yaml -n argocd

At this point all of the objects should begin to deploy and the applications within Argo CD should post a healthy state. As with
all backup solutions it makes sense to test this DR procedure frequently. This could be done per application basis on another cluster or with
Code Ready Containers.
In the coming weeks we will publish another disaster recovery post containing information on what to do if your cluster fails and how
a Global Load Balancer can keep the lights on.
The post Disaster Recovery with GitOps appeared first on Red Hat OpenShift Blog.
Quelle: OpenShift

Unique Identifier helps troubleshooting VPC Service Controls perimeter

VPC Service Controls is a powerful tool to help mitigate the risk of cloud data breaches stemming from stolen credentials, compromised clients, malicious insiders, and misconfigured IAM policies. It allows admins to define policies and enforce security perimeters that segment and isolate resources of multi-tenant services such as Cloud Storage, BigQuery, and Stackdriver Logging. VPC Service Controls secures communication across three network interfaces of such resources: internet, VPC networks, and service backend paths. Managing a powerful and centrally configured policy requires admins to understand the impact of the policy on specific service interactions. Today, we are making it easier to understand and debug denials caused by VPC Service Controls with the VPC Service Controls Unique Identifier. This feature allows Google Cloud users to easily communicate errors that arise from VPC Service Controls denials to security admins, and lets admins quickly correlate the denied requests to corresponding Cloud Audit Log entries. This helps admins resolve access issues quickly while controls to mitigate exfiltration risks remain in place.  Configuring and troubleshooting VPC Service ControlsWhen you use VPC Service Controls, you define service perimeters that protect the Google Cloud services used in specific projects under your organization. Service perimeter configurations include: 1. Protected services (i.e. BigQuery, Cloud Storage, etc.) 2. Protected projects including the network projects identifying authorized networks3. Access Levels that define the IP ranges and identities of clients outside the perimeter that can access resources within the perimeter.When VPC Service Controls denies an incoming data access request, a 403 error message is shown and a Cloud Audit Log entry is generated. Now, with Unique Identifier, we are making it easier to connect the 403 error message to the relevant Cloud Audit Log entry to help customers troubleshoot VPC Service Controls faster.Here’s how it works:1. When users are denied access by VPC Service Controls, the 403 error messages now include a unique identifier (UID) that does not expose the underlying policy details to the potentially unauthorized or compromised client.2. Users communicate with security admins about their issue and include the UID.3. Security admins use Stackdriver Logging and search for the UID.4. Because the UID is used, only relevant log entries are displayed, which now contain links to the relevant VPC Service Controls perimeter and Access Levels pages.5. Security admins fix the issue by updating the VPC Service Controls perimeter or access level configurations.VPC Service Controls Unique Identifier helps you efficiently communicate, debug, and resolve issues associated with VPC Service Controls denials with minimal effort—helping ensure your users have access to the data they need while mitigating the risks of a data breach.To learn more about VPC Service Controls, check out our documentation.
Quelle: Google Cloud Platform

Keep a better eye on your Google Cloud environment

Monitoring, managing and understanding your cloud environment can be a challenging task for large-scale organizations. We built Google Cloud Asset Inventory so IT, security, and ops admins can get easy visibility into their Google Cloud Platform (GCP) environment. Cloud Asset Inventory is a fully managed metadata inventory service that offers various services to access GCP assets and see asset history. Two new features can make it even easier for you to do continuous asset monitoring and deep asset analysis across your GCP assets. Real-time notification feature for continuous monitoringCloud Asset Inventory now brings the real-time notification feature to beta, letting you do real-time config monitoring. For example, you can get notifications as soon as a firewall rule is changed for your web front end, or if an IAM policy binding in your production project has changed. The notifications are sent through Cloud Pub/Sub, from where you can then trigger actions. The example diagram below shows you how to monitor an IAM policy and trigger actions using Cloud Asset Inventory. In this scenario, a Gmail account was added to an IAM policy, which is generally against organizational security policy. If real-time notifications are set up on that IAM policy, Cloud Asset Inventory will send a Cloud Pub/Sub message containing the new change as soon as the change occurs. You can then write Cloud Functions to trigger an email notification, as well as directly revert the change back. You can see the IAM policy’s previous state by getting the change history of the IAM policy through the existing Cloud Asset Inventory export history feature.Native BigQuery export feature for in-depth asset analysisGiven high demand from customers, and the popularity of the related open source tool, we’ve launched native BigQuery export support in Cloud Asset Inventory. You can directly export your asset snapshots and write to a BigQuery table using the same API or CLI. This enables lots of in-depth asset analysis, asset validation, and rule-based scannings. One of our customers from Paypal has been a longtime Cloud Asset Inventory customer, and recently got a chance to adopt the BigQuery export feature. Here’s how they’ve been using it:“With the adoption of GCP and all of the associated services, Paypal was drowning in unorganized data. With multiple organizations and thousands of projects, we needed a method to gain insight and control of our cloud usage,” says Micah Norman, cloud engineer at Paypal. He initially created a Python application that queried all of the relevant APIs individually and stored the results in CloudSQL and BigQuery. This application worked well, but since Paypal has such a large number of assets, the entire job took about three hours per run. “The release of the Asset Export API allowed me to cut out nearly half of the code,” says Norman. “No longer did I have to query multiple APIs for each project. Now, with a simple bash script of around 60 lines, I was able to collect all of the relevant data in seconds. The remaining code primarily dealt with reading the resulting data and storing it correctly in CloudSQL and BigQuery.”With the most recent release of the Asset Export API, Norman was able to write directly to BigQuery from the Asset Export API, thus eliminating 40% of the remaining code. The only code remaining was rewritten in Go, and supported the collection of data external to GCP, such as G Suite data. Analysis is supported using SQL to denormalize the collected information to support reporting, auditing, and compliance efforts.Here’s a look at how the table looks in BigQuery with Cloud Asset Inventory data:For example, you can easily query the following common questions in BigQuery:1. Find the quantity of each asset type:2. Find Cloud IAM policies containing Gmail accounts as a member:With the broad resource and policy coverage from Cloud Asset Inventory, plus the powerful query capability of BigQuery, in-depth inventory analysis has gotten so much easier. Read more about how to analyze your asset data in BigQuery.Try these new real-time notifications and BigQuery export features for better inventory management, monitoring, and deep analysis.
Quelle: Google Cloud Platform