Better together: Google Cloud Load Balancing, Cloud CDN, and Google Cloud Armor

Like many Google Cloud customers, you probably use Global Load Balancing platform to get benefits such as high availability, low latency, and the convenience of a single anycast IP to front-end your global load balancing capacity. But did you know that by adding Cloud CDN and Google Cloud Armor to your existing Global HTTP(S) load balancer deployment, you can get improved web protection and faster web performance. Read on to learn more.Accelerate web performance by enabling Cloud CDNAt Google we are committed to making the web faster. For example, Cloud Load Balancing supports modern protocols such as Google QUIC and HTTP/2, which improve performance and reduce latency, especially for users on mobile networks. Then there’s Cloud CDN, which runs on our globally distributed edge points to reduce network latency by caching content closer to your users. Whenever a request is served from the Cloud CDN cache, the load balancer doesn’t need to retrieve content from the backend infrastructure. This allows you to scale seamlessly and easily handle large spikes in demand (e.g., from holiday shopping). As static web elements such as images, videos, etc., can be served from Google’s global edge instead of your backend systems, your users can enjoy faster page loads and a smoother web experience. Finally, Cloud CDN helps you optimize and reduce the cost of delivery: it keeps load off your web servers, keeping down compute usage, and content served out of Google’s edge cache is billed at a lower egress cost. Improve web protection by enabling Cloud ArmorGoogle Cloud Armor is the web-application firewall (WAF) and DDoS mitigation service that defends your web apps and services at Google scale. Cloud Armor automatically protects HTTP(S) Load Balancer workloads from volumetric and protocol based DDoS attacks. Users can configure Cloud Armor security policies for custom layer 7 filtering to further protect against application layer attacks.Cloud Armor helps protect your applications from the threats from the internet while satisfying your organization’s security and compliance requirements and providing near-real time visibility and telemetry about the traffic targeting your applications. With Cloud Armor’s pre-configured WAF rules, you can easily help mitigate the OWASP Top 10 web application security risks and prevent exploit attempts such as SQL injection (SQLi), Cross-Site Scripting (XSS), or Remote Code Execution (RCE). Cloud Armor allows users to customize the behavior of the edge of Google’s network to suit your business needs. Custom rules can be created using our comprehensive rules language to narrowly tailor what traffic is able to reach your web apps or services by filtering on request headers, parameters, and cookies. For example, you can create geography based access controls, leveraging Google’s own geo-ip database, to make your application available only in desired geographies. We recently launched Cloud Armor Managed Protection Plus (Beta), which is a managed application protection service bundling Cloud Armor WAF, DDoS Mitigation, and Google-curate rules, and other associated services. Managed Protection Plus is offered as a monthly subscription with enterprise-friendly predictable pricing to further help mitigate the impact of DDoS attacks. Getting started with enabling Google Cloud Armor and Cloud CDN With Google Cloud Load Balancing, Google Cloud Armor and Cloud CDN deployed at the edge, your users can get fast, reliable and secure web delivery with global scale and reach.Once you have set up the HTTP(S) load balancing, Cloud CDN can be enabled by clicking a single checkbox. For details on how to enable Cloud CDN, look at the Cloud CDN how-to guides. You can learn more about the benefits of Cloud CDN in this infographic.For details on how to enable Cloud Armor for your external HTTP(S) load balancer, look at the Google Cloud Armor how-to guides.Related ArticleGoogle Cloud networking in depth: Cloud Load Balancing deconstructedTake a deeper look at the Google Cloud networking load balancing portfolio.Read Article
Quelle: Google Cloud Platform

Cloud migration: What you need to know (and where to find it)

Migrating to the cloud for an enterprise that has been running workloads on premises for years can be very daunting. To be successful, a migration plan needs to factor in many different aspects relating to people, process and technology. If you are designing the migration, you need guidance and best practices to help steer you through this process.Building on our experience as solutions architects, we have put together a comprehensive set of documents for IT practitioners who are planning, designing, and implementing a migration to Google Cloud. At our Migration to Google Cloud page, you’ll find extensive technical information and advice you need to help plan and execute a successful migration. To help you get started faster, this blog post provides a high-level outline and links into the relevant part of the documentation where you can get more information.Getting started with the migrationBefore you start your migration, you should gather some foundational understanding about Google Cloud, your environment, and different migration approaches:1. Understand the difference between Google Cloud and the current environment. The source environment could be on-premises or a private hosting environment. These environments have a different operational model compared to a public cloud, from a physical security, networking, power, hardware and virtualization standpoint.2. Identify the type of workloads that need to be migrated. We recommend you start your migration by classifying workloads as either legacy, or cloud-native. Legacy workloads were developed without any consideration for cloud environments, with limited support for scaling resources such as disks and compute. As a result, these workloads can be difficult to modify and expensive to run and maintain. When designed following best practices, cloud-native workloads are natively scalable, portable, available, and secure. As a result, cloud-native workloads tend to increase developer productivity and agility, because developers can focus on the actual workloads, rather than spending effort to manage development and runtime environments.3. Determine your organization’s maturity level for cloud technologies. When identified early, skill gaps can be addressed as part of the migration process through actions like self-study, training or peer mentorship. You can use Google Cloud’s Cloud Adoption Framework to measure your organization’s cloud adoption maturity.4. Familiarize yourself with the different types of migration approaches and their tradeoffs, because different workloads might require different migration approaches. We define three types of migrations:Lift and shift. You migrate the workload, applying the least amount of changes.Improve and move. You modify parts of the workload to adopt cloud-native approaches as part of the migration.Rip and replace. You decommission the workload, and write a new workload, adopting a cloud-native approach.For more information on migration types refer to the migration guide’s section on Types of migration.The four phases of migrationBroadly speaking, the migration journey can be captured as a four-phase process: Assess, Plan, Deploy and Optimize. It’s easier to show this linearly, but it’s rarely so straightforward, with these phases often happening in parallel for different workloads. Phase 1: Assess the workloads to be migratedThis phase builds on any pre-work that you’ve done, with a focus on taking an inventory of the workloads that you plan to migrate and their respective dependencies. Things to think about include (but are not limited to) hardware and performance requirements, users, licensing, compliance needs and workload dependencies. Then, map this information into an app catalog that summarizes the information across some key axis questions—for example:Whether the workload has dependencies, or is a dependency for other workloadsHow critical the workload is to the business How difficult it is to migrate the workloadThe app catalog will provide you with a high-level view of the amount of effort to migrate all your different workloads. You can also use automated tools such as StratoZone that can scan your existing workloads and provide you with information based on the data gathered. StratoZone not only helps with discovery but can also help you map your instances to matching Google Compute Engine instances. Check out this blog post for an introduction to StratoZone.  Additional information on how to conduct discovery is also available in the Categorizing your apps section. To further get a sense of the size of risk or effort, you should conduct a proof of concept (POC) that tests the different use cases and requirements of the workload, with a focus on the more complicated workloads. This aids with getting more information early as well as reducing unknowns.You should also perform a total cost of ownership (TCO) calculation at this phase, giving the business visibility into what their cloud expenditure will look like as a result of the migration, compared to your existing environment. When moving from an on-prem to a cloud environment, there are often hidden costs that are missed when calculating the costs in the old data center. We list out some of the things to look out for when building this TCO in the Calculating total cost of ownership section of our guide. Getting the business to understand the shift in cost models and all of the additional benefits gained will be crucial to migration success. Lastly, you need to decide on which workloads to migrate first. The answer will vary from business to business depending on many different factors such as business value of the workload, complexity of migration, and the availability and requirements of the workload. To help guide this decision, it’s a good idea to call a meeting of the subject matter experts of the different workloads and go through a jointly agreed list of factors. Succeeding with the first workload is key to the overall success of your migration journey, as early success yields trust and goodwill, whereas early challenges can sometimes derail entire migration projects. Phase 2: Plan the foundationThe next phase is to plan the foundational pieces of the new cloud environment, which consist of but are not limited to:1. Establishing user and service identities. How will users and service accounts be created and managed? You can choose between G Suite or Cloud Identity domains, and optionally integrating with your existing Identity Provider (IdP). Read up on this in the Identity and Access management section.2. Designing a resource organization hierarchy. How are the different Google Cloud resources structured hierarchically? Organization nodes, folders and projects provide the building blocks to set up a resource organization hierarchy. A properly designed resource organization simplifies access control and billing management. Examples of different types of designs are:Environment oriented hierarchy – This design separates out your production, quality assurance and development environments. Function orientated hierarchy – This design breaks different business functions into their own folders at the top level, and implements an environment-orientated hierarchy beneath it.Granular orientated hierarchy – This design builds on top of the function-orientated hierarchy by adding a business unit organization at the top level.You can dive deep on this topic in the resource hierarchy section.3. Defining groups and roles for resource access. What are the different roles of users who will be accessing your cloud environment? What permissions should these different roles have? You need to create manager roles such as organizational admin, network admin and security admins to manage the cloud resources. It is also a best practice to create specific roles for the different classes of users who will be using the cloud environment, for example developers, testers and site reliability engineers (SREs). All these roles will have a minimum set of permissions associated with them to carry out their tasks. The Best practices for enterprise organizations document provides more details on this topic.4. Designing your network topology and connectivity. Into which regions will you deploy your application? Will there be connectivity back into the source environment? How many separate networks will you need to set up? The answers to these questions will feed into how you design your Virtual Private Cloud (VPC), which is your private network within Google Cloud. One VPC maps to one standalone network within your cloud environment. A VPC has subnets, firewall rules and routes that allow you to mimic the characteristics of a physical network. It’s important to also ensure you are applying security best practices; you can read about those in the Security section, as well as in the Secure your apps and data section of our Best practice for enterprise organizations guide. Connectivity back to the source environment is also possible using options such as direct interconnect, peering or a VPN. For more information read the Connectivity and networking section.Phase 3: Deploy the workloadsOnce the foundation for your migration is in place, the next step is to determine the best approach to deploy your workloads to your cloud environment. You don’t need to take the same approach for all your workloads, however, the more standardized the process is, the more opportunity for cross-team learning and improvement of the deployment process. Example of different deployment approaches are:1. Fully manual deployments. This approach is the simplest and quickest way to get your workload up and running, and can be performed from the Cloud Console or Cloud SDK directly. Although a manual deployment might be all right for some experimentation, we do not recommend this approach for production workload deployments because it is error prone, not repeatable and tends to be poorly documented. If you are currently using manual deployments, the Migration from manual deployments to automated, containerized deployments section will be able to help you improve your process. For production environments, , a more practical option is to use a service that can automatically replicate the existing workloads in your current environment and deploy it to GCP. Google Cloud offers several such services:Migrate for Compute Engine – This allows you to migrate VM-based applications from your existing environment (e.g. VMware, Azure, AWS) to GCP with minimal downtime and risk. Migrate for Anthos – Instead of migrating VMs as-is, you can intelligently convert and workloads running in VMs and migrate those workloads into containers in GKE. This often results in a reduction of cost and management. Database Migration Solutions – Whether through third parties such as Striim, or using native replication support in Google Cloud SQL, there are lots of different techniques to getting your data into Google Cloud.VMware Engine – Migrate any existing VMware-based workloads from your on-prem infrastructure without any changes directly to Google Cloud VMware Engine. This allows you to reuse any existing VMware deployment tooling and get started immediately with your migration, and easily add new workloads with the VMware framework within Google Cloud.2. Deploy using configuration management tools. Using configuration management (CM) tools such as Ansible, Chef or Puppet provides a repeatable, automated and controlled way to run your deployment. However, these tools are best suited for provisioning and configuration, and less suitable for workload deployments. This is because the tools require bespoke deployment logic to handle procedures such as zero-downtime deploys, blue-green deployments or rolling updates, and end up becoming more difficult to manage and maintain over the long run. 3. Deploy by using container orchestration tools. If your workloads are containerized you can use Google Kubernetes Engine (GKE) to handle the deployment process. The Kubernetes orchestrator supports many types of deployment logic such as zero-downtime deploys and rolling updates out of the box. Alternatively if your workloads are still on VMs running GCE, Azure or AWS Migrate for Anthos allows you to convert your VMs into containers automatically. This allows you to gain the benefit of running on containers quicker. 4. Deploy automatically. An automated deployment process is triggered based on some action that results in a change in the workload and can be built on top of any orchestration tool that can be scripted. Automated deployments allow you to streamline and standardize your deployment process reducing human error.You can use tools such as Jenkins, SonarQube, Cloud Build or Spinnaker to build an end-to-end automated deployment pipeline on top of your existing orchestration tools. The key steps of an automated deployment process are:Code review. Every change to your codebase should be reviewed by a peer to ensure the quality of the change before merging it into the codebase.Continuous integration (CI). Once merged, the CI tool runs all existing tests against the new version for the codebase and ensures that no tests fail. Only then does it mark the build as successful.Artifact production. For every successful build an artifact is produced. A container is an example of an artifact. Tests can also be run by using tools such as Serverspec to ensure that the artifacts are working well.Continuous deployment (CD).A successful artifact is then deployed into your development or quality assurance cloud environment, after which another set of functional tests could be run against the deployment to ensure that its running well. Once those tests pass, the deployment can then be deployed to your production environment, either automatically, or after being manually triggered by an operator.5. Deploy by applying the infrastructure-as-code pattern. The idea behind infrastructure as code is to treat configuring and provisioning your cloud resources in the same way you treat the source code for your workloads. Similar to how new versions of workloads are deployed by going through a series of automated steps and tests, any changes to the infrastructure configuration also go through a series of steps that involve testing before being deployed to the target cloud environment. This is our recommended best practice as it provides repeatability and traceability, which improve overall deployment velocity. This process can be implemented using tools such as Terraform and managed services such as a Deployment Manager.Phase 4: Optimize your environmentOnce a basic deployment of your workloads is running and tested in your new Google Cloud environment, you can start to improve on this foundation. This includes critical pieces that should be completed before cutting over live traffic, for example training your team on new cloud operational playbooks as well as ensuring that logging, monitoring and alerting for these workloads are in place.Other aspects that you can optimize once the workload is serving production traffic include: Cost optimization with autoscalingMoving to managed workloads to reduce operational overheadAutomating the deployment process Read up on how best to approach this in the Optimizing your environment section. Read on to ensure a successful cloud migrationA large migration can be daunting for the most ambitious of teams. But with the right methodology, planning, and testing before deployment, you can break the problem down into smaller, more manageable steps. Our Migration to Google Cloud solution guide covers the above in more detail, and also provides additional resources, like our ‘Finding Help’ section, that you can use to help start migrating your workloads to the cloud.  If you require more assistance from professionals who have a track record of successful migrations, the Google Cloud Professional Services Organization offers consulting services directly or via a host of partners with a wide range of specialties. Just reach out and we can help you get on your way!
Quelle: Google Cloud Platform

Microsoft’s newest sustainable datacenter region coming to Arizona in 2021

 

On our journey to become carbon negative by 2030, Microsoft is continually innovating and advancing the efficiency and sustainability of our cloud infrastructure, with a commitment to use 100 percent renewable energy in all of our datacenters and facilities by 2025. Today, we are taking a significant step toward that goal, revealing plans for our newest sustainable datacenter region in Arizona, which will become our West US 3 region.

 

Companies are not only digitally transforming their operations and products to become more sustainable—they’re also choosing partners with shared goals and values. In developing the new West US 3 region, we have water conservation and replenishment firmly in mind. Today, Microsoft announced an ambitious commitment to be water positive for our direct operations by 2030. We’re tackling our water consumption two ways: reducing our consumption and replenishing water in the regions we operate. Since we announced our plans to invest in solar energy in Arizona to build more sustainable datacenters last year, we have been working with the communities of El Mirage and Goodyear on water conservation, education and sustainability projects to support local priorities and needs.

 

 

Sustainable design delivering the full Microsoft cloud for global scale, security and reliability

Our datacenter design and operations will contribute to the sustainability of our Arizona facilities. In Arizona, we’re pursuing Leadership in Energy and Environmental Design (LEED) Gold certification, which will help conserve additional resources including energy and water, generate less waste and support human health. We’re also committed to zero waste-certified operations for this new region, which means a minimum of 90 percent of waste will be diverted away from landfills through reduction, reuse and recycling efforts.

The new datacenter region will deliver enterprise-grade cloud services, all built on a foundation of trust:

Microsoft Azure, an ever-expanding set of cloud services that offers computing, networking, databases, analytics, AI and IoT services.
Microsoft 365, the world’s productivity cloud that delivers best-of-breed productivity apps integrated through cloud services and delivered as part of an open platform for business processes.
Dynamics 365 and Power Platform, the next generation of intelligent business applications that enable organizations to grow, evolve and transform to meet the needs of customers and capture new opportunities.
Compliance, security and privacy, Microsoft offers more than 90 certifications and spends $1 billion every year on cybersecurity to address security at every layer of the cloud.

To support customer needs for high-availability and resiliency in their applications, the new region will also include Availability Zones, which are unique physical locations of datacenters with independent power, network, and cooling for additional tolerance to datacenter failures.

Our construction partner Nox Innovations is helping build these sustainable datacenters with the help of Microsoft HoloLens 2, Microsoft Dynamics 365 Remote Assist and Microsoft mixed reality partner solution VisualLive, to visualize building information modeling (BIM) data in the form of holograms and overlay the 3D assets in the context of the physical environment. VisualLive’s solution is powered by Azure Spatial Anchors, a new Azure mixed reality service that maps, persists and restores 3D experiences in the real-world, VisualLive’s solution. The hands-free and remote work environment enabled by HoloLens 2 and cloud services enables virtual collaboration that has led to greater efficiency, safety and accuracy.

Delivering renewable solar energy and replenishing water in Arizona

Our commitment in Arizona includes a sustainable datacenter design and operations as well as several local initiatives to support water conservation. First, Microsoft is collaborating with First Solar, an Arizona-headquartered global leader in solar energy, on their Sun Streams 2 photovoltaic (PV) solar power plant, which will offset the day one energy usage of the new campus, available in 2021, with solar energy once the facility is operational. Clean solar PV energy displaces the water needed in the traditional electricity generation process. First Solar’s lowest carbon solar PV technology does not require water to generate electricity and is ideally suited to meet the growing energy and water needs of arid, water-limited regions. By displacing conventional grid electricity in Arizona, First Solar’s Sun Streams 2 Project is expected to save 356 million liters of water annually.

Microsoft’s Arizona datacenters will use zero water for cooling for more than half the year, leveraging a method called adiabatic cooling, which uses outside air instead of water for cooling when temperatures are below 85 degrees Fahrenheit. When temperatures are above 85 degrees, an evaporative cooling system is used, which is similar to “swamp coolers” in residential homes. This system is highly efficient, using less electricity and a fraction of water used by other water-based cooling systems, such as cooling towers.

For the last year, we have also been investing in water conservation to have a longer-lasting impact on replenishing water in Arizona to sustain water levels in Lake Mead, with the goal of supporting the state to meet its Drought Contingency Plan Commitments. Microsoft’s investment in this project has also generated a one-to-one cash match from the Water Funder Initiative that will support the state’s efforts and further expand project impact. The project will benefit the Colorado River Indian Tribes, ultimately resulting in more water in Lake Mead and more efficient water infrastructure.

Lastly, Microsoft and Gila River Water Storage, LLC are recharging and replenishing groundwater levels in the Phoenix Active Management Area with long term storage credits dedicated to the cities of Goodyear and El Mirage to balance a portion of Microsoft’s future water use, contributing an estimated additional 610,000 cubic meters. Microsoft is also collaborating with The Nature Conservancy to support water conservation in the Verde River Basin, installing a new pipe in the leakiest part of the Eureka Ditch to increase resilience for local farmers.

Supporting local growth, opportunities in Arizona

Through our Datacenter Community Development initiative, we are actively engaged in El Mirage, Goodyear, and across Arizona to advance community priorities in education, workforce development and further community connection. These investments in local projects total more than $800,000 and employee volunteer time as well as community partnerships to clean up the Gila River, provide WiFi connectivity for 1,000 students across the Navajo Nation and support the expansion of Mathematics, Engineering, Science Achievement (MESA) to serve 1,500+ middle school and high students across Arizona. In addition, Microsoft is collaborating with two Maricopa Community Colleges, including Estrella Mountain Community College in Avondale and Glendale Community College in Glendale, to develop workforce training that prepares workers for jobs in the IT sector, including work in Microsoft datacenters.

The new datacenter region and related work is expected to create over 100 permanent jobs across a variety of functions, including mechanical engineers, electrical engineers and datacenter technicians, when the facilities are fully operational, and more than 1,000 construction jobs over the initial building phases. Once the datacenters are operating, they’re expected to have an annual economic impact of approximately $20 million across communities in Arizona.
Quelle: Azure

Best practices for using Docker Hub for CI/CD

According to the 2020 Jetbrains developer survey 44% of developers are now using some form of continuous integration and deployment with Docker Containers. We know a ton of developers have got this setup using Docker Hub as their container registry for part of their workflow so we decided to dig out the best practices for doing this and provide some guidance for how to get started. To support this we will be publishing a series of blog posts over the next few weeks to answer the common questions we see with the top CI providers.

We have also heard feedback that given the changes Docker introduced relating to network egress and the number of pulls for free users, that there are questions around the best way to use Docker Hub as part of CI/CD workflows without hitting these limits. This blog post covers best practices that improve your experience and uses a sensible consumption of Docker Hub which will mitigate the risk of hitting these limits and how to increase the limits depending on your use case. 

To get started, one of the most important things when working with Docker and really any CI/CD is to work out when you need to test with the CI or when you can do this locally. At Docker we think about how how developers work in terms of their inner loop (code, build, run, test) and their outer loop (push change, CI build, CI test, deployment) 

Before you think about optimizing your CI/CD, it is always important to think about your inner loop and how it relates to the outer loop (the CI). We know that most people aren’t a fan of ‘debugging via the CI’, so it is always better if your inner loop and outer loop are as similar as possible. To this end it can be a good idea to run unit tests as part of your docker build command by adding a target for them in your Dockerfile. That way as you are making changes and re-building locally you can run the same unit tests you would run in the CI on your local machine with a simple command. Chris wrote a blog post earlier in the year about Go development with Docker, this is a great example of how you can use tests in your Docker project and re-use them in the CI. This creates a shorter feedback loop on issues and reduces the amount of pulls and builds your CI needs to do.

Once you get into your actual outer loop and Docker Hub, there are a few things we can do to get the most of your CI and deliver the fastest Docker experience. 

Firstly and foremost stay secure! When you are setting up your CI make sure you are using a Docker Hub access token rather than your password, you can create new access tokens from your security page on Docker Hub. 

Once you have this and have added it to whatever secrets store is available on your platform you will want to look at when you decide to push and pull in your CI/CD along with where from depending on the change you are making. The first thing you can do here to reduce the build time and reduce your number of calls is make use of the Buildcache to reuse layers you have already pulled. This can be done on many platforms by using BuildX/buildkits caching functionality and whatever cache your platform provides.

The other change you may want to make is only have your release images go to DockerHub, this would mean setting up functions to push your PR images to a more local image store to be quickly pulled and tested rather than promoting them all the way to production.

We know there are a lot more tips and tricks for using Docker in CI but really looking at how to do this around the recent Hub rate changes we think these are the top things you can do.

If you are still finding you have issues with Pull limits once you are authenticated you can consider upgrading to either a Pro or a Team account. This will give you unlimited authenticated pulls from Docker Hub, along with giving you unlimited private repos and unlimited image retention. In the near future this will also include Image Scanning (powered by Snyk) on push of new images to Docker Hub.

Look out for the next blog post in the series about how to put some of these practices into place with Github actions and feel free to give us ideas of what CI providers you would like to see us covering by dropping us a message on Twitter @Docker.
The post Best practices for using Docker Hub for CI/CD appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/