How Google Cloud blocked the largest Layer 7 DDoS attack at 46 million rps

Over the past few years, Google has observed that distributed denial-of-service (DDoS) attacks are increasing in frequency and growing in size exponentially. Today’s internet-facing workloads are at constant risk of attack with impacts ranging from degraded performance and user experience for legitimate users, to increased operating and hosting costs, to full unavailability of mission critical workloads. Google Cloud customers are able to use Cloud Armor to leverage the global scale and capacity of Google’s network edge to protect their environment from some of the largest DDoS attacks ever seen.On June 1, a Google Cloud Armor customer was targeted with a series of HTTPS DDoS attacks which peaked at 46 million requests per second. This is the largest Layer 7 DDoS reported to date—at least 76% larger than the previously reported record. To give a sense of the scale of the attack, that is like receiving all the daily requests to Wikipedia (one of the top 10 trafficked websites in the world) in just 10 seconds.Cloud Armor Adaptive Protection was able to detect and analyze the traffic early in the attack lifecycle. Cloud Armor alerted the customer with a recommended protective rule which was then deployed before the attack ramped up to its full magnitude. Cloud Armor blocked the attack ensuring the customer’s service stayed online and continued serving their end-users.Figure 1: DDoS attack graph peaking at 46M requests per second.What happened: Attack analysis and timelineStarting around 9:45 a.m. PT on June 1, 2022, an attack of more than 10,000 requests per second (rps) began targeting our customer’s HTTP/S Load Balancer. Eight minutes later, the attack grew to 100,000 requests per second. Cloud Armor Adaptive Protection detected the attack and generated an alert containing the attack signature by assessing the traffic across several dozen features and attributes. The alert included a recommended rule to block on the malicious signature. The following is the alert showing details of the attack before it ramped to its peaks.Figure 2: Cloud Armor Adaptive Protection alert listing the top region codes detected as a part of the attack.Our customer’s network security team deployed the Cloud Armor-recommended rule into their security policy, and it immediately started blocking the attack traffic. In the two minutes that followed, the attack began to ramp up, growing from 100,000 rps to a peak of 46 million rps. Since Cloud Armor was already blocking the attack traffic, the target workload continued to operate normally. Over the next few minutes, the attack started to decrease in size, ultimately ending 69 minutes later at 10:54 a.m. Presumably the attacker likely determined they were not having the desired impact while incurring significant expenses to execute the attack. Analyzing the attackIn addition to its unexpectedly high volume of traffic, the attack had other noteworthy characteristics. There were 5,256 source IPs from 132 countries contributing to the attack. As you can see in Figure 2 above, the top 4 countries contributed approximately 31% of the total attack traffic. The attack leveraged encrypted requests (HTTPS) which would have taken added computing resources to generate. Although terminating the encryption was necessary to inspect the traffic and effectively mitigate the attack, the use of HTTP Pipelining required Google to complete relatively few TLS handshakes.  Approximately 22% (1,169) of the source IPs corresponded to Tor exit nodes, although the request volume coming from those nodes represented just 3% of the attack traffic. While we believe Tor participation in the attack was incidental due to the nature of the vulnerable services, even at 3% of the peak (greater than 1.3 million rps) our analysis shows that Tor exit-nodes can send a significant amount of unwelcome traffic to web applications and services.The geographic distribution and types of unsecured services leveraged to generate the attack matches the Mēris family of attacks. Known for its massive attacks that have broken DDoS records, the Mēris method abuses unsecured proxies to obfuscate the true origin of the attacks.  How we stopped the attackThe attack was stopped at the edge of Google’s network, with the malicious requests blocked upstream from the customer’s application. Before the attack started, the customer had already configured Adaptive Protection in their relevant Cloud Armor security policy to learn and establish a baseline model of the normal traffic patterns for their service. As a result, Adaptive Protection was able to detect the DDoS attack early in its life cycle, analyze its incoming traffic, and generate an alert with a recommended protective rule–all before the attack ramped up. The customer acted on the alert by deploying the recommended rule leveraging Cloud Armor’s recently launched rate limiting capability to throttle the attack traffic. They chose the ‘throttle’ action over a ‘deny’ action in order to reduce chance of impact on legitimate traffic while severely limiting the attack capability by dropping most of the attack volume at Google’s network edge. Before deploying the rule in enforcement mode, it was first deployed in preview mode, which enabled the customer to validate that only the unwelcome traffic would be denied while legitimate users could continue accessing the service. As the attack ramped up to its 46 million rps peak, the Cloud Armor-suggested rule was already in place to block the bulk of the attack and ensure the targeted applications and services remained available. Protecting your applications in the cloudAttack sizes will continue to grow and tactics will continue to evolve. To be prepared, Google recommends using a defense-in-depth strategy by deploying defenses and controls at multiple layers of your environment and your infrastructure providers’ network to protect your web applications and services from targeted web attacks. This strategy includes performing threat modeling to understand your applications’ attack surfaces, developing proactive and reactive strategies to protect them, and architecting your applications with sufficient capacity to manage unanticipated increases in traffic volume. With Google Cloud Armor, you are able to protect your internet facing applications at the edge of Google’s network and absorb unwelcome traffic far upstream from your applications.Related ArticleIntroducing new Cloud Armor features including rate limiting, adaptive protection, and bot defenseCloud Armor strengthens its already formidable defenses with new features to counter advanced L7 attacks and block malicious bots.Read Article
Quelle: Google Cloud Platform