Ensuring scale and compliance of your Terraform Deployment with Cloud Build

Terraform is an open source Infrastructure as Code tool that is popular with platform developers building reusable cloud automation. The Terraform Provider for Google Cloud Platform continues to add support for the latest Google Cloud features, such as Anthos on GKE, and our teams continue to expand Terraform integrations including Cloud Foundation Toolkit and Terraform Validator.How do teams use Terraform on Google Cloud? While the simplest approach is to run terraform init, plan and apply directly from your terminal,  it cannot be recommended for automating your production deployments. First, there is a decision on how to store your Terraform state in a way that is secure, compliant and enables team collaboration. Secondly there’s a question of scale and reliability. Over the course of even the simplest cloud deployment, Terraform can end up making thousands of Create/Read/Update/Delete API calls to the endpoints used by the Terraform providers, some of which will inevitably hit quota issues or need to be retried for other reasons. For platform administrators, who are looking to ensure the best deployment practices for their curated Terraform solutions,  while benefiting from the simplicity of Google Cloud Console, there’s Terraform Private Catalog integration that we enabled earlier this year.Outside of Private Catalog, Cloud Build and Cloud Storage have been the recommended approach to use Terraform on Google Cloud. Using a remotebackend prevents race conditions and simplifies sharing reusable modules between different configurations. With Cloud Build you can configure a GitOps CI/CD pipeline to automatically plan and apply your Terraform configuration when changes are pushed into the repo. These are widely popularized benefits explored in Managing infrastructure as code with Terraform, Cloud Build, and GitOps. In addition, there are lesser known advantages of Cloud Build, particularly for enterprise customers: Cloud Build’s concurrency capabilities and VPC-SC support, Cloud Storage versioning, security and compliance. Let’s explore these benefits in more detail.Cloud Build’s ability to scale makes it capable to process multiple Terraform deployments across the regions globally and simultaneously. By default, Cloud Build supports 30 concurrent builds, with additional builds queued and processed after the running builds complete. In some cases it may not be enough. Customers who initiate parallel deployments to multiple zones, or, those who provision infrastructure on behalf of multiple tenants, often require running more concurrent deployments to complete all of them within the allotted deployment window. Cloud Build private pool feature allows up to 100 concurrent builds which may be further adjusted upon request. This is an example of creating a private pool and then using it when submitting a build:A full step by step example of creating a private pool and submitting 80+ Terraform deployments with Cloud Build simultaneously is available here.Using Cloud Build removes the need to build a custom high-scale Terraform provisioning service and provides observability and diagnostics for each of the build instances launched and their results. Using Cloud Build with private pools enables recommended security features, such as VPC Service Controlsthat allows setting secure perimeter to protect against data exfiltration, with additional restrictions to further restrict it to using the specified private pools. This makes it unnecessary to configure a dedicated bastion host inside the perimeter, which improves the overall security posture.Beyond just using Cloud Storage for remote storage, additional reasons to use Cloud Storage include versioning, security and compliance. Enabling versioning protects against state file corruption and allows you to view earlier versions. Versioning can be enabled with gsutil command:In addition to versioning, you can use Customer-Supplied Encryption Keys to encrypt the Terraform state file. After you generated the key you can specify it as encryption_key parameter of your backend object:Once encrypted you can still view the contents of your state by adding encryption_key option to boto configuration file. Finally, Cloud Storage is one of the Google Cloud services covered by FedRAMP High, which is important for enterprises  that are seeking their own FedRAMP on top of Google Cloud (for more details see Compliance resource center).To summarize, using Cloud Build and Cloud Storage for your Terraform deployments enable high scalability, security and compliance with simpler configuration and via familiar gcloud and Google Cloud console interface. Please check out this sample for step by step guidance.Related ArticleIntroducing Cloud Build private pools: Secure CI/CD for private networksWith new private pools, you can use Google Cloud’s hosted Cloud Build CI/CD service on resources in your private network or in other clouds.Read Article
Quelle: Google Cloud Platform

Enabling keyless authentication from GitHub Actions

GitHub Actions is a third-party CI/CD solution popular among many Google Cloud customers and developers. When a GitHub Actions Workflow needs to read or mutate resources on Google Cloud – such as publishing a container to Artifact Registry or deploying a new service with Cloud Run – it must first authenticate.Traditionally, authenticating from GitHub Actions to Google Cloud required exporting and storing a long-lived JSON service account key, turning an identity management problem into a secrets management problem. Not only did this introduce additional security risks if the service account key were to leak, but it also meant developers would be unable to authenticate from GitHub Actions to Google Cloud if their organization has disabled service account key creation (a common security best practice) via organization policy constraints like constraints/iam.disableServiceAccountKeyCreation.But now, with GitHub’s introduction of OIDC tokens into GitHub Actions Workflows, you can authenticate from GitHub Actions to Google Cloud using Workload Identity Federation, removing the need to export a long-lived JSON service account key.Fine-grained scoping. Workload Identity Pools and Providers can define fine-grained attribute mappings between the OIDC token and the available permissions in Google Cloud. Whereas a JSON service account key is either accessible or inaccessible, Workload Identity Federation can be configured to selectively allow authentication based on properties in the downstream OIDC tokens. For GitHub Actions, that means you can, for example, restrict authentication to certain repositories, usernames, branch names, or published claims. You can also combine and build more complex and compound constraints using CEL.Short-lived credentials. Unlike JSON service account keys, Workload Identity Federation generates short-lived OAuth 2.0 or JWT credentials. By default, these credentials automatically expire one hour after they are created, potentially reducing the time a malicious actor would be able to exploit a compromised credential.Minimal management overhead. JSON service account keys must be securely stored, rotated, and managed. Even at a small scale, this can be toilsome and prone to errors. Because Workload Identity Federation uses short-lived credentials, there are no secrets to rotate or manage beyond the initial configuration.A new GitHub Action – auth!To ease the process of authenticating and authorizing GitHub Actions Workflows to Google Cloud via Workload Identity Federation, we are introducing a new GitHub Action – auth! The auth action joins our growing collection of Google-managed GitHub Actions and makes it simple to set up and configure authentication to Google Cloud:This will use the configured workload_identity_provider and service_account to authenticate future steps. The gcloud command-line tool, official Google Cloud client libraries, and popular third-party tools like Terraform will automatically detect and use this authentication. Additionally, all the Google GitHub Actions support this authentication mechanism. For example, you can use the auth GitHub Action with the get-gke-credentials GitHub Action:If you are using third-party tools that do not support Application Default Credentials, or if you want to invoke Google Cloud APIs manually via curl, the auth GitHub Action can create OAuth 2.0 tokens and JWTs for use in future steps. The following example creates a short-lived OAuth 2.0 access token and then uses that token to access a secret from Google Secret Manager using curl:To ease in migration and to support legacy workflows, the auth GitHub Action also supports authenticating via a Google Cloud service account key JSON file:Learn more about the auth GitHub Action and check out the examples at google-github-actions/auth.Setting up Identity Federation for GitHub ActionsTo use the new GitHub Actions auth action, you need to set up and configure Workload Identity Federation by creating a Workload Identity Pool and Workload Identity Provider:The attribute mappings map claims in the GitHub Actions JWT to assertions you can make about the request (like the repository or GitHub username of the principal invoking the GitHub Action). These can be used to further restrict the authentication using –attribute-condition flags. For example, you can map the attribute repository value (which can be used later to restrict the authentication to specific repositories):Finally, allow authentications from the Workload Identity Provider to impersonate the desired Service Account:For more configuration options, see the Workload Identity Federation documentation. If you are using Terraform to automate your infrastructure provisioning, check out the GitHub OIDC Terraform module too.Towards invisible securityAt first, authenticating to Google Cloud from a GitHub Action without a long-lived JSON service account key might seem like magic, but it’s all part of Google Cloud’s ongoing efforts to make security invisible and our platform secure-by-default. Using Workload Identity Federation to replace long-lived JSON service account keys in GitHub Actions delivers  improvements in security and auditability.To get started, check out the auth GitHub Action today!Related ArticleKeyless API authentication—Better cloud security through workload identity federation, no service account keys necessaryWith workload Identity federation, you can securely operate your workloads and no longer have to worry about managing service account keys.Read Article
Quelle: Google Cloud Platform

Amazon SQS verbessert die Verwaltung von Warteschlangen für unzustellbare Nachrichten für Standardwarteschlangen

Amazon Simple Queue Service (SQS) kündigt Support für die erneute Ausführung für Warteschlangen für unzustellbare Nachrichten in die Quellwarteschlange (DLQ) an, sodass Sie den Lebenszyklus nicht verarbeiteter Nachrichten besser kontrollieren können. Warteschlangen für unzustellbare Nachrichten sind eine vorhandene Funktion von Amazon SQS, die es Kunden ermöglicht, Nachrichten zu speichern, die Anwendungen nicht erfolgreich verarbeiten konnten. Sie können Nachrichten jetzt effizient aus Ihrer Warteschlange für unzustellbare Nachrichten in Ihre Quellwarteschlange auf der Amazon-SQS-Konsole umleiten. DQL erweitert das Management-Erfahrung der Warteschlange für unzustellbare Nachrichten für Entwickler und ermöglicht es ihnen, Anwendungen mit der Gewissheit zu erstellen, dass sie ihre nicht verarbeiteten Nachrichten untersuchen, Fehler in ihrem Code beheben und Nachrichten in ihren Warteschlangen für unzustellbare Nachrichten erneut verarbeiten können.
Quelle: aws.amazon.com

AWS Managed Microsoft AD hilft bei der Optimierung von Skalierungsentscheidungen mit Verzeichnismetriken in Amazon CloudWatch

AWS Directory Service for Microsoft Active Directory (AWS Managed Microsoft AD) hilft jetzt bei der Optimierung von Skalierungsentscheidungen für verbesserte Leistung und Ausfallsicherheit mit Amazon CloudWatch. Ab heute stellt AWS Managed Microsoft AD automatisch Metriken zur Domänencontroller- und Verzeichnisauslastung in Amazon CloudWatch für neue und vorhandene Verzeichnisse bereit. Die Analyse dieser Nutzungsmetriken hilft Ihnen, Ihre durchschnittlichen und Spitzenlastzeiten zu quantifizieren, um den Bedarf an zusätzlichen Domänencontrollern zu ermitteln. Damit können Sie die Anzahl der Domänencontroller definieren, um Ihre Leistungs-, Ausfallsicherheits- und Kostenanforderungen zu erfüllen.
Quelle: aws.amazon.com

Amazon Virtual Private Cloud (VPC) kündigt Network Access Analyzer an, damit Sie unbeabsichtigten Netzwerkzugriff leicht erkennen können

Amazon VPC Network Access Analyzer ist eine neue Funktion, mit der Sie unbeabsichtigten Netzwerkzugriff auf Ihre Ressourcen in AWS erkennen können. Mit Network Access Analyzer können Sie überprüfen, ob der Netzwerkzugriff für Ihre Virtual Private Cloud (VPC)-Ressourcen Ihren Sicherheits- und Compliance-Richtlinien entspricht. Mit Network Access Analyzer können Sie Verbesserungen Ihres Cloud-Sicherheitsstatus bewerten und identifizieren. Darüber hinaus erleichtert Ihnen Network Access Analyzer den Nachweis, dass Ihr Netzwerk bestimmte behördliche Anforderungen erfüllt.
Quelle: aws.amazon.com

Ankündigung von AWS Direct Connect SiteLink

AWS hat heute die allgemeine Veröffentlichung von AWS Direct Connect SiteLink angekündigt. SiteLink macht es einfach, private Netzwerkverbindungen zwischen Ihren On-Premises-Standorten wie Büros und Rechenzentren herzustellen, indem Sie diese mit Direct-Connect-Standorten auf der ganzen Welt verbinden.
Quelle: aws.amazon.com