Cloud Computing Köln - Seite 2207 von 7044 - Neues zu Cloud Computing, Internet of Things und Technologien

Service accounts on Google Cloud are used when a workload needs to access resources or conduct actions without end-user involvement. There are multiple methods of authenticating using service accounts, including using service accounts as part of Google Compute Engine instances, impersonating service accounts, or using service accounts with a key file — an option which should be carefully considered. A common objective is to achieve keyless service account architectures on Google Cloud, but this can be difficult across an entire organization. There are a number of reasons why teams may opt to generate service account keys, ranging from developer validation to third-party software integration requirements. In this post, we will look at ways you can reduce risk when the use of service account keys can’t be avoided. We’ll focus on providing insights into understanding the usage of a service account within Google Cloud, which can enable us to reduce risk of unintended application failures when rotating service account keys. Let’s get started!Guidance from the CIS BenchmarksThe Center for Information Security (CIS) Benchmarks provide a set of recommended security hardening guidelines. With respect to Google Cloud Platform, CIS has most recently published the CIS Google Cloud Platform Foundation Benchmark version 1.2.0 which offer a series of controls, descriptions, rationales, impacts, audit steps, and remediations to improve your overall security posture across foundational GCP services.Let’s take a look at a direct example from Section 1.7 of the CIS Google Cloud Platform Foundation Benchmark version 1.2.0. The benchmark reads:“Ensure user-managed/external keys for service accounts are rotated every 90 days or less (Automated)”This section is accompanied by a rationale, impact, CLI commands and more, providing valuable insights into the what, the why and how you can meet this benchmark. CIS indicates that the rotation of service account keys “reduces the window of opportunity” for access that is associated with a potentially compromised account. The remediation is to rotate your service account keys: Audit and identify keys older than 90 daysDelete keys in scopeCreate a new key (if needed) In referencing the above 3 steps, the process is seemingly straightforward. However, there are nuanced use cases that you should be aware of. When deleting a Service Account Key, you effectively eliminate all current and future access for the associated private key. This can result in unintended consequences, such as loss of access for applications, pipelines, or third-party integrations that were dependent on the underlying keyfile. So how can we guard against this?Investigating the access rights and usage of a Service AccountOne method is to conduct an investigation of access and usage of the GCP Service Account and Service Account Key. Let’s bring in 3 GCP services: Policy Analyzer, Policy Intelligence, and Cloud Logging. This tooling can help us identify the impact of deleting our intended service account key. For our investigation, we should consider the following questions: 1. What can this service account do? (Policy Analyzer)IAM Roles in GCP define what a service account can do. Roles are inherited hierarchically from the organization node to the folders to the projects. IAM roles can be defined on many resources within a GCP project as well, including GCS buckets, KMS key rings, Service Accounts and more. Writing a script to determine all IAM bindings that a service account has can be quite tedious. The pseudocode could look like:If your organization happens to have more than just a few resources, this can become far too tiresome to meaningfully process. So what can we do? That’s where Policy Analyzer comes in to help save the day! Policy Analyzer enables access visibility for audit-related tasks and allows for queries across the entire organization. The following screenshot shows the key components of a Policy Analyzer Query, the Query scope, the Principal (or Service Account), and a set of advanced options. The queried result will be the set of all roles for all resources that the service account has been granted across the entire organization. For ease and reuse, here is a templated link: https://console.cloud.google.com/iam-admin/analyzer/query;identity=serviceAccount:[YOUR_SA]@[YOUR_PROJECT_ID].iam.gserviceaccount.com;expand=groups;scopeResource=[YOUR_ORG_ID];scopeResourceType=2/report?organizationId=[YOUR_ORG_ID]&supportedpurview=project This query can assist significantly in determining the range of access that this service account may have. The service account could have access in a single GCP project, access at the organization level, or access across arbitrary resources. Using Policy Analyzer enables us to fully understand where our service account may be used. 2. When was this Service Account last used? (with Policy Intelligence)Understanding when our service account was last used can also be valuable. For example, if our service account has not been used in the past year, it is likely a safe assumption that it can be deleted. The Policy Intelligence service allows us to query the activity of a service account. The command supports activity for Service Account Last Authentication or Service Account Key Last Authentication for a given GCP project. Here’s an example:Knowing that the service account authenticated during a given observation period is an indicator of whether this service account has been used. If a service account has recently been used (I’ll leave the definition of recent up to you and your organization), you may want to exercise more caution before deleting the service account key. Additionally, using Policy Intelligence to identify the last authentication for a key can provide even more granularity if you happen to have more than one service account key (this is not recommended practice). On the other hand, if we do not see recent activity for the queried service account, we should have more confidence that deleting the service account is unlikely to have unintended consequences. Policy Intelligence is project-scoped, so if our service account has roles across multiple projects, we will need to run this query within each project. Also, Policy Intelligence returns a result from a given observation period. The observation period may not include the most recent activity (such as an activity from a few minutes ago), so we can also refer to Cloud Logging… 3. What has this service account done recently? (Cloud Logging)In order to determine everything our service account has done recently, we will need to leverage Cloud Logging. We will query for a list of activities over a specific timeframe. We can use the following query:For a given service account key, we can go one step further and run the following logging query. When conducting this Cloud Logging-based investigation, there should be a few watchpoints. First, not all log types in GCP (for example, data access logs or VPC flow logs) are enabled by default. If we want to see that level of log type granularity, we would need to ensure logs are enabled on the corresponding resources. Second, querying across multiple projects may be tedious, so we may wish to make a risk-based decision around the key rotation or alternatively, export our logs to a centralized location through the use of a logging sink and analyze them holistically in BigQuery or a similar tool. Concluding thoughtsYou should now have a good sense of some of the risks around service account key use and strategies to mitigate them. First, try to avoid service account key creation whenever possible. Using GCP-managed options such as workload identity, the virtual machine service account or service account impersonation can limit the number of use cases requiring generation of service account keys.Second, if you do have user-managed service account keys, ensure that you rotate the keys. 90 days is a reasonable baseline, but ultimately, it would be better to automate the rotation of keys more frequently than 90 days. Finally, if you have a project with service account keys that you do not know the intended usage of (e.g., as might happen in a single project shared across many users), leverage the 3 GCP tooling recommendations of log queries, Policy Analyzer, and Policy Intelligence to reduce the chance of unintended application failures occurring upon key deletion. While these recommendations do not provide a full-proof mechanism for identifying usage of your service accounts, they should improve your confidence in your ability to safely rotate your service account keys and reduce risk in the process.
Quelle: Google Cloud Platform

14. Dezember 2021

da Agency

Betting on Cloud SQL pays off for oddschecker

oddschecker is the UK’s largest sports odds comparison website. We collect bookmaker pricing of various sports teams, games, and best prices, in multiple territories, and present them to our customers in a collated view. This unique view benefits our customers because of the pricing difference between one bookmaker and another. In the UK, we work with 25 different bookmakers and recently launched a U.S. site, as well as Spanish and Italian platforms — each with their own set of bookmakers. In the spring of 2018, we successfully migrated to Google Cloud. We were interested in Google from the start because we knew they would be far better at managing our database needs than we ever could be. We decided to take advantage of their managed services; Cloud SQL for MySQL, Memorystore for Redis, and Google Kubernetes Engine (GKE). As a result of the move, we’ve been able to free up developer time that used to be spent on the day-to-day hassles of database management. Because of our move to Cloud SQL, we have set the organization up to be able to make further architecture and roadmap innovations in the future. Google Cloud’s managed services were a sure betWe chose Google Cloud because it supports the most popular engines, MySQL, PostgreSQL and SQL Server, which means we can work the way we want to. Specifically, we opted for Cloud SQL primarily because of its ease of use. We were originally using on-premises databases, so we had to have large, custom virtual machines (VMs), disks, and cards to get the power we needed. Prior to the migration to Google Cloud, we were running in a private data center on custom hardware using MySQL 5.2, with about half a terabyte of data. When testing Cloud SQL, we replicated our system and found Cloud SQL performed the way we were hoping it would. Though we had initially considered a hybrid migration -where we would run parts through the cloud and chip away at making the full move over time — we ultimately decided to go all in with Google Cloud. We spun up an entire, all-around auto oddschecker infer in parallel, then did an overnight migration. After backing up and restoring to Cloud SQL, everything was ready to go.Cloud SQL covers the performance spread and then someWe were counting on Cloud SQL to meet our performance demands while radically transforming customer experiences, and it delivered everything we needed. The oddschecker site features an aggregation of platforms with a convenient, collated view which shows the odds and the status of each game. We pull 8,000 updates a second from our different operators because our customers need to get prices in a performant manner, otherwise they’re out of date, irrelevant, and, ultimately, bad for business. Things are now running on a large, single Cloud SQL database that’s a 64 CPU machine with a terabyte of storage that auto adjusts its size as needed (we use about 800 gigabytes of that consistently). We’re able to handle a couple thousand reads per second on average and about half that number of writes, running on MySQL 7.0. Because it’s our one source of truth for all onsite data, the database is critical. This content includes tips and articles, as well as our user database for new onsite customer registrations. We also have our hierarchy. It’s like a tree of information that structures each sport and team, all the way down to the markets and the matches customers bet on. In addition, we keep sports data, odds, and commentary — in short, basically every data point on the site comes from that Cloud SQL database.GKE provides the juice for delivering prices to customersCurrently, more than 90% of our workload, including our website, runs on GKE, with a few VMs running some of our legacy kit as well. We have multiple abstractions in our GKE clusters and we pull information from the various platforms, each through their own APIs. When data comes in via the ingress, we have our API reroute down to the services underneath. From there, it eventually falls into our proxies and on to Cloud SQL. On average, we try to deliver a price to customers within seven seconds of its publication by an operator. That’s complicated by some language processing that needs to happen since bookmakers often call the same teams by different names. We have to do aggregation as well as the odds comparison through a complex, homegrown mapping system that normalizes the data. Once again, Google Cloud delivered by making it possible for us to come through for our customers in a big way.MemoryStore for Redis delivers the cache, consistent key values stores, and moreAs for Memorystore for Redis, we’re using versions 3.2 and 4.0, with different teams using it for different purposes. We have 16 Memorystore instances running, with our API being one of the largest. Other instances include the caching of site content, price updates, and unmapped bets. We also use it as a key value store and to keep some of our services stateless. This way, if we want to autoscale, we have a consistent key value store that doesn’t live inside the app, so we can quickly share odds data across services. For some data, we don’t want to be hitting the databases as much, so we have them sitting in Redis instances for ease of lookup. If you look at the odds data grid for each game on our site, it shows the map mark — all the prices for a specific bet. One line across is one key value, stored in Redis, which we can look up and share across services.Removing headaches and breaking monolithsAs we suspected, Google Cloud is better at managing, patching, and upgrading our database than we’d ever be. Since our migration, we haven’t had to touch the database, except for some minor sizing. As for future goals, we’d like to break up our monolithic database because it’s become less nimble over the years — in part to alleviate blast radius concerns. Over the past three years, we’ve been building functional platforms, like our recently completed, reengineered, backend aggregation platform. It performs the same use cases so we can start chipping away at the monolith.The instance contains 30-40 critical and non-critical databases, and we don’t want to have read replicas or failover on all of them. Another goal is to move away from storing content in the database, which accounts for about half a terabyte. By adopting more of a microservices architecture, our services could be more flexible if each had their own part of the database.For example, we could have a completely different Cloud SQL profile for each. Some databases are write-heavy, some are read-heavy, and some are both. We could have custom, individually scalable machine types that cater to those use cases and provide a general improvement in functionality across the site. We could also begin breaking apart the database so we have the freedom to change when we know that it’s not going to impact the other 90 databases.In conclusion, Cloud SQL and Google Cloud’s suite of fully integrated managed services helped oddschecker make a painless move to the cloud, and meet the demands of odds comparisons, where every second matters. Learn more about oddschecker and Cloud SQL. You can also check out How BBVA is using Cloud SQL for its next-generation IT initiatives.Related ArticleHow BBVA is using Cloud SQL for its next generation IT initiativesBBVA prioritizes managed services for speed, ease of maintenance, and centralized control features. Learn how Cloud SQL fits perfectly wi…Read Article
Quelle: Google Cloud Platform