Cloud Computing Köln - Seite 2523 von 7077 - Neues zu Cloud Computing, Internet of Things und Technologien

At Google Cloud, we believe you get most benefits from the cloud when you scale infrastructure based on changing demand. Compute Engine allows you to configure autoscaling to save costs during periods of low demand, and add capacity to support peak loads. When you use a managed instance group (MIG), you can have an autoscaler automatically create or delete virtual machine (VM) instances based on increases or decreases in load. However, if your application takes several minutes to initialize, creating VMs in response to growing load might not increase your application’s capacity quickly enough. For example, if there’s a large increase in load (like when users first wake up in the morning), some users might experience delays while your application is initializing on new instances.A good way to solve this problem would be to create VMs ahead of demand so that your application has enough time to initialize beforehand. This requires knowing upcoming demand. If only we could predict the future… Well, now we can!Introducing predictive autoscalingPredictive autoscaling uses Google Cloud’s machine learning capabilities to forecast capacity needs. It creates VMs ahead of growing demand allowing enough time for your application to initialize.Figure 1. Autoscaling creates VMs as demand grows leaving no buffer for application to initialize. Predictive autoscaling creates VMs ahead of demand allowing enough time for your application to initialize and start serving new load.How does it work?Predictive autoscaling uses your instance group’s CPU history to forecast future load and calculate how many VMs are needed to meet your target CPU utilization. Our machine learning adjusts the forecast based on recurring load patterns for each MIG. You can specify how far in advance you want autoscaler to create new VMs by configuring the application initialization period. For example, if your app takes 5 minutes to initialize, autoscaler will create new instances 5 minutes ahead of the anticipated load increase. This allows you to keep your CPU utilization within the target and keep your application responsive even when there’s high growth in demand. Many of our customers have different capacity needs during different times of the day or different days of the week. Our forecasting model understands weekly and daily patterns to cover for these differences. For example, if your app usually needs less capacity on the weekend our forecast will capture that. Or, if you have higher capacity needs during working hours, we also have you covered.Why should you try it?Predictive autoscaling continuously adapts forecasted capacity to best match upcoming demand. Autoscaler checks the forecast several times per minute and creates or deletes VMs to match its prediction. The forecast itself is updated every few minutes to match recent load trends so if your growth rate is higher or lower than usual we will adjust the forecast accordingly. This gives you capacity needed to cover peak load while saving on cost when demand goes down. You can start using predictive autoscaling without worry as it’s fully compatible with the current autoscaler. Autoscaler will calculate enough VMs to cover both forecasted as well as real-time CPU load—whichever is higher. This works with other autoscaling features as well: you can scale based on schedule, your Load Balancer request target or Cloud Monitoring metrics. Autoscaler provides enough capacity to all of your configurations by taking the highest number of VMs needed to meet all your targets.Getting startedYou can enable predictive autoscaling in the Google Cloud Console. Select an autoscaled MIG from the instance groups page and click Edit group. Change predictive autoscaling configuration from Off to Optimize for availability.To better understand whether predictive autoscaling is good for your application, click the link See if predictive autoscaling can optimize your availability. This will show you a comparison of the last seven days with your current autoscaling configuration vs. with predictive autoscaling enabled.In the above chart, Average VM minutes overloaded per day shows how often your VMs exceed your CPU utilization target. This happens when demand is higher than available capacity. Predictive autoscaling can reduce this by starting VMs ahead of anticipated load. Average VMs per day is a proxy for cost. This shows how much additional VM capacity you need to keep your CPU utilization within the target you have set. You can optimize your cost by adjusting Minimum instancesand CPU utilization as explained below. Optimizing your configurationMake sure your Cool down period reflects how long it takes for your application to initialize from VM boot time until it’s ready to serve the load. Predictive autoscaling will use this value to start VMs ahead of forecasted load. If you set it to 10 minutes (600 seconds) your VMs will start 10 minutes before the load is expected to increase.Review your autoscaling CPU utilization target and Minimum number of instances. With predictive autoscaling you no longer need a buffer to compensate for the time it takes for a VM to start. If your application works best at 70% CPU utilization you don’t need to set target to a much lower value as predictive autoscaling will start VMs ahead of usual load. A higher CPU utilization and lower Minimum number of instances allows you to reduce the cost as you don’t need to pay for additional capacity to prepare for growing demand.Try predictive autoscaling todayPredictive autoscaling is generally available across all Google Cloud regions. For more information on how to configure, simulate and monitor predictive autoscaling, consult the documentation.Related ArticleAt your service! With schedule-based autoscaling, VMs are at the readySchedule-based autoscaling for Compute Engine lets you improve the availability of your workloads by scheduling capacity ahead of anticip…Read Article
Quelle: Google Cloud Platform

4. Juli 2021

da Agency

Kinguin helps shoppers find products faster with Recommendations AI

Over 2.14 billion people worldwide are expected to buy online this year, according to Statista. Online retail sales will account for 22% of all purchases by 2023. But in a competitive retail landscape, positive interactions can mean the difference between a sale and an abandoned shopping cart.One of the leading global marketplaces – Kinguin.net is a haven for gamers. Their bustling ecommerce business conducts over 500,000 new transactions monthly. Users will encounter over 50,000 unique digital products, from video games, gift cards, in-game items to computer software and services. With over 10 million registered users, Kinguin improved their experience by helping users find items quickly and deliver service at scale.Helping customers find what they want, fastBecause of Kinguin’s high volume of users—both buyers and sellers—and breadth of digital products, browsing and shopping can be challenging. “Customers shop online for choice and convenience, but it can sometimes be overwhelming. We want anyone who shops at Kinguin to find what they are looking for quickly and easily,” says Viktor Romaniuk Wanli, Kinguin CEO and Founder.Today’s retailers know that creating personalized shopping experiences is crucial for establishing and maintaining customer loyalty. Kinguin discovered their users were getting a rather standard retail experience. They wondered how they could offer them a more tailored, personalized experience.They knew product recommendations were a great way to personalize experiences because they help customers discover products that match their tastes and preferences. But it’s not that easy to recommend products. Various shifting factors make recommendations much more complex:Customer behavior. Understanding customers is tough. How do you recommend something to a cold start user who’s never been to your site before? What happens when their behavior changes?Omnichannel context. According to Harvard Business Review, 73% of all customers use many channels when they buy. What happens when they go from desktop to mobile or from social media shopping to a proprietary app?Product data challenges. How do you recommend new products within a large catalog of items? What if your product data has sparse labeling or unstructured metadata?Data wasn’t a problem for Kinguin. They had data orders, history, wishlists, and could collect events based on their platform interactions. It was the machine learning model expertise they lacked. So rather than building their own solution, they determined it was more cost effective for them to find a reliable partner. It was also essential that the solution integrated easily with Kubernetes, which enabled their global network.With these considerations in mind, they applied for the Google Recommendations AI beta program. Kinguin became the first gaming e-commerce platform in Europe to use Recommendations AI when it launched in 2020.Pro gamer move: using a fully managed AI service Google Recommendations AI uses algorithms to deliver highly personalized suggestions tailored to a customer’s preferences. Google Cloud based these algorithms on the same research that powers models by YouTube search and Google Shopping. Algorithms are always being tuned and adjusted to focus on individuals themselves—not just items.Many shopping AIs rely on manually provisioning infrastructure and training machine learning models. Instead, Recommendations AI’s deep learning models use item and user metadata to gain insights. It processes Kinguin’s thousands of products at scale, iterating in real time. First, Kinguin pieces together a customer’s history and shopping journey. Then, using Recommendations AI, they can serve up personalized products—even for long-tail products and cold-start users. By leveraging internal tools, Kinguin didn’t need to start implementation from scratch. After a few trial sessions with Google Cloud engineers, they got started right away. Due to the fast-paced nature of a marketplace—i.e., price changes, out-of-stock items—Kinguin needed their recommendations to be as close to real time as possible. They used internal event buses to stream events and their product catalog directly to the recommendations API.Kinguin rolled out in high-traffic areas, including their home page, product page, and category pages. They analyzed heat maps and scroll maps to figure out where to test placements. They also experimented with different recommendation models such as “recently bought together” and “you may like.” Engineers also factored in where they were implementing the models. For example, the “others you might like” model would fit best on the homepage, while “frequently bought together” made sense at checkout.Understanding how product recommendations influence financials is critical for demonstrating the impact of personalization. Using BigQuery, Kinguin could analyze different cost projection models. BigQuery helped them dig into specific financial data to understand their margins and revenue gains.Playing to win: enhanced customer experienceSince adopting Recommendations AI, Kinguin has improved both customer experience and satisfaction. Search times have shortened by 20 seconds. Additionally, their average cart value has increased by 5 EUR. Conversion rates have quadrupled since the outset. Click-thru rates have doubled, increasing by 2.16 on product pages and 2.8 times on recommendations pages.“Google Recommendations AI has helped us evolve our service, increase customer loyalty and satisfaction. It has also contributed to a significant rise in sales,” says Wanli. Kinguin is already thinking about other ways of enhancing user experiences with recommendations. Ideas include their checkout process, other landing pages, and email marketing.Kinguin’s journey with Google Cloud shows how companies can leverage AI to optimize sales and deliver high-performing, low-latency recommendations to any customer touchpoint. Learn more about Recommendations AI andGoogle Cloud AI and machine learning solutions.Related ArticleAI in Retail: Google Cloud transforms Cartier’s product search technologyWith Google Cloud, Cartier developed an application to identify any watch ever designed in its 174-year history using visual recognition …Read Article
Quelle: Google Cloud Platform

3. Juli 2021

da Agency

BigQuery admin reference guide: Tables & routines

Last week in our BigQuery Reference Guide series, we spoke about the BigQuery resource hierarchy – specifically digging into project and dataset structures. This week, we’re going one level deeper and talking through some of the resources within datasets. In this post, we’ll talk through the different types of tables available inside of BigQuery, and how to leverage routines for data transformation. Like last time, we’ll link out to the documentation so you can learn more about using these resources in practice. What is a table?A BigQuery table is a resource that lives inside a dataset. It contains individual records organized into rows, with each record composed of columns (also called fields) where a specified data type is enforced. BigQuery supports numerous differentdata types including GEOGRAPHY for geospatial data, STRUCT and ARRAY for more complex data, and new parameterized data types to add specific constraints like the number of characters in a string. Data access can also be controlled at the table, row and column levels; more details on data governance will be covered later in the series. Metadata, such as descriptions and labels, can be used for surfacing information to end users and as tags for monitoring. You can create and manage a table directly in the UI, through the API / Client SDKs or in a SQL query using a DDL statement.Managed and external tablesManaged tables are tables that are backed by native BigQuery storage, which has many benefits that improve query performance including support for partitions and clusters. We’ll cover more details on BigQuery storage later in this series. Another advantage of using a managed table is that BigQuery allows you to use time travelto access data from any point within the last seven days and query data that was updated, expired or deleted. And now you can even create a snapshot of your table, to preserve its contents at a given time. While managed tables store data inside BigQuery storage, external tables are backed by storage external to BigQuery. BigQuery currently supports creating an external table from Cloud Storage, Cloud Bigtable and Google Drive. Besides an external table, you can create a connection to Cloud SQL, which is somewhat analogous to an external dataset. Here, you can leverage federated queriesto send a query that executes in Cloud SQL but returns the results to be used within BigQuery.Using external tables or federated queries may result in queries that aren’t as fast as if the data had been stored in BigQuery itself. However, they can be useful for some data transformation patterns – for example, you may want to schedule a DDL/DML query that hydrates a managed table using a federated query, which selects and transforms data from Cloud SQL. An external table might also be useful for multi-consumer workflows where BQ storage isn’t the source of truth. Like, if you have a dataproc cluster accessing data in a Cloud Storage bucket that you’re not quite ready to port into BigQuery (although I do recommend taking a look at our connector if you need some convincing). You can learn more about querying external data in this video. Logical and materialized viewsIn BigQuery, you can create a virtual table with a logical view or a materialized view. With logical views, BigQuery will execute the SQL statement to create the view at run time, it will not save the result anywhere. Additionally, you can grant users access to an authorized view to share query results without giving them access to the underlying tables. On the other hand, materialized viewsare re-computed in the background when the base data changes. No user action is required – they are always fresh! Better yet, if a query, or part of a query, against the source table can be resolved by querying the materialized view, BigQuery will reroute for improved performance. However, materialized views use a restricted SQL syntax and a limited set of aggregation functions. You can find details on limitations here. Temporary and cached results tablesAside from the tables we’ve mentioned so far, you can also create atemporary managed tableusing the TEMP or TEMPORARY keyword. This table is saved in BigQuery storage and can be referenced for the duration of the script. Temporary tables can be a good alternative to WITH clauses because the defining query is only executed once as opposed to being inlined every place the alias is referenced.Original codeOptimizedwith a as ( select …),b as ( select … from a …),c as ( select … from a …)select b.dim1, c.dim2from b, c;create temp table a asselect …;with b as ( select … from a …),c as ( select … from a …)select b.dim1, c.dim2from b, c;It’s also important to mention that BigQuery writes all query results to a table – one either explicitly identified by the user or to a cached results table. Temporary, cached results tablesare maintained per-user, per-project. There are no storage costs for temporary tables.User defined functions & proceduresIn BigQuery, a routine is either a user defined function (UDF) or a procedure. Routines allow you to re-use logic and handle your data in a unique way. A UDF is a function that is created using either SQL or Javascript, it takes arguments as input and returns a single value as an output. UDFs are often used for cleaning or re-formatting data. For example, extracting parameters from a URL string, restructuring nested data, or cleaning up strings:We even have a community driven open-source repositoryof BigQuery UDFs! Just like logical views, you can create an authorized UDF that protects aspects of the underlying data. For more details on UDFs checkout our video here. You might also want to take a look attable functions – a preview feature where you can create a SQL UDF that returns a table instead of a scalar value. Procedures, on the other hand, are blocks of SQL statements that can be called from other queries. Unlike UDFs, stored procedures can return multiple values or no values – which means you can run them to create or modify tables. In BigQuery, you can also leverage scripting capabilitieswithin procedures to control execution flow with IF and WHILE statements. Plus, you can call your UDFs within your procedure! These aspects make procedures great for extract-load-transform (ELT) driven workflows.To ensure consistent analytics across your organization, I recommend that you create a library dataset to house UDFs and procedures. You can easily grant everyone in your organization the BigQuery Data Viewer roleto the library dataset so that all analysts use consistent and up-to-date logic in their queries. Stay tuned!We hope this gave you an understanding of how to leverage some of the different resources inside of a BigQuery dataset, and to help you make decisions like using native versus external storage, logical versus materialized views, and user defined functions or procedures. Next up we’ll be talking about workload management in BigQuery by taking a look at jobs and the reservation model. Be sure to keep an eye out for more in this series by following me on LinkedIn and Twitter, and subscribing to our Youtube channel.Related ArticleThe BigQuery admin reference guide: Resource HierarchyLearn about the BigQuery Resource Hierarchy, and how to structure Projects, in the first part of our series to help BigQuery administrato…Read Article
Quelle: Google Cloud Platform