Mirantis Joins Mission of the Common Vulnerabilities and Exposures (CVE) Program

Initiative to identify, define and catalog publicly-disclosed cybersecurity vulnerabilities CAMPBELL, Calif., January 18, 2022 — Mirantis, the open cloud company, today announced the company’s designation as a CVE Numbering Authority (CNA) by the CVE Program, which is sponsored by the U.S. Department of Homeland Security. As a CNA, the Mirantis Product Security Incident Response Team … Continued
Quelle: Mirantis

Understanding Firestore performance with Key Visualizer

Firestore is a serverless, scalable, NoSQL document database. It is ideal for rapid and flexible web and mobile application development, and uniquely supports real-time client device syncing to the database.To get the best performance out of Firestore, while also making the most out of Firestore’s automatic scaling and load balancing features, you need to make sure the data layout of your application allows requests to be processed optimally, particularly as your user traffic increases. There are some subtleties to be aware of when it comes to what could happen when traffic ramps up, and to help make this easier to identify, we’re announcing the General Availability of Key Visualizer, an interactive, performance monitoring tool for Firestore.Key Visualizer generates visual reports based on Firestore documents accessed over time, that will help you understand and optimize the access patterns of your database, as well as troubleshoot performance issues. With Key Visualizer, you can iteratively design a data model or improve your existing application’s data usage pattern.Tip: While Key Visualizer can be used with production databases, it’s best to identify performance issues prior to rolling out changes in production. Consider running application load tests with Firestore in a pre-production environment, and using Key Visualizer to identify potential issues.Viewing a visualizationThe Key Visualizer tool is available to all Firestore customers. Visualizations are generated at every hour boundary, covering data for the preceding two hours. Visualizations are generated as long as overall database traffic during a selected period meets the scan eligibility criteria.To get an overview of activity using Key Visualizer, first select a two-hour time period and review the heatmap for the “Total ops/s” metric. This view estimates the number of operations per second and how they are distributed across your database. Total ops/s is an estimated sum of write, lookup, and query operations averaged by seconds.Firestore automatically scales using a technique called range sharding. When using Firestore, you model data in the form of documents stored in hierarchies of collections. The collection hierarchy and document ID is translated to a single key for each document. Documents are logically stored and ordered lexicographically by this key. We use the term “key range” to refer to a range of keys. The full key range is then automatically split up as-needed, driven by storage and traffic load, and served by many replicated servers inside of Firestore.The following example of Key Visualizer visualization shows a heatmap where there are some major differences in the usage pattern across the database. The X-axis is time, and the Y-axis is the key range for your database, broken down into buckets by traffic.Ranges shown in dark colors have little or no activity.Ranges in bright colors have significantly more activity. In the example below, you can see the “Bar” and “Qux” collections going beyond 50 operations per second for some period of time.Additional methods of interpreting Key Visualizer visualizations are detailed in our documentation.Besides the total number of operations, Key Visualizer also provides views with metrics for ops per second, average latency, and tail latency, where traffic is broken down for writes and deletes, lookups, and queries. This capability allows you to identify issues with your data layout or poorly balanced traffic that may be contributing to increased latencies.Hotspots and heatmap patternsKey Visualizer gives you insight into how your traffic is distributed, and lets you understand if latency increases correlate with a hotspot, thus providing you with information to determine what parts of your application need to change. We refer to a “hotspot” as a case where traffic is poorly balanced across the database’s keyspace. For the best performance, requests should be distributed evenly across a keyspace. The effect of a hotspot can vary, but typically hotspotting causes higher latency and in some cases, even failed operations.Firestore automatically splits a key range into smaller pieces and distributes the work of serving traffic to more servers when needed. However, this has some limitations. Splitting storage and load takes time, and ramping up traffic too fast may cause hotspots while the service adjusts. The best practice is to distribute operations across the key range, while ramping up traffic on a cold database with 500 operations per second, and then increasing traffic by up to 50% every 5 minutes. This is called the “500/50/5″ rule, and allows you to rapidly warm up a cold database safely. For example, ramping to 1,000,000 ops/s can be achieved in under two hours.Firestore can automatically split a key range until it is serving a single document using a dedicated set of replicated servers. Once this threshold is hit, Firestore is unable to create further splits beyond a single document. As a result, high and sustained volumes of concurrent operations on a single document may result in elevated latencies. You can observe these high latencies using Key Visualizer’s average and tail latency metrics. If you encounter sustained high latencies on a single document, you should consider modifying your data model to split or replicate the data across multiple documents.Key Visualizer will also help you identify additional traffic patterns:Evenly distributed usage: If a heatmap shows a fine-grained mix of dark and bright colors, then reads and writes are evenly distributed throughout the database. This heatmap represents an effective usage pattern for Firestore, and no additional action is required.Sequential Keys: A heatmap with a single bright diagonal line can indicate a special hotspotting case where the database is using strictly increasing or decreasing keys (document IDs). Sequential keys are an anti-pattern in Firestore, which will result in elevated latency especially at higher operations per second. In this case, the document IDs that are generated and utilized should be randomized. To learn more, see the best practices page.Sudden traffic increase: A heatmap with a key range that suddenly changes from dark to bright indicates a sudden spike in load. If the load increase isn’t well distributed across the key range, and exceeds the 500/50/5 rule best practice, the database can experience elevated latency in the operations. In this case, the data layout should be modified to reflect a better distribution of usage and traffic across the keyspace.Next stepsFirestore Key Visualizer is a performance monitoring tool available to administrators and developers to better understand how their applications interact with Firestore. With this launch, Firestore joins our family of Cloud-native databases, including Cloud Spanner and Cloud Bigtable, in offering Key Visualizer to its customers. You can get started with Firestore Key Visualizer for free, from the Cloud Console.AcknowledgementSpecial thanks to Minh Nguyen, Lead Product Manager for Firestore, for contributing to this post.
Quelle: Google Cloud Platform

How can demand forecasting approach real time responsiveness? Vertex AI makes it possible

Everyone wishes they had a crystal ball—especially retailers and consumer goods companies looking for the next big trend, or logistics companies worried about the next big storm. With a veritable universe of data now at their fingertips (or at least at their keyboards), these companies can now get closer to real-time forecasting across a range of areas when they leverage the right AI and machine learning tools.For retailers, supply chain, and consumer goods organizations, accurate demand forecasting has always been a key driver of efficient business planning, inventory management, streamlined logistics and customer satisfaction. Accurate forecasting is critical to ensure that the right products, in the right volumes, are delivered to the right locations. Customers don’t like to see items out of stock, but too much inventory is costly and wasteful. Retailers lose over a trillion dollars a year in mismanaged inventory, according to IHL Group, whereas a 10% to 20% improvement in demand forecasting accuracy can directly produce a 5% reduction in inventory costs and a 2% to 3% increase in revenue (Notes from the AI Frontier, McKinsey & Company).Yet, inventory management is only one of the applications among many that demand forecasting can support—retailers need to also staff their stores and their support centers for busy periods, plan promotions and evaluate different factors that can impact store or online traffic. As retailers’ product catalog and global reach broaden, the available data becomes more complex and more difficult to forecast accurately. Unconstrained activities through the pandemic have only accentuated supply chain bottlenecks and forecasting challenges as the pace of change has been so rapid. Retailers can now infuse machine learning into their existing demand forecasting to achieve high forecast accuracy, by leveraging Vertex AI Forecast. This is one of the latest innovations born of Google Brain researchers and being made available to enterprises within an accelerated time frame. Top performing models within two hoursVertex AI Forecast can ingest datasets of up to 100 million rows covering years of historical data for many thousands of product lines from BigQuery or CSV files. The powerful modeling engine would automatically process the data and evaluate hundreds of different model architectures and package the best ones into one model which is easy to manage, even without advanced data science expertise. Users can include up to 1,000 different demand drivers  (color, brand, promotion schedule, e-commerce traffic statistics, and more) and set budgets to create the forecast. Given how quickly market conditions change, retailers need an agile system that can learn quickly. Teams can build demand forecasts at top-scoring accuracy with Vertex AI Forecast within just two hours of training time and no manual model tuning.The key part of the Vertex AI Forecast is model architecture search, where the service evaluates hundreds of different model architectures and settings. This algorithm allows Vertex AI Forecast to consistently find the best performing model setups for a wide variety of customers and datasets. Google has effectively built the brain that is applied towards demand forecasting in a non-intrusive and contextual way, to merge the art and (data) science of accurate demand forecasting. In benchmarking tests based on Kaggle datasets, Vertex AI Forecast performed in the highest 3% of accuracy in M5, the World’s Top Forecasting Competition. Leading retailers are already transforming their operations and reaping the benefits of highly accurate forecasting. ​​”Magalu has deployed Vertex AI Forecast to transform our forecasting predictions, by implementing distribution center level forecasting and reducing prediction errors simultaneously” said Fernando Nagano, director of Analytics and Strategic Planning at Magalu. “Four-week live forecasting showed significant improvements in error (WAPE) compared to our previous models,” Nagano added. “This high accuracy insight has helped us to plan our inventory allocation and replenishment more efficiently to ensure that the right items are in the right locations at the right time to meet customer demand and manage costs appropriately.”From weather to leather, Vertex AI can handle all kind of inputsWith the hierarchical forecast capabilities of Vertex AI Forecast, retailers can generate a highly accurate forecast that works on multiple levels (for example, tying together the demand at the individual item, store level, and regional levels) to minimize challenges created by organizational silos. Hierarchical models can also improve overall accuracy when historical data is sparse. When the demand for individual items is too random to forecast, the model can still pick up on patterns at the product category level.Vertex AI can ingest large volumes of structured and unstructured data, allowing planners to include many relevant demand drivers such as weather, product reviews, macroeconomic indicators, competitor actions, commodity prices, freight charges, ocean shipping carrier costs, and more. Vertex AI Forecast explainability features can show how each of these drivers contributes to the forecast and help the decision makers understand what drives the demand to take the corrective action early.The demand driver attributions are available not only for the overall forecast but for each individual item at every point. For instance, planners may discover that promotions are the main drivers of demand in the clothing category on weekdays, but not during the holidays. These kinds of insights can be invaluable when decisions are made on how to act on forecasts.Vertex AI Forecast is already helping Lowe’s with a range of models at the company’s more than 1,700 stores, according to Amaresh Siva, senior vice president for Innovation, Data and Supply Chain Technology at Lowe’s.“At Lowe’s, our stores and operations stretch across the United States, so it’s critical that we have highly accurate SKU-level forecasts to make decisions about allocating inventory to specific stores and replenishing items in high demand,” Siva said. “Using Vertex AI Forecast, Lowe’s has been able to create accurate hierarchical models that balance between SKU and store-level forecasts. These models take into account our store-level, SKU-level, and region-level inventory, promotions data and multiple other signals, and are yielding more accurate forecasts.”Key retail and supply chain partners, including o9 Solutions and Quantiphi, are already integrating Vertex AI Forecast into to provide value added services to customers. To learn more about demand forecasting with Vertex AI, please contact your Field Sales Representative, or try Vertex AI for free here.Related ArticleGoogle Cloud unveils Vertex AI, one platform, every ML tool you needGoogle Cloud launches Vertex AI, a managed platform for experimentation, versioning and deploying ML models into production.Read Article
Quelle: Google Cloud Platform