Shopify engineers deliver on peak performance during Black Friday Cyber Monday 2021

Many predicted this would be the biggest Black Friday/Cyber Monday (BFCM) weekend ever recorded. And it looks like those predictions were right — especially for independent and direct to consumer (DTC) brands. Shopify, a leading provider of essential internet infrastructure for commerce, works with more than 1.7 million merchants worldwide. Over the course of the long weekend, the company’s merchants welcomed a record number of consumers purchasing from independent and DTC brands (47 million globally) and drove $6.3 billion in global sales for a 23% increase y/y (up from $5.1 billion in global sales in 2020). These results are incredible on their own, and even more so when we remember that these increases are on top of last year’s pandemic-fueled shift to online shopping.But what does it take to handle this type of scale? Noted across the industry for their innovative approaches to peak time shopping solutions, Shopify engineers are leveraging Google services to enhance performance for merchants and shoppers like never before. Shopify engineers are showing commerce how big event performance is doneShopify knows how critical peak seasons are for its merchants. With peak sales of more than $3.1 million per minute at 12:02 PM EST on Black Friday, November 26, Shopify leverages one of the most skilled teams of engineers to develop pioneering tools that improve scalability and velocity for merchants.“Achieving a record-breaking sales weekend for BFCM 2021 is only possible with an infrastructure that’s built for performance and scale,” said Delaney Manders, VP of Engineering for Shopify. “With the incredible collaboration between Shopify engineering and Google Cloud, we averaged about 30TB/min of egress traffic across our infrastructure and helped our merchants deliver near-perfect uptime for their consumers during peak sales periods.” Through customer experience and operational optimization tools designed with Google Cloud services, Shopify delivers streamlined and reliable performance for merchants and consumers during high traffic events. With tools like Shopify Inbox and Shop Pay, Shopify engineers have designed some of the finest merchant and consumer experience optimization tools in commerce. Through their solutions, Shopify engineers work smarter, not harder, and enable merchants to meet peak event demand without added hassle or stress. Black Friday/Cyber Monday may be the event of the season for many Shopify merchants but the massive scalability, low latency, and optimized uptime make high traffic and high risk events as easy to execute as any other day.From the merchant, to the backend, to the consumer, Shopify’s engineers have developed tools to optimize performance for merchants and shoppers, while simplifying their workloads. With these approaches, big event days at Shopify go as smoothly as any other.Shop Pay takes the lines (and the forms) out of shoppingFor the end customer, a time-consuming checkout experience can easily discourage or delay a purchase. When it comes to big events like Black Friday/Cyber Monday, Shopify recognizes the particular significance of an intelligent and seamless checkout experience. The company’s engineers developed Shop Pay, a solution that helps accelerate the purchase process. In addition to supporting faster checkout, Shop Pay also personalizes the shopping experience by remembering a shopper’s preferences and encrypting everything for optimal safety.Shopify’s data shows that Shop Pay increases checkout speed by 4x. Following an analysis of 10,000 of its largest merchants, Shopify found merchants who enabled Shop Pay had an average checkout-to-order rate 1.72x times higher than those going through regular checkouts. This innovation in platform performance significantly increased growth and retention for merchants. Shop Mover gets the backend ready for a crowd  To keep operations as agile and modern as possible, Shopify developed and open sourced a general purpose MySQL data migration tool, Ghostferry. Supported by Google Cloud services, Ghostferry moves data across different MySQL instances while the application is still running and with minimal downtime (<5 seconds). Shopify’s Shop Mover is built on top of Ghostferry and Google Cloud services, enabling load-balancing of data shards across multiple databases. In addition to being the tool used for the initial migration out of Shopify’s data centers and into Google Cloud, Shop Mover now moves hundreds of thousands of shops every year and is a reason Shopify merchants could handle the high volume of BFCM 2021. Shopify’s global reach means better deals for merchants and shoppers. To expand access to new regions while optimizing platform performance for merchants, Shopify engineers leveraged the global infrastructure of Google Cloud. Shopify supports merchants from around the world with Google Cloud’s network of 25 regions and 76 availability zones. With lower latency and higher reliability, performance is optimized for shoppers and merchants around the world. Through its partnership with Google, Shopify engineers are leveragingGlobal Virtual Private Cloud (VPC) to streamline the writing and deployment of applications that span multiple regions, and in-country disaster recovery which helps Shopify maintain business continuity across the globe.  The culture of creativity, scale, and velocity among Shopify’s engineering team continues to drive exciting solutions for enhanced peak event performances. Backed by fresh approaches to improving the customer experience and streamlining operations at scale, Shopify engineers can expect a streamlined workday and merchants can confidently deliver for shoppers without a hassle. And this translates to a better holiday shopping experience for merchants and shoppers alike.
Quelle: Google Cloud Platform

Vertex AI NAS: higher accuracy and lower latency for complex ML models

Vertex AI launched with the premise “one AI platform, every ML tool you need.” Let’s talk about how Vertex AI streamlines modeling universally for a broad range of use cases. The overall purpose of Vertex AI is to simplify modeling so that enterprises can fast track their innovation, accelerate time to market, and ultimately increase return on ML investments. Vertex AI facilitates this in several ways. Features like Vertex AI Workbench, for example, speed up training and deployment of models by five times compared to traditional notebooks. Vertex AI Workbench’s native integration with BigQuery and Spark means that users without data science expertise can more easily perform machine learning work. Tools integrated into the unified Vertex AI platform, such as state of the art pre-trained APIs and AutoML, make it easier for data scientists to build models in less time. And for modeling work that lends itself best to custom modeling, Vertex AI’s custom model tooling supports advanced ML coding, with nearly 80% fewer lines of code required (compared to competitive platforms) to train a model with custom libraries. Vertex AI delivers all this while maintaining a strong focus on Explainable AI. Yet organizations with the largest investments in AI and machine learning, with teams of ML experts, require extremely advanced toolsets to deliver on their most complex problems. Simplified ML modeling isn’t relegated to simple use cases only.Let’s look at Vertex AI Neural Architecture Search (NAS), for instance. Vertex AI NAS enables ML experts at the highest level to perform their most complex tasks with higher accuracy, lower latency, and low power requirements. Vertex AI NAS originates from the deep experience Alphabet has with building advanced AI at scale. In 2017, the Google Brain team recognized we need a better way to scale AI modeling, so they developed Neural Architecture Search technology to create an AI that generates other neural networks, trained to optimize their performance in a specific task the user provides. To the astonishment of many in the field, these AI-optimized models were able to beat a number of state of the art benchmarks, such as ImageNet and SOTA mobilenets, setting a new standard for many of the applications we see in use today, including many Google-internal products. Google Cloud saw the potential of such a technology and shipped in less than a year a productized version of the technique (under the brand AutoML). Vertex AI NAS is the newest and most powerful version of this idea, using the most sophisticated innovation that has emerged since the initial research.Customer organizations are already implementing Vertex AI NAS for their most advanced workloads. Autonomous vehicle company Nuro is using Vertex AI NAS, and Jack Guo, Head of Autonomy Platform at the company, states, “Nuro’s perception team has accelerated their AI model development with Vertex AI NAS. Vertex AI NAS have enabled us to innovate AI models to achieve good accuracy and optimize memory and latency for the target hardware. Overall, this has increased our team’s productivity for developing and deploying perception AI models.” And our partner ecosystem is growing for Vertex AI NAS. Google Cloud and Qualcomm Technologies have collaborated to bring Vertex AI NAS to the Qualcomm Technologies Neural Processing SDK, optimized for Snapdragon 8. This will bring AI to different device types and use cases, such as those involving IoT, mixed reality, automobiles, and mobile.Google Cloud’s commitments to making machine learning more accessible and useful for data users, from the novice to the expert, and to increasing the efficacy of machine learning for enterprises are at the core of everything we do. With the suite of unified machine learning tools within Vertex AI, organizations can take advantage of every ML tool they need on one AI platform. Ready to start ML modeling with Vertex AI? Start building for free. Want to know how Vertex AI Platform can help your enterprise increase return on ML investments? Contact us.Related ArticleNew to ML: Learning path on Vertex AIIf you’re new to ML, or new to Vertex AI, this post will walk through a few example ML scenarios to help you understand when to use which…Read Article
Quelle: Google Cloud Platform

Ankündigung von Amazon-Athena-ACID-Transaktionen, unterstützt von Apache Iceberg (Vorversion)

Wir freuen uns, die öffentliche Vorversion von Amazon-Athena-ACID-Transaktionen ankündigen zu können, einer neuen Funktion, die der SQL-Datenbearbeitungssprache (DML) von Athena-Schreib-, Lösch-, Aktualisierungs- und Zeitreisevorgänge hinzufügt. Athena-ACID-Transaktionen ermöglichen es mehreren gleichzeitigen Benutzern, zuverlässige Änderungen auf Zeilenebene an ihren Amazon-S3-Daten von Athenas Konsole, API und ODBC- und JDBC-Treibern vorzunehmen. Basierend auf dem Apache-Iceberg-Tabellenformat sind Athena-ACID-Transaktionen mit anderen Diensten und Engines wie Amazon EMR und Apache Spark kompatibel, die das Iceberg-Tabellenformat unterstützen.
Quelle: aws.amazon.com