The need for speed: Using C2 machines for your HPC workloads

Cloud opens many new possibilities for High Performance Computing (HPC). But while the cloud offers the latest technologies and a wide variety of machine types (VMs), not every VM is suited to the demands of HPC workloads. Google Cloud’s Compute-optimized (C2) machines are specifically designed to meet the needs of the most compute-intensive workloads, such as HPC applications in fields like scientific computing, Computer-aided Engineering (CAE), biosciences, and Electronic Design Automation (EDA), among many others.The C2 is based on the second generation Intel® Xeon® Scalable Processor and provides up to 60 virtual cores (vCPUs) and 240GB of system memory. C2s can run at a sustained frequency of 3.8GHz and offer more than 40% improvement compared to previous generation VMs for general applications. Compared to previous generation VMs, total memory bandwidth improves by 1.21X and memory bandwidth/vCPU improves by 1.94X.1 Here we take a deeper look at using C2 VMs for your HPC workloads on Google Cloud.Resource isolationTightly-coupled HPC workloads rely on resource isolation for predictable performance. C2 is built for isolation and consistent mapping of shared physical resources (e.g., CPU caches, and memory bandwidth). The result is reduced variability and more consistent performance. C2 also exposes and enables explicit user control of CPU power states (“C-States”) on larger VM sizes, enabling higher effective frequencies and performance.NUMA nodesIn addition to hardware improvements, Google Cloud has enabled a number of HPC-specific optimizations on C2 instances. In many cases, tightly-coupled HPC applications require careful mapping of processes or threads to physical cores, along with care to ensure processes access memory that is closest to their physical cores. C2s provide explicit visibility and control of NUMA domains to the guest operating system (OS), enabling maximum performance.AVX-512 supportSecond generation Xeon processors support Intel Advanced Vector Extension 512 (Intel AVX-512) for data parallelism. AVX-512 instructions are SIMD (Single Instruction Multiple Data) instructions, and along with additional and wider registers enable packing of 64 single-precision (or 32 double-precision) floating point operations into one instruction. This means that more can be done in every clock cycle, reducing overall execution time. The latest generation of AVX-512 instructions in the 2nd generation Xeon processor include DL Boost instructions that significantly improve performance for AI inferencing by combining three INT8 instructions into one—thereby maximizing the use of compute resources, utilizing the cache better, and avoiding potential bandwidth bottlenecks.Low- latencyHPC workloads often scale out to multiple nodes in order to accelerate time to completion. Google Cloud has enabled “Compact Placement Policy” on the C2, which allocates up to 1320 vCPUs placed in close physical proximity, minimizing cross-node latencies. Compact placements, in conjunction with Intel MPI library, optimizes multi-node scalability of HPC applications. You can learn more about best practices for ensuring low latency on multi-node workloads here.Development toolsAlong with the hardware optimizations, Intel offers a comprehensive suite of development tools (including performance libraries, Intel Compilers, and performance monitoring and tuning tools) to make it simpler to build and modernize code with the latest techniques in vectorization, multithreading, multi-node parallelization, and memory optimization. Learn more about Intel’s Parallel Studio XE here.Bringing it all together Combining all the improvements in hardware and optimizations done in Google Cloud stack, C2 VMs perform up to 2.10X better compared to previous generation N1 for HPC workloads for roughly the same size VM.2In many cases HPC applications can scale up to the full node. A single C2 node (60 vCPUs and 240GB) offers up to 2.49X better performance/price compared to a single N1 node (96 vCPUs and 360GB).3C2s are offered in predefined shapes intended to deliver the most appropriate vCPU and memory configurations for typical HPC workloads. In some cases, it is possible to further optimize performance or performance/price via a custom VM shape. For example, if a certain workload is known to require less than the default 240GB of a C2 standard 60vCPU VM, a custom N2 machine with less memory can deliver roughly the same performance at a lower cost. We were able to achieve up to 1.09X better performance/price by tuning the VM shape to the needs of several common HPC workloads.4Get started todayAs more HPC workloads start to benefit from the agility and flexibility of cloud, Google Cloud and Intel are joining forces to create optimized solutions for specific needs of these workloads. With the latest optimizations in Intel 2nd generation Xeon processors and Google Cloud, C2 VMs deliver the best solution for running HPC applications in Google Cloud, while giving you the freedom to build and evolve around your unique business needs. Many of our customers with need for high performance have moved their workloads to C2 VMs and confirmed our expectations.To learn more about C2 and the second generation of Intel Xeon Scalable Processor, contact your sales representative or reach out to us here. And if you’re participating in SC20 this week, be sure to check out our virtual booth, where you can watch sessions, access resources, and chat with our HPC experts.1. Based on internal analysis of our c2-standard-60 and n1-standard-96 machine types, using the STREAM Triad Best Rate benchmark.2. Based on internal analysis of our c2-standard-60 and n1-standard-96 machine types, using our Weather Research Forecasting (WRF) benchmark.3. Based on the High Performance Conjugate Gradients (HPCG) benchmark, analyzing Google Cloud VM Instance pricing on C2-standard-60 ($3.1321/hour) and N1-standard-96 ($4.559976) as of 10/15/20204. Based on GROMACS and NAMD benchmarks, analyzing Google Cloud VM Instance pricing on N2-custom-80 with 160GB ($3.36528) and C2-standard-60 ($3.1321/hour) as of 10/15/2020Related ArticleIntroducing Compute- and Memory-Optimized VMs for Google Compute EngineGoogle Cloud is the first public cloud provider to offer Compute-Optimized VMs and Memory-Optimized VMs based on Intel 2nd Generation Xeo…Read Article
Quelle: Google Cloud Platform

Empowering customers and the ecosystem with an open cloud

Every organization that moves to the cloud has a unique journey driven by many factors including evolving operating environments and regulatory requirements. For all organizations, including those experiencing growth across regions and dynamic market circumstances, we recommend an open cloud approach that ensures operational and technical consistency across public clouds or private data centers and effective management of infrastructure, applications, and data across the organization.We believe that an open cloud can meet the needs of diverse companies, providing choice, flexibility and openness. Our open cloud philosophy is grounded in the belief that customers need autonomy and control over their infrastructure. Giving customers options to build, migrate and deploy their applications across multiple environments both in the cloud and on-premises allows them to avoid vendor lock-in and innovate across environments faster. We are proud of our leadership in advancing an open cloud, and this commitment underpins and drives our contributions to the open source and open data communities, as well as our approach to building technology solutions. Advancing computing through open source Open source plays a critical role in an open cloud. Many companies have mission-critical workloads or sensitive data that have “survivability requirements” in the event that a provider is forced to suspend or terminate cloud services due to country or region policy changes. To move workloads to other clouds, it’s important to develop them using open source and open standards.At Google Cloud, we don’t think it’s possible to fully address survivability requirements with a proprietary solution. Instead, solutions based on open source tools and open standards are the route to addressing customer and policymaker concerns. More importantly, open source gives customers the flexibility to deploy—and, if necessary, migrate—critical workloads across or off public cloud platforms.Google has a long history of sharing technology through open source—from projects like Kubernetes, which is now the industry standard in container portability and interoperability in the cloud, to TensorFlow, a platform to help everyone develop and train machine learning models. As Google’s Chief Economist Hal Varian said, “Open data and open source are good not only for us and our industry, but also benefit the world at large.” Our belief in customer choice is fundamental to how we develop our technology and rooted in leveraging open source APIs and interoperable solutions. In addition, we partner with the leading organizations in the fields of data management and analytics to build products that combine the benefits of open source with managed cloud solutions.Another way we provide flexibility is hybrid and multi-cloud environments. Anthos, our hybrid and multi-cloud platform, is built on open technologies like Kubernetes, Istio, and Knative, enabling an ecosystem that fosters competition and that unlocks new partnerships. In this spirit, last week OVHcloud and Google Cloud announced a strategic partnership to jointly build a trusted cloud solution in Europe. This partnership will focus on delivering the best of Google Cloud technology innovation and value in the most agile way, and help European customers accelerate their business transformation in the cloud while meeting addressing their strict data security and privacy requirements.Break down data silos and uncover new insights with public datasetsCustomers rely on Google Cloud to get better insights from their data. Our data analytics solutions such as BigQuery help them harness the potential of that data. One of our newest analytics solutions, BigQuery Omni, allows customers to cost-effectively access and securely analyze data across multi-cloud environments. Those tools enable customers to make their own data more open—both in and out of their organization. As we help enable data accessibility and portability, our highest priority is to do so securely and responsibly. At the same time, through the Google Cloud Public Datasets program, we work with data providers to host 100+ high-demand public datasets to allow customers and the research community to discover unique insights for solving real business and societal problems. For example, earlier this year, we added critical COVID-19 public datasets to support the global response to the novel coronavirus.We also share our discoveries and tools with the community to help everyone share data safely and in a manner that advances the important work of researchers, developers and journalists. Teams at Google have released over 80 open datasets through our research site, and share other aggregated, anonymized product insights. Take for example YouTube-8M, a large-scale, labeled video dataset used by researchers to further computer vision and video understanding. In addition, with more than 31 million datasets, Dataset Search allows anyone to discover and filter relevant data sets in accordance with usage rights, formats, and other key parameters. And our Kaggle community of nearly 5 million users hosts 50,000 public datasets and 400,000 public notebooks to support machine learning and artificial intelligence research. Paving the way to an open cloud through continued collaborationGoogle Cloud will continue to build towards an open cloud and work with partners and policymakers to support our customers, open-source communities, and society at large. We are excited to see the important work organizations achieve through openness, and remain committed to supporting them through our continued contributions to open source and open data.
Quelle: Google Cloud Platform

AWS Managed Services (AMS) ist jetzt in der AWS-Region Europa (Paris) verfügbar

Amazon Managed Services (AMS) unterstützt nun die AWS-Region Europa (Paris). Sie können AMS verwenden, um den Betrieb Ihres Kontos zu überwachen, einschließlich Vorfall-, Problem-, Patch- und Sicherheitsverwaltung für Ihre AWS-Ressourcen. Zusätzlich zum Betrieb der AWS-Services bietet AMS eine sichere AWS Landing Zone und Funktionen, die Ihnen helfen, die Anforderungen der Compliance-Programme für HIPAA, HITRUST, GDPR, SOC, ISO und PCI zu erfüllen.
Quelle: aws.amazon.com

Amazon Polly startet eine neue australisch-englische neuronale Text-zu-Sprache-Stimme

Amazon Polly ist ein Service, der Text in natürliche Sprache konvertiert. Wir freuen uns, heute die allgemeine Verfügbarkeit von Olivia, Pollys erster australisch-englischer Stimme, über neurale Text-zu-Sprache (NTTS) verfügbar, bekanntzugeben. Dank ihrer einzigartigen stimmlichen Persönlichkeit klingt die Stimme ausdrucksstark, natürlich und leicht verständlich. Kunden von Amazon Polly können jetzt eine Auswahl von drei australisch-englischen Stimmen genießen: Russell, Nicole und Olivia. 
Quelle: aws.amazon.com

Automatische ABR (Adaptive Bit Rate)-Konfiguration ist jetzt in AWS Elemental MediaConvert verfügbar

In AWS Elemental MediaConvert wurde eine neue Funktion namens Automatische ABR-Konfiguration hinzugefügt, die automatisch die ABR-Kodierungskonfiguration (Adaptive Bitrate) für jedes Quellvideo anpasst. Die automatische ABR-Konfiguration vereinfacht die Einrichtung der Transkodierung, optimiert die Videoqualität und reduziert die ABR-Paketgröße. Die automatische ABR-Konfiguration ist ohne zusätzliche Kosten für MediaConvert-Aufgaben in der Preisstufe On-Demand, Professional verfügbar. Um mehr zu erfahren, lesen Sie bitte die Dokumentation zu Automatische ABR-Konfiguration.
Quelle: aws.amazon.com