da Agency

2021 Accelerate State of DevOps report addresses burnout, team performance

Over the past seven years, more than 32,000 professionals worldwide have taken part in the Accelerate State of DevOps reports, making it the largest and longest-running research of its kind. Year over year, the Accelerate State of DevOps reports provide data-driven industry insights that examine the capabilities and practices that drive software delivery as well as operational and organizational performance. That is why Google Cloud’s DevOps Research and Assessment (DORA) team is very excited to announce our 2021 Accelerate State of DevOps Report. Our research continues to illustrate that excellence in software delivery and operational performance drives organizational performance in technology transformations. This year we also investigated the effects of SRE best practices, a secure software supply chain, quality documentation, and multicloud—all while gaining a deeper understanding of how this past year affected team’s culture and burnout.  Read below to find some of the new findings from this year’s report:Software delivery performance metricsBased on key findings from previous Accelerate State of DevOps reports, we again used four metrics to classify teams as elite, high, medium or low performers based on their software delivery: deployment frequency, lead time for changes, mean-time-to-restore, and change fail rate. This year we saw that elite performers continue to accelerate their pace of software delivery, increasing their lead time for changes from less than one day to less than one hour. Not only that, but elite performers deploy 973x more frequently than low performers, have a 6570x faster lead time to deploy, a 3x lower change failure rate, and an impressive 6570x faster time-to-recover from incidents when failure does happen. You read that right: compared to low performers, elite performers are continually able to empirically demonstrate organizational success with DevOps.The fifth metric: from availability to reliabilityHistorically we have measured availability rather than reliability, but because availability is a specific focus of reliability engineering, we’ve expanded our measure to reliability so that availability, latency, performance, and scalability are more broadly represented. Specifically, we asked respondents to rate their ability to meet or exceed their reliability targets. We found that teams with varying degrees of delivery performance see better outcomes when they also prioritize operational performance. 2021 insights: the impact of reliability, COVID and secure software supply chainsIn addition to measuring the impact of DevOps adoption on software delivery performance, this year’s DORA report also revealed many other new trends. Here’s a sampling. 1) A healthy team culture mitigates burnout during challenging timesRespondents who worked from home because of the pandemic experienced more burnout than those who stayed in the office (a small portion of our sample). Inclusive teams with a generative culture were half as likely to experience burnout during the COVID-19 pandemic. 2) The highest performers continue to raise the barFor the first time, high and elite performers make up two-thirds of respondents—compared to the 2019 report where low and medium performers made up 56% of respondents. We can confidently say that as the industry continues to accelerate its adoption of DevOps principles teams see meaningful benefits as a result.3) SRE and DevOps are complementary philosophies Extending from its core principles, Site Reliability Engineering (SRE) provides practical techniques, including the service level indicator/service level objective (SLI/SLO) metrics framework. The SRE framework offers definitions on practices and tooling that can enhance a team’s ability to consistently keep promises to their users. Teams that prioritize both delivery and operational excellence report the highest organizational performance. To investigate this, we included operations questions in the survey for the first time this year. The evidence from the survey indicated teams who excel at modern operational practices are 1.4 times more likely to report greater software delivery and operational (SDO) performance, and 1.8 times more likely to report better business outcomes.4) Cloud adoption continues to drive performanceTeams continue to move workloads to the cloud and those that leverage all five capabilities of cloud see increases in SDO performance, as well as in organizational performance. Multicloud adoption is also on the rise so that teams can leverage the unique capabilities of each provider. In fact, respondents who use hybrid or multicloud were 1.6 times more likely to exceed their organizational performance targets. 5) A secure software supply chain is both essential and drives performance Security can no longer be an afterthought—it must be integrated throughout every stage of the software development lifecycle to build a secure software supply chain. Elite performers who met or exceeded their reliability targets were twice as likely to have shifted their security practices left, i.e., implemented security practices earlier on in the software development lifecycle, and deliver reliable software quickly, and safely. 6) Good documentation is foundational for successfully implementing DevOps capabilitiesFor the first time, we measured the quality of internal documentation and its effect on other capabilities and practices. We found documentation is foundational for successfully implementing DevOps capabilities. Teams with high-quality documentation are 3.8x more likely to implement security best practices and 2.5x more likely to fully leverage the cloud to its fullest potential.Introducing the DevOps AwardsNow that we have shared some of our DevOps best practices with you, we would love to hear about how you are transforming your organization with DevOps. In our first annual DevOps Awards, we’ll recognize Google Cloud customers that have improved their deployment frequency, successfully shifted left on security, or improved their change fail rate percentage, etc. Tell us about the positive impact that DevOps has had on your teams, customers, and organization. Enter your submission here today!Thanks to everyone who took our 2021 survey. We hope this Accelerate State of DevOps report helps organizations of all sizes, industries, and regions improve their DevOps capabilities, and we look forward to hearing your thoughts and feedback. To learn more  about the report and implementing DevOps with Google cloud, check out the following resources:Download the reportTo find out more about how your organization stacks up against others in your industry, take theDevOps Quick CheckFor customized DevOps solutions for your organization, check out our newly launchedCAMP websiteLearn more about DevOps capabilities for elite performance
Quelle: Google Cloud Platform

da Agency

What is Cloud Composer?

When you are building data pipelines, you need to manage and monitor the workflows in the pipeline and often automate them to run periodically. Cloud Composer is a fully managed workflow orchestration service built on Apache Airflow that helps you author, schedule, and monitor pipelines spanning hybrid and multi-cloud environments. By using Cloud Composer instead of managing a local instance of Apache Airflow, you can benefit from the best of Airflow with no installation, management, patching, and backup overhead because Google Cloud takes care of that technical complexity. Cloud Composer is also enterprise-ready and offers a ton of security features so you don’t have to worry about it yourself. Last but not least, the latest version of Cloud Composer supports autoscaling, which provides cost efficiency and additional reliability for workflows that have bursty execution patterns. How does Cloud Composer work? In data analytics, a workflow represents a series of tasks for ingesting, transforming, analyzing, or utilizing data. In Airflow, workflows are created using directed acyclic graphs (DAGs).A DAG is a collection of tasks that you want to schedule and run, organized in a way that reflects their relationships and dependencies. DAGs are created in Python scripts, which define the DAG structure (tasks and their dependencies) using code. The purpose of a DAG is to ensure that each task is executed at the right time, in the right order, and with the right issue handling.Each task in a DAG can represent almost anything—for example, one task might perform data ingestion, another sends an email, and yet another runs a pipeline.Click to enlargeHow to run workflows in Cloud Composer? After you create a Cloud Composer environment, you can run any workflows your business case requires.  The Composer service is based on a distributed architecture running in GKE and other Google Cloud services. You can schedule a workload at a specific time or you can start a workflow when a specific condition is met, for example when an object is saved to a storage bucket. Cloud Composer comes with built-in integrations to almost all Google Cloud products including BigQuery and Dataproc; it also supports integrations (enabled by provider packages from vendors) with applications running on-prem or on another cloud. Here is a list of built-in integrations and provider packages.   Cloud Composer security featuresPrivate IP: Using private IP means that the compute node in Cloud Composer is not publicly accessible and therefore is protected from the public internet. Developer can access the internet but cannot be accessed from outside. Private IP + Web Server ACLs: The user interface for Airflow is protected by authentication. Only authenticated customers can access the specific Airflow user interface. For additional network level security you can use web server access controls along with Private IP which helps limit access from the outside world by whitelisting a set of IP addresses. VPC Native Mode: In conjunction with other features VPC native mode helps limit access to Composer components in the same VPC network, keeping it protected.VPC Service Controls: Provides increased security by enabling you to configure a network service perimeter that prevents access from the outside world and also prevents access to the outside world.Customer Managed Encryption Keys (CMEK): Enabling CMEK lets you provide your own encryption keys to encrypt/decrypt environment data.Restricting Identities By Domain: This features enables you to restrict the set of identities that can access Cloud Composer environments to specific domain names, e.g.  @yourcompany.com.Integration with Secrets Manager: You can use a built-in integration with Secrets Manager to protect keys and passwords used by your DAGs for authentication to external systems.If you are building data pipelines, then you need to check out Cloud Composer for easy and fully managed workflow orchestration. For a more in-depth look into Cloud Composer check out the documentation.For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.Related ArticleOrchestrating your data workloads in Google CloudThe Data Orchestration is becoming more important as workflows expand and become more complex on the Cloud. This blog touches on how to t…Read Article
Quelle: Google Cloud Platform