da Agency

Unlock the power of change data capture and replication with new, serverless Datastream, now GA

We’re excited to announce that Datastream, Google Cloud’s serverless change data capture (CDC) and replication service, is now generally available. Datastream allows you to synchronize data across disparate databases, storage systems, and applications reliably and with minimal latency to support real-time analytics, database replication, and event-driven architectures. You can easily and seamlessly deliver change streams from Oracle and MySQL databases into Google Cloud services such as BigQuery, Cloud SQL, Google Cloud Storage and Cloud Spanner, saving time and resources and ensuring your data is accurate and up to date. Get started with Datastream today.Datastream provides an integrated solution for CDC replication use cases with custom sources and destinations*Check the documentation page for all supported sources and destinations.Since our public preview launch earlier this year, we’ve seen Datastream used across a variety of industries, by customers such as Chess.com, Cogeco, Schnuck Markets, and MuchBetter. This early adoption strengthens the message we’ve been hearing from customers about the demand for change data capture to provide replication and streaming capabilities for real-time analytics and business operations. MuchBetter is a multi-award-winning e-wallet app, providing a truly secure and enjoyable banking alternative for customers all over the world. Working with Google Cloud Premier Partner Datatonic, they’re leveraging Datastream to replicate real-time data from MySQL OLTP databases into a BigQuery data warehouse to power their analytics needs. According to Andrew McBrearty, Head of Technology at MuchBetter, “from MuchBetter’s point of view, leveraging Dataflow, BigQuery and Looker has unlocked additional insights from our ever-increasing operational data. Using Datastream in our solution ensured continued real-time capability – we now have trend analysis in place, improved efficiency across the business, and the ability to use our data to derive actionable insights and to make data-driven decisions. This means we can continue to grow and adapt at a pace our customers have come to expect from MuchBetter. And for the first time, the world of ML and AI is open to us.”Getting to know DatastreamGoogle Cloud customers are choosing Datastream for real-time change data capture because of its differentiated approach:Simple experienceReal-time replication of change data shouldn’t be complicated: database preparation documentation, secure connectivity setup, and stream validation should be built right into the flow. Datastream delivers on this experience, as MuchBetter discovered during their evaluation of the product. “Datastream’s ease-of-use and immediate availability (serverless) meant we could start our evaluation and immediately see results”, says Mark Venables, Principal Data Engineer at MuchBetter. “For us, this meant getting rid of the considerable pre-work needed to align proof of concept tests with third-party CDC suppliers.”Datastream guides you to success by providing detailed pre-requisites and step-by-step configuration guidelines to prepare your source database for CDC ingestion.End-to-end solutionBuilding pipelines to replicate changes from your source database shouldn’t take up all of your team’s time. Use pre-built Dataflow templates to easily replicate data into BigQuery, Cloud Spanner or Cloud SQL. Out of the box, these Dataflow templates will automatically create the tables and update the data at the destination, taking care of any out-of-order or duplicate events, and providing error resolution capabilities. Leverage the templates’ flexibility to fine-tune Dataflow to fit your specific needs. “Google-managed Dataflow templates meant getting our pipelines up and running with minimal effort and fuss – this allowed more time to be spent on more complex pipeline development whilst tactically delivering solutions to our users,” says Venables.Secure Datastream keeps your migrated data secure, supporting private connectivity between source and destination databases. “Establishing connectivity is often viewed as hard. Datastream surprised us with its ease of use & setup, even in more secure modes,” says Grzegorz Dlugolecki, Principal Cloud Architect at Chess.com, a leading online chess community and mobile application, hosting more than ten million chess games every day. “Datastream’s private connectivity configuration allowed us to easily create a private connection between our source and the destination, and ensure our data is safe and secure.”Datastream provides a simple wizard to automatically set up private, secure connectivity to your source databaseHigh throughput, low latencyWith Datastream’s serverless architecture, you don’t need to worry about provisioning, managing machines, or scaling up resources to meet fluctuations in data throughput. Datastream guarantees high performance – a single stream can process 10’s of MBs per second, while ensuring minimal latency. “We evaluated several market-leading ETL solutions”, says Dlugolecki,  “Datastream was the only tool able to successfully sync our complex, single-table datasets, doing this in weeks instead of years estimated by the other vendors.”Getting started with DatastreamYou can start streaming real-time changes from your Oracle and MySQL databases today using Datastream:Navigate to the Datastream area of your Google Cloud console, under Big Data, and click Create Stream.Choose the source database type, and see what actions you need to take to set up your source.Create your source connection profile, which can later be used for additional streams.Define how you want to connect your source.Create and configure your destination connection profile.Validate your stream and make sure the test was successful. Start the stream when you’re ready.Once the stream is started, Datastream will backfill historical data and will continuously replicate new changes as they happen. Learn more and start using Datastream todayDatastream is now generally available for Oracle and MySQL sources. Datastream supports sources both on-premises and in the cloud, and captures historical data and changes into Cloud Storage. Integrations with Cloud Data Fusion and Cloud Dataflow (our data integration and stream processing products, respectively) replicate changes to other Google Cloud destinations, including: BigQuery, Cloud Spanner, and Cloud SQL.For more information, head on over to the Datastream documentation, see our step-by-step Datastream + Dataflow to BigQuery tutorial, or start training with this Datastream Qwiklab.Related ArticleUsing Datastream to unify data for machine learning and analyticsWhile machine learning model architectures are becoming more sophisticated and effective, the availability of high-quality, fresh data fo…Read Article
Quelle: Google Cloud Platform

da Agency

How data and AI can help media companies better personalize; and what to watch out for

Media companies now have access to an ever-expanding pool of data from the digitally connected consumer. And over the past two years, as content consumption and audience behaviors have shifted in response to the world around us, direct-to-consumer has only accelerated. As media organizations pivot from third-party to first-party data, this presents challenges with the volume, velocity and fragmentation of data. It’s also an opportunity to better understand how to acquire, engage and retain audiences ⁠— and inject agility into their business amidst a competitive landscape. How should media companies be thinking about their data, and its value, to capitalize on this opportunity?  To help answer these questions, we sat down with Gloria Lee, Executive Account Director in Media & Entertainment and John Abel, Technical Director for the Office of the CTO at Google Cloud.Data and the growing importance of personalization There’s no doubt customer needs and expectations are in a constant state of flux. Across the media industry, audiences are increasingly expecting personalized content. In fact, a  PWC study conducted in 2020 found that nearly one-third (31%) of survey respondents said easy, personalized content recommendations would be a reason for staying with a streaming service.Audience engagement is the currency, and in a crowded space where attention is finite, media companies need a granular understanding of their audience. To do this, there’s an opportunity to capture and capitalize on first party data, so they can better serve their audience. Not just what audiences are consuming, but also when, where, and on which platform (and increasingly, those platforms are digital). These data points are key in understanding audiences deeply to deliver hyper-personalized experiences that audiences are expecting.    “If you look across the world today, we know that through digitalization, [that] hyper-personalization is required,” says John. “So that hyper-personalization, the volume of data and the value of the data is super critical across all industries. Media and entertainment is no different,” he adds.Enriching storytelling through AI & ML   Extracting insights on how, where and when the consumer wants to receive content will accelerate the need for data research; AI and ML will be critical to unlocking data’s full potential. “The most valuable data is generated data, typically from machine learning or AI, where you’re seeing new insights in data that give you new opportunities.” explains Gloria.  New technologies are providing insights — often in real time — about audiences, making personalization an easier task. An example use case would be recommending a new song based on a user’s listening history. This kind of personalization is just the start, as AI/ML unlocks more novel opportunities. For example, AL/ML can also be used to enrich the watching experience by finding opportune moments to integrate brands. As Gloria puts it, “Artificial intelligence, and machine learning is what enables people to quickly look through their content to find relevant moments for marketing purposes”. Getting personalization right, while making sure to keep consumer information safe and private is a challenge for all consumer companies; not just M&E. John explains “there’s a blend of how they move technology to the edge and they don’t break privacy.” Media and Entertainment companies will need to keep their data secure and private, using sophisticated practices like data federation. In this model, individual data is not exchanged.  Rather, data is first aggregated into cohorts to anonymize the individual. The goal of methods like this is to obtain useful insights while retaining privacy and security.How data is driving audience experiences Spotify is a prime example of a media company using data-led insights to provide personalized content for their customers — making it easier for users to discover new audio content and connect with their favorite artists or podcasts. “[With] Google Cloud…we can iterate quicker on key needs, like data insights and machine learning…[streamlining] our ability to concentrate on what’s important to our users and give them the experiences they know and love about Spotify.”—Tyson Singer, VP of technology and platform at SpotifySky, one of Europe’s  leading broadcasters is also transforming its data strategy to better serve their customers. By creating a scalable cloud-based architecture, Sky can keep up with increasing amounts of TV box diagnostic data on service uptime and delivery ⁠— meaning less data lost and more time to focus on improving user experience through personalization. “The data will sit right at the heart of Sky’s future strategy. It will help ensure that our products are intuitive and easy to use and that we can keep seamlessly connecting customers with the content and services they know and love,” says Oliver Tweedie, Director of Data Engineering at Sky.Transforming to a data-oriented Media companyKeeping up with new technology trends, inside and outside of the industry, will play a critical role in how media and entertainment companies can survive and thrive into the future. And without a way to centralize and draw insights from their data quickly, media organizations will struggle to stay in the race. With any type of change comes resistance. But at the end of the day, it all comes down to people. When navigating digital transformations, Gloria touches on the three categories of people: ⁠supporters, those excited about the change, those who couldn’t care less and detractors, those who are opposed to it. “It’s really tapping into the leaders for those three different groups within the company and trying to get them on board and seeing what their drivers are,” Gloria explains.So what advice do John and Gloria have for Media players looking into the data-led future?Related ArticleRead Article
Quelle: Google Cloud Platform

da Agency

Expanding our infrastructure with cloud regions around the world

Businesses and organizations around the world depend on Google Cloud to help them digitally transform, innovate across their industries, and drive operational efficiencies for long-term growth. Our global network of Google Cloud Platform regions is the foundation of this capability, delivering high-performance, low latency cloud-based services to customers and their users in more than 200 countries and territories globally. With 29 cloud regions and 88 zones, we operate more regions with multiple availability zones than any other hyperscale cloud provider. So far in 2021, we’ve opened new regions in Warsaw (Poland), Delhi NCR (India), Melbourne (Australia), and Toronto (Canada), bringing the cleanest cloud in the industry closer to more customers across multiple continents. Building on this momentum, today we are excited to share further updates to our expansion strategy.Extending our Google Cloud region roadmapChileToday, we’re proud to announce that our Santiago cloud region is now operational, ready to help more South American customers and partners build a digital-first future. This marks our first cloud region in Chile and second in South America, complementing São Paulo, which opened in 2017. The Santiago cloud region brings our high-performance, low-latency cloud services closer to customers across Latin America, from financial institutions like Caja Los Andes to health providers like Red Salud and enterprises like LATAM Airlines. Join us as we celebrate the opening of the Santiago cloud region with our customers and local Google leaders. IsraelToday, we are excited to share that our Google Cloud region in Israel will be located near Tel Aviv. When operational, the Tel Aviv region will enable us to meet growing demand for cloud services in Israel across industries, from retail to financial services to the public sector.Israeli companies like BreezoMeter, Haaretz, PayBox, and Wix already run on our cloud, making it easier to operate their businesses faster, securely, and more reliably. “Google Cloud’s global network helps Wix to achieve the best performance around the globe. For example, with Google Cloud CDN, we are able to serve tens of millions of requests per day seamlessly, while ensuring that our customers get a consistently great web experience worldwide.” – Eugene Olshenbaum, VP Technology at WixRecently, Google Cloud was selected by the Israeli government to provide public cloud services to all government entities from across the state, including ministries, authorities, and government-owned companies.GermanyOur second cloud region in Germany will be located in Berlin-Brandenburg, complementing our existing cloud region in Frankfurt. Once launched, our cloud region in Berlin-Brandenburg will strengthen our safe and secure platform for customers in Germany, including both public sector organizations and businesses like BMG, helping them scale and adapt to changing requirements.”At BMG, we’re continuously pushing digitization further to help ensure that when our artists release new music, it is promoted effectively around the world. With autoscaling via BigQuery, excellent customer support, and a clean and simple user interface, Google Cloud has been a partner to our technology team and beyond. The new Google Cloud region in Berlin-Brandenburg will further improve collaboration company-wide and make data more accessible to all teams.” – Gaurav Mittal, VP IT & Systems at BMGSaudi ArabiaLast year, we announced our plans to deploy and operate a cloud region in Saudi Arabia, while a local strategic reseller, sponsored by Aramco, will offer cloud services to businesses in the Kingdom. Today, we are announcing Dammam as the location for this cloud region. As we prepare for launch, we will start hiring out of a Riyadh-based office to support the cloud region’s deployment and operation.  United StatesOver the next year, we will add cloud regions in Columbus, Ohio, and Dallas, Texas, providing customers operating in North America with the capacity they need to run mission-critical services at the lowest possible latency. These new U.S. regions will bring our services even closer to existing customers such as J.B. Hunt Transport, Inc., which is implementing Google Cloud solutions to help create the most efficient transportation network in North America.Beyond performance and capacityAs we add regions across the Americas, Asia, Europe and the Middle East, Google Cloud is committed to continuing to help build a more sustainable future and create opportunities for everyone. We operate the cleanest cloud in the industry. Google was the first organization of its size to become carbon neutral in 2007, and we were the first major company to match 100% of our electricity consumption with renewable energy starting in 2017 and every year since then. As we now work toward achieving carbon-free energy 24/7 by 2030, we’re proud to support our customers with cloud infrastructure and tools to reduce their environmental impact. In addition to decarbonizing our energy consumption around the world, we are committed to upholding human rights in every country where we operate. This includes respecting the Universal Declaration of Human Rights, as well as the standards established in the United Nations Guiding Principles on Business and Human Rights and Global Network Initiative Principles. We’re a proud founding member of the Global Network Initiative, in which we work closely with civil society, academics, investors and industry peers to protect and advance freedom of expression and privacy globally as we deliver high-quality, relevant and useful content.Whenever we expand operations in a new country, we undertake thorough human-rights due diligence. This often includes external human-rights assessments, which identify risks that we review carefully and decide how to address. We maintain a clear position on requests from governments for access to data. We also recently announced the Trusted Cloud Principles initiative, led by Google, Amazon, Microsoft, and other technology companies, to protect the rights of customers as they move to the cloud.From the beginning, Google’s mission has been to organize the world’s information and make it universally accessible and useful. Within Google Cloud, we aim to do the same for enterprise organizations, in ways that meet international and local standards. As the global landscape continues to evolve, we are committed to collaborating with human rights organizations and the broader technology industry to uphold human rights in every country where we operate. Learn more about our human rights efforts and global cloud infrastructure.Related ArticleExpanding our global footprint with new cloud regionsGoogle Cloud is expanding its global network with new regions in South America, Europe, and Asia.Read Article
Quelle: Google Cloud Platform