Cloud Computing Köln - Seite 1807 von 7139 - Neues zu Cloud Computing, Internet of Things und Technologien

Editor’s note: The post is part of a series highlighting our awesome partners, and their solutions, that are Built with BigQuery.“True Fit is in a unique position where we help shoppers find the right sizes of apparel and footwear through near real time, adaptable ML models thereby rendering a great shopping experience while also enabling retailers with higher conversion, revenue potential and more importantly access to actionable key business metrics. Our ability to process volumes of data, adapt our core ML models, reduce complex & slower ecosystems, exchange data easily with our customers etc, have been propelled multi-fold via BigQuery and Analytics Hub” – says Raj Chandrasekaran, CTO, True FitTrue Fit has built the world’s largest dataset for apparel and footwear retailers over the last twelve years, connecting purchasing and preference information for more than 80 million active shoppers to data for 17,000 brands across its global network of retail partners. This dataset powers fit personalization for shoppers as they buy apparel and footwear online and connects retailers with powerful data and insight to inform marketing, merchandising, product development and ecommerce strategy.Gathering data, correlating and analyzing it are the underlying foundation to making smart business decisions to grow and scale a retailer’s business. This is especially important for retailers that are utilizing data packages to target which consumers to market their brands and products to. Deriving meaningful insights from data has become a larger focus for retailers as the market grows digitally, competition for share of wallet increases and consumer expectations for more personalized shopping experiences continues to rise.But, how do companies share datasets regardless of the magnitude amongst each other in a scalable manner by optimizing infrastructure costs and securely sharing data? How can companies access and use the data into their own environment without needing a complicated / time consuming process to physically move the data? How would a company know how to utilize this data to suit their own business needs? True Fit partnered with Google and the Built with BigQuery initiative to solve these questions.Google Cloud services such as BigQuery and Analytics Hub have become vital to how True Fit optimizes the entire lifecycle of data from ingestion to distribution of data packages with its retail partners. BigQuery is a fully managed, serverless and limitless scale data warehousing solution with tighter integration with several Google Cloud products. Analytics Hub, powered by BigQuery, allows easy creation of data exchanges for producers and simplifies the discovery and consumption of the data for the consumers. Data shared via the exchanges can further be enriched with more datasets available in the Analytics Hub marketplace.Using the above diagram, let us take a look at how the process works across different stages:Event Ingestion – True Fit leverages Cloud Logging with Fluentd to stream logging events into BigQuery as datasets. BigQuery’s unique capability in real time streaming allows for real time debugging and analysis of all activity across the True Fit ecosystem.Denormalization – Scheduled queries are set up to take the normalized data in the event logs and convert them into denormalized core tables. These tables are easy to query and contain all information needed to assist BI analysts and data scientists with their research without the need for complicated table joins.Aggregations – Aggregations are created and updated on the fly as data is ingested using a mix of scheduled queries and direct BigQuery table updates. Reports are always fresh and can be delivered without ever having to worry about stale data.Alerting – Alerts are set up all across the True Fit architecture which leverage the real-time aggregations. These alerts not only inform True Fit when there are data discrepancies or missing data but have also been configured to inform our partners when the data they provide contains errors. For example, True Fit will notify a retailer if the product catalog provided drops below specific thresholds we’ve previously seen from them. Alerts range from anything like an email, SMS message, or even a real-time toast message in a True Fit UI that a retailer is using to provide their data.Secure Distribution – Exchange’s are created in the Analytics Hub. The datasets are published as one or more listings into the Exchange. Partners subscribe to the listing as a linked dataset to get instant access to data and act upon it accordingly. This unlocks use cases that range from everywhere from marketing; to shopper chat bots; and even real-time product recommendations based on a shopper’s browsing behavior. Analytics Hub allows True Fit to expose only the data they intend to share to specific partners using simple to understand IAM roles. Adding the built-in Analytics Hub Subscriber role to a partner’s service account on a specific listing of dataset created just for them makes it so that they are the only one to get access to that data. Gone are the days of dealing with API keys or credential management!True Fit’s original data lake was built using Apache Hive prior to switching to BigQuery. At roughly 450TiB, extracting data from this data lake became quite a challenge to do in real-time. It took approximately 24 hours before data packages would become available to core retail partners which impacted our ability to produce reports and data packages at scale. Even after the data packages were available, partners had difficulty downloading these data packages and importing them into their own data warehouses to utilize due to the size and formats. The usefulness of the data packages would occasionally get put into question due to the data becoming stale and it was difficult to alert on any data discrepancies because of the time delay before these data packages would be available.BigQuery has allowed True Fit to produce these same data packages in real time as events occur; unlocking new marketing opportunities. Retail partners have also praised how easily consumable Analytics Hub has made the process because the data “just appears” alongside their existing data warehouse as linked datasets. True Fit publishes a number of BigQuery data packages for its retail partners via Analytics Hub which allows them to generate personalized onsite and email campaigns for their own shoppers in a manner far beyond the capabilities not available in the past. Below are just a sample of ways in which True Fit partners personalize their campaigns utilizing the True Fit data packages. Partners have the ability to:Find the True Fit shoppers of a desired category near real-time who’ve been browsing extra specific products in the last couple weeksEnhance their understanding of their shopper demographic data and category affinitiesRetrieve size and fit recommendations for specific in-stock products for a provided set of shoppers or have True Fit determine what the ideal set of shoppers for those products would beMatch their in-stock, limited run styles and sizes to applicable True Fit shoppersEnhance emails and on-site triggers based on products the shopper has recently viewed or purchased across the True Fit networkIf you’re a retailer looking to unlock your own growth using real-world data in real-time, be sure to check out the data packages offered by True Fit!To learn more about True Fit on Google Cloud, visit https://www.truefit.com/businessThe Built with BigQuery advantage for ISVs Through the Built with BigQuery Program launched in April ‘22 as part of the Google Data Cloud Summit, Google is helping data-driven companies like True Fit build innovative applications on Google’s data cloud with simplified access to technology, helpful and dedicated engineering support, and joint go-to-market programs. Participating companies can: Get started fast with a Google-funded, pre-configured sandbox. Accelerate product design and architecture through access to designated experts from the ISV Center of Excellence who can provide insight into key use cases, architectural patterns, and best practices. Amplify success with joint marketing programs to drive awareness, generate demand, and increase adoption.BigQuery gives ISVs the advantage of a powerful, highly scalable data warehouse that’s integrated with Google Cloud’s open, secure, sustainable platform. And with a huge partner ecosystem and support for multi-cloud, open source tools, and APIs, Google provides technology companies the portability and extensibility they need to avoid data lock-in.Click here to learn more about Built with BigQuery.We thank the many Google Cloud and True Fit team members who contributed to this ongoing collaboration and review, especially Raj Chandrasekaran, CTO True Fit and Sujit Khasnis, Cloud Partner Engineering
Quelle: Google Cloud Platform

3. November 2022

da Agency

Sharing the latest improvements to efficiency in Microsoft’s datacenters

In April, I published a blog that explained how we define and measure energy and water use at our datacenters, and how we are committed to continuous improvements.

Now, in the lead up to COP27, the global climate conference to be held in Egypt, I am pleased to provide a number of updates on how we’re progressing in making our datacenters more efficient across areas such as waste, renewables, and ecosystems. You can also visit Azure Sustainability—Sustainable Technologies | Microsoft Azure to explore this further.

Localized fact sheets in 28 regions

To share important information about the impact of our datacenters regionally with our customers, we have published localized fact sheets in 28 regions across the globe. These fact sheets provide a wide range of information and details about many different aspects of our datacenters and their operations.

A review of PUE (Power Usage Effectiveness) and WUE (Water Usage Effectiveness)

PUE is an industry metric that measures how efficiently a datacenter consumes and uses the energy that powers a datacenter, including the operation of systems like powering, cooling, and operating the servers, data networks and lights. The closer the PUE number is to “1,” the more efficient the use of energy.
While local environment and infrastructure can affect how PUE is calculated, there are also slight variations across providers.

Here is the simplest way to think about PUE:

WUE is another key metric relating to the efficient and sustainable operations of our datacenters and is a crucial aspect as we work towards our commitment to be water positive by 2030. WUE is calculated by dividing the number of liters of water used for humidification and cooling by the total annual amount of power (measured in kWh) needed to operate our datacenter IT equipment.

In addition to PUE and WUE, below are key highlights across carbon, water, and waste initiatives at our datacenters.

Datacenter efficiency in North and South America

As I illustrated in April, our newest generation of datacenters have a design PUE of 1.12; this includes our Chile datacenter that is under construction. We are constantly focused on improving our energy efficiency, for example in California, our San Jose datacenters will be cooled with an indirect evaporative cooling system using reclaimed water all year and zero fresh water. Because the new datacenter facilities will be cooled with reclaimed water, they will have a WUE of 0.00 L/kWh in terms of freshwater usage.

In addition, as we continue our journey to achieve zero waste by 2030, we are proud of the progress we are making with our Microsoft Circular Centers. These centers sit adjacent to a Microsoft datacenter and process decommissioned cloud servers and hardware. Our teams sort and intelligently channel the components and equipment to optimize, reuse or repurpose.

In October, we launched a Circular Center in Chicago, Illinois that has the potential capacity to process up to 12,000 servers per month for reuse, diverting up to 144,000 servers annually. We plan to open a Circular Center in Washington state early next year and have plans for Circular Centers in Texas, Iowa, and Arizona to further optimize our supply chain and reduce waste.

Furthermore, our team has successfully completed an important water reuse project at one of our datacenters. This treatment facility, the first of its kind in Washington state and over 10 years in the making, will process water for reuse by local industries, including datacenters, decreasing the need for potable water for datacenter cooling.

Innovative solutions in Europe, the Middle East, and Africa

This winter Europeans face the possibility of an energy crisis, and we have made a number of investments in optimizing energy efficiency in our datacenters to ensure that we are operating our facilities as effectively as possible. Datacenters are the backbone of modern society and as such it is important that we continue to provide critical services to the industries that need us most in a way that constantly mitigates energy consumption.

Across our datacenters in EMEA, we have made steady progress across carbon, waste, water, and ecosystems. We are committed to shifting to 100 percent renewable energy supply by 2025, meaning that we will have power purchase agreements for green energy contracted for 100 percent of carbon emitting electricity consumed by all our data centers, buildings, and campuses. This will add additional gigawatts of renewable energy to the grid, increasing energy capacity. With that said we have added more than 5 gigawatts of renewable energy to the grid globally, this has culminated in more than 15 individuals deals in Europe spanning Ireland, Denmark, Sweden, and Spain.

In Finland, we recently announced an important heat reuse project that will take excess heat from our datacenters and transfer that heat to the local districts’ heating systems that can be used for both domestic and commercial purposes.

To reduce waste from our datacenters in EMEA, the Circular Center we opened in Amsterdam in 2020, which has since already delivered an 83 percent reuse of end-of-life datacenter assets and components. This is progress towards our target of 90 percent reuse and recycling of all servers and components for all cloud hardware by 2025. In addition, in January 2022, we opened a Circular Center in Dublin, Ireland, and have plans to open another Circular Center in Sweden to serve the region.

As we continue to seek out efficiencies in our operations, recently we turned to nature for inspiration, to understand how much of the natural ecosystem we could replenish on the site of a datacenter, essentially integrating the datacenter into nature with the goal of renewing and revitalizing the surrounding area so that we can restore and create a pathway to provide regenerative value for the local community and environment. In the Netherlands we have begun construction of a lowland forested area around the datacenter as well as forested wetland. This was done to support the growth of native plants to mirror a healthy, resilient ecosystem and support biodiversity, improve storm water control and prevent erosion.

Updates in Asia Pacific

Finally, I’d like to highlight some of the sustainability investments we have made across Asia Pacific. In June 2022, we launched our Singapore Circular Center that is capable of processing up to 3,000 servers per month for reuse, or 36,000 servers annually. We have plans to open additional Circular Centers in Australia and South Korea in fiscal year 2025 and beyond. Across our datacenters in APAC, we have formed partnerships with local energy providers for renewable energy that is sourced from wind, solar, and hydro power and we have plans to further these partnerships and investments in renewable energy. In our forthcoming datacenter region in New Zealand, we have signed an agreement that will enable Microsoft to power all of its datacenters with 100 percent renewable energy from the day it opens.

Innovating to design the hyperscale datacenter of the future

What these examples from across our global datacenter portfolio show is our ongoing commitment to make our global Microsoft datacenters more sustainable and efficient, enabling our customers to do more with less.

Our objective moving forward is to continue providing transparency across the entire datacenter lifecycle about how we infuse principles of reliability, sustainability, and innovation at each step of the datacenter design, construction, and operations process.

Design: How do we ensure we design for reliability, efficiency, and sustainability, to help reduce our customers' scope three emissions?
Construction: How do we reduce embodied carbon and create a reliable supply chain?
Operation: How do we infuse innovative green technologies to decarbonize and operate to the efficient design standards?
Decommissioning: How do we recycle and reuse materials in our datacenters?
Community: How do we partner with the community and operate as good neighbors?

We have started by sharing datacenter region-specific data around carbon, water, waste, ecosystems, and community development and we will continue to provide updates as Microsoft makes further investments globally.

Learn more

You can learn more about our global datacenter footprint across the 60+ datacenter regions by visiting datacenters.microsoft.com.

View our Microsoft datacenter factsheets.
Learn more about Azure sustainability.
Discover new sustainability guidance in the Azure Well-Architected Framework.
Take a virtual tour of Microsoft’s datacenters.

Quelle: Azure

3. November 2022

da Agency

Build a globally resilient architecture with Azure Load Balancer

Azure Load Balancer’s global tier is a cloud-native global network load balancing solution. With cross-region Load Balancer, customers can distribute traffic across multiple Azure regions with ultra-low latency and high performance. To better understand the use case of Azure’s cross-region Load Balancer, let’s dive deeper into a customer scenario. In this blog, we’ll learn about a customer, their use case, and how Azure Load Balancer came to the rescue.

Who can benefit from Azure Load Balancer?

This example customer is a software vendor in the automotive industry. Their current product offerings are cloud–based software, focused on helping vehicle dealerships manage all aspects of their business including sales leads, vehicles, and customer accounts. While it is a global company, most of its business is done in Europe, the United Kingdom (UK), and Asia Pacific regions. To support its global business, the customer utilizes a wide range of Azure services including virtual machines (VMs), a variety of platform as a service (PaaS) solutions, Load Balancer, and MySQL to help meet an ever-growing demand.

What are the current global load balancing solutions?

The customer is using domain name service (DNS)–based traffic distribution to direct traffic to multiple Azure regions. At each Azure region, they deploy regional Azure Load Balancers to distribute traffic across a set of virtual machines. However, if a region went down, they experienced downtime due to DNS caching. Although minimal, this was not a risk they could continue to take on as their business expanded globally.

What are the problems with the current solutions?

Since the customer’s solution is global, as traffic increased, they noticed high latency when requesting information from their endpoints across regions. For example, users located in Africa noticed high latency when they tried to request information. Often their requests were routed to an Azure region on another continent, which caused the high latency. Answering requests with low latency is a critical business requirement to ensure business continuity. As a result, they needed a solution that withstood regional failover, while simultaneously providing ultra-low latency with high performance.

How did Azure’s cross-region Load Balancer help?

Given that low latency is a requirement for the customer, a global layer 4 load balancer was a perfect solution to the problem. The customer deployed Azure’s cross-region Load Balancer, giving them a single unique globally anycast IP to load balance across their regional offices. With Azure’s cross-region Load Balancer, traffic is distributed to the closest region, ensuring low latency when using the service. For example, if a customer connected from Asia Pacific regions, traffic is automatically routed to the closest region, in this case Southeast Asia. The customer was able to add all their regional load balancers to the backend of the cross-region Load Balancer and thus improved latency without any additional downtime. Before the update was deployed across all regions, the customer verified that their metrics for data path availability and health probe status are 100 percent on both its cross-region Load Balancer and each regional Load Balancer.

After deploying cross-region Load Balancer, traffic is now distributed with ultra-low latency across regions. Since the cross-region Load Balancer is a network load balancer, only the TCP/UDP headers are quickly inspected instead of the entire packet. In addition, the cross-region Load Balancer will send traffic to the closest participating Azure region to a client. These benefits are seen by the customer who now sees traffic being served with lower latency than before.

Learn More

Visit the Cross-region load balancer overview to learn more about Azure’s cross-region Load Balancer and how it can fit into your architecture.
Quelle: Azure

3. November 2022

da Agency

Developing Go Apps With Docker

Go (or Golang) is one of the most loved and wanted programming languages, according to Stack Overflow’s 2022 Developer Survey. Thanks to its smaller binary sizes vs. many other languages, developers often use Go for containerized application development.

Mohammad Quanit explored the connection between Docker and Go during his Community All-Hands session. Mohammad shared how to Dockerize a basic Go application while exploring each core component involved in the process:

Follow along as we dive into these containerization steps. We’ll explore using a Go application with an HTTP web server — plus key best practices, optimization tips, and ways to bolster security.

Go application components

Creating a full-fledged Go application requires you to create some Go-specific components. These are essential to many Go projects, and the containerization process relies equally heavily on them. Let’s take a closer look at those now.

Using main.go and go.mod

Mohammad mainly highlights the main.go file since you can’t run an app without executable code. In Mohammad’s case, he created a simple web server with two unique routes: an I/O format with print functionality, and one that returns the current time.

What’s nice about Mohammad’s example is that it isn’t too lengthy or complex. You can emulate this while creating your own web server or use it as a stepping stone for more customization.

Note: You might also use a package main in place of a main.go file. You don’t explicitly need main.go specified for a web server — since you can name the file anything you want — but you do need a func main () defined within your code. This exists in our sample above.

We always recommend confirming that your code works as expected. Enter the command go run main.go to spin up your application. You can alternatively replace main.go with your file’s specific name. Then, open your browser and visit http://localhost:8081 to view your “Hello World” message or equivalent. Since we have two routes, navigating to http://localhost:8081/time displays the current time thanks to Mohammad’s second function.

Next, we have the go.mod file. You’ll use this as a root file for your Go packages, module path for imports (shown above), and for dependency requirements. Go modules also help you choose a directory for your project code.

With these two pieces in place, you’re ready to create your Dockerfile!

Creating your Dockerfile

Building and deploying your Dockerized Go application means starting with a software image. While you can pull this directly from Docker Hub (using the CLI), beginning with a Dockerfile gives you more configuration flexibility.

You can create this file within your favorite editor, like VS Code. We recommend VS Code since it supports the official Docker extension. This extension supports debugging, autocompletion, and easy project file navigation.

Choosing a base image and including your application code is pretty straightforward. Since Mohammad is using Go, he kicked off his Dockerfile by specifying the golang Docker Official Image as a parent image. Docker will build your final container image from this.

You can choose whatever version you’d like, but a pinned version like golang:1.19.2-bullseye is both stable and slim. Newer image versions like these are also safe from October 2022’s Text4Shell vulnerability.

You’ll also need to do the following within your Dockerfile:

Include an app directory for your source codeCopy everything from the root directory into your app directoryCopy your Go files into your app directory and install dependenciesBuild your app with configurationTell your Docker container to listen on a certain port at runtimeDefine an executable command that runs once your container starts

With these points in mind, here’s how Mohammad structured his basic Dockerfile:

# Specifies a parent image
FROM golang:1.19.2-bullseye

# Creates an app directory to hold your app’s source code
WORKDIR /app

# Copies everything from your root directory into /app
COPY . .

# Installs Go dependencies
RUN go mod download

# Builds your app with optional configuration
RUN go build -o /godocker

# Tells Docker which network port your container listens on
EXPOSE 8080

# Specifies the executable command that runs when the container starts
CMD [ “/godocker” ]

From here, you can run a quick CLI command to build your image from this file:

docker build –rm -t [YOUR IMAGE NAME]:alpha .

This creates an image while removing any intermediate containers created with each image layer (or step) throughout the build process. You’re also tagging your image with a name for easier reference later on.

Confirm that Docker built your image successfully by running the docker image ls command:

If you’ve already pulled or built images in the past and kept them, they’ll also appear in your CLI output. However, you can see Mohammad’s go-docker image listed at the top since it’s the most recent.

Making changes for production workloads

What if you want to account for code or dependency changes that’ll inevitably occur with a production Go application? You’ll need to tweak your original Dockerfile and add some instructions, according to Mohammad, so that changes are visible and the build process succeeds:

FROM golang:1.19.2-bullseye

WORKDIR /app

# Effectively tracks changes within your go.mod file
COPY go.mod .

RUN go mod download

# Copies your source code into the app directory
COPY main.go .

RUN go mod -o /godocker

EXPOSE 8080

CMD [ “/godocker” ]

After making those changes, you’ll want to run the same docker build and docker image ls commands. Now, it’s time to run your new image! Enter the following command to start a container from your image:

docker run -d -p 8080:8081 –name go-docker-app [YOUR IMAGE NAME]:alpha

Confirm that this worked by entering the docker ps command, which generates a list of your containers. If you have Docker Desktop installed, you can also visit the Containers tab from the Docker Dashboard and locate your new container in the list. This also applies to your image builds — instead using the Images tab.

Congratulations! By tracing Mohammad’s steps, you’ve successfully containerized a functioning Go application.

Best practices and optimizations

While our Go application gets the job done, Mohammad’s final image is pretty large at 913MB. The client (or end user) shouldn’t have to download such a hefty file.

Mohammad recommends using a multi-stage build to only copy forward the components you need between image layers. Although we start with a golang:version as a builder image, defining a second build stage and choosing a slim alternative like alpine helps reduce image size. You can watch his step-by-step approach to tackling this.

This is beneficial and common across numerous use cases. However, you can take things a step further by using FROM scratch in your multi-stage builds. This empty file is the smallest we offer and accepts static binaries as executables — making it perfect for Go application development.

You can learn more about our scratch image on Docker Hub. Despite being on Hub, you can only add scratch directly into your Dockerfile instead of pulling it.

Develop your Go application today

Mohammad Quanit outlined some user-friendly development workflows that can benefit both newer and experienced Go users. By following his steps and best practices, it’s possible to create cross-platform Go apps that are slim and performant. Docker and Go inherently mesh well together, and we also encourage you to explore what’s possible through containerization.

Want to learn more?

Check out our Go language-specific guide.Read about the golang Docker Official Image.See Go in action alongside other technologies in our Awesome Compose repo.Dig deeper into Dockerfile fundamentals and best practices.Understand how to use Go-based server technologies like Caddy 2.
Quelle: https://blog.docker.com/feed/