Introducing #AskGoogleCloud: A community driven YouTube live series

We’re excited to introduce a new series on the Google Cloud Tech YouTube channel connecting the cloud community directly with our Google Cloud product experts. We’ll be selecting questions that use #AskGoogleCloud on Twitter and YouTube and answer them in a featured premiere each quarter by experts in that area. Each premiere will be paired with a live chat so you can ask your questions live and get them answered with the speakers featured.Our first segment will be aired on March 12th, 2021 on the topic of serverless architectures featuring Developer Advocates Stephanie Wong, Martin Omander and James Ward. They’ll be addressing questions on the best workloads for serverless, the differences between “serverless” and “cloud native”, how to accurately estimate costs for using Cloud Run and much more. Serverless content on demandServerless ExpeditionsWe have tons of serverless content on demand on the Google Cloud Tech YouTube channel. Be sure to check out Serverless Expeditions, a series that covers all things serverless from using Python on Google Cloud with Cloud Run, to Cloud Functions vs. Cloud Run to Securing a REST API with JWT, concepts.Serverless containers with Cloud RunCheck out this video to learn how to deploy serverless containers in 3 environments using Cloud Run and Knative. In this demo, we deploy a serverless microservice that transforms word documents to PDFs.Building APIs for serverless workloads with Google CloudThis video demonstrates Google Cloud’s API Gateway—a tool that helps you create, secure, and monitor APIs for many of Google Cloud serverless backends such as Cloud Functions, Cloud Run, and more. Learn how you can provide secure access to your backend services through a well-defined REST API, providing consistency across all of your services, regardless of the service implementation.6 strategies for scaling your serverless applicationsThis video walks you through a few tips that can help you scale serverless workloads while protecting underlying resources. Learn how to configure instance scaling limits, use Cloud Tasks to limit the rate of work done, utilize stateful storage, and more. This is a great episode to understand how to improve performance and scalability for your serverless applications.Stay updated with the latest videosThe Google Cloud Tech YouTube channel has new daily videos to help you build what’s next with secure infrastructure, developer tools, APIs, data analytics and machine learning. Subscribe to get notified of our newest content!
Quelle: Google Cloud Platform

Guest Post: Calling the Docker CLI from Python with Python-on-whales

Image: Alice Lang, alicelang-creations@outlook.fr

At Docker, we are incredibly proud of our vibrant, diverse and creative community. From time to time, we feature cool contributions from the community on our blog to highlight some of the great work our community does. The following is a guest post by Docker community member Gabriel de Marmiesse. Are you working on something awesome with Docker? Send your contributions to William Quiviger (@william) on the Docker Community Slack and we might feature your work!   

The most common way to call and control Docker is by using the command line.

With the increased usage of Docker, users want to call Docker from programming languages other than shell. One popular way to use Docker from Python has been to use docker-py. This library has had so much success that even docker-compose is written in Python, and leverages docker-py.

The goal of docker-py though is not to replicate the Docker client (written in Golang), but to talk to the Docker Engine HTTP API. The Docker client is extremely complex and is hard to duplicate in another language. Because of this, a lot of features that were in the Docker client could not be available in docker-py. Sometimes users would sometimes get frustrated because docker-py did not behave exactly like the CLI.

Today, we’re presenting a new project built by Gabriel de Marmiesse from the Docker community: Python-on-whales. The goal of this project is to have a 1-to-1 mapping between the Docker CLI and the Python library. We do this by communicating with the Docker CLI instead of calling directly the Docker Engine HTTP API.

If you need to call the Docker command line, use Python-on-whales. And if you need to call the Docker engine directly, use docker-py.

In this post, we’ll take a look at some of the features that are not available in docker-py but are available in Python-on-whales:

Building with Docker buildxDeploying to Swarm with docker stackDeploying to the local Engine with Compose

Start by downloading Python-on-whales with 

pip install python-on-whales

and you’re ready to rock!

Docker Buildx0

Here we build a Docker image. Python-on-whales uses buildx by default and gives you the output in real time.

>>> from python_on_whales import docker
>>> my_image = docker.build(“.”, tags=”some_name”)
[+] Building 1.6s (17/17) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.6 1.4s
=> [python_dependencies 1/5] FROM docker.io/library/python:3.6@sha256:293 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 72.86kB 0.0s
=> CACHED [python_dependencies 2/5] RUN pip install typeguard pydantic re 0.0s
=> CACHED [python_dependencies 3/5] COPY tests/test-requirements.txt /tmp 0.0s
=> CACHED [python_dependencies 4/5] COPY requirements.txt /tmp/ 0.0s
=> CACHED [python_dependencies 5/5] RUN pip install -r /tmp/test-requirem 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 1/7] RUN apt-get update && 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 2/7] RUN curl -fsSL https: 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 3/7] RUN add-apt-repositor 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 4/7] RUN apt-get update & 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 5/7] WORKDIR /python-on-wh 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 6/7] COPY . . 0.0s
=> CACHED [tests_ubuntu_install_without_buildx 7/7] RUN pip install -e . 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:e1c2382d515b097ebdac4ed189012ca3b34ab6be65ba0c 0.0s
=> => naming to docker.io/library/some_image_name

Docker Stacks

Here we deploy a simple Swarmpit stack on a local Swarm. You get a Stack object that has several methods: remove(), services(), ps().

>>> from python_on_whales import docker
>>> docker.swarm.init()
>>> swarmpit_stack = docker.stack.deploy(“swarmpit”, compose_files=[”./docker-compose.yml”])
Creating network swarmpit_net
Creating service swarmpit_influxdb
Creating service swarmpit_agent
Creating service swarmpit_app
Creating service swarmpit_db
>>> swarmpit_stack.services()
[<python_on_whales.components.service.Service object at 0x7f9be5058d60>,
<python_on_whales.components.service.Service object at 0x7f9be506d0d0>,
<python_on_whales.components.service.Service object at 0x7f9be506d400>,
<python_on_whales.components.service.Service object at 0x7f9be506d730>]
>>> swarmpit_stack.remove()

Docker Compose

Here we show how we can run a Docker Compose application with Python-on-whales. Note that, behind the scenes, it uses the new version of Compose written in Golang. This version of Compose is still experimental. Take appropriate precautions.

$ git clone https://github.com/dockersamples/example-voting-app.git
$ cd example-voting-app
$ python
>>> from python_on_whales import docker
>>> docker.compose.up(detach=True)
Network “example-voting-app_back-tier” Creating
Network “example-voting-app_back-tier” Created
Network “example-voting-app_front-tier” Creating
Network “example-voting-app_front-tier” Created
example-voting-app_redis_1 Creating
example-voting-app_db_1 Creating
example-voting-app_db_1 Created
example-voting-app_result_1 Creating
example-voting-app_redis_1 Created
example-voting-app_worker_1 Creating
example-voting-app_vote_1 Creating
example-voting-app_worker_1 Created
example-voting-app_result_1 Created
example-voting-app_vote_1 Created
>>> for container in docker.compose.ps():
… print(container.name, container.state.status)
example-voting-app_vote_1 running
example-voting-app_worker_1 running
example-voting-app_result_1 running
example-voting-app_redis_1 running
example-voting-app_db_1 running
>>> docker.compose.down()
>>> print(docker.compose.ps())
[]

Bonus section: Docker objects attributes as Python attributes

All information that you can access with docker inspect is available as Python attributes:

>>> from python_on_whales import docker
>>> my_container = docker.run(“ubuntu”, [”sleep”, “infinity”], detach=True)
>>> my_container.state.started_at
datetime.datetime(2021, 2, 18, 13, 55, 44, 358235, tzinfo=datetime.timezone.utc)
>>> my_container.state.running
True
>>> my_container.kill()
>>> my_container.remove()

>>> my_image = docker.image.inspect(“ubuntu”)
>>> print(my_image.config.cmd)
[’/bin/bash’]

What’s next for Python-on-whales ?

We’re currently improving the integration of Python-on-whales with the new Compose in the Docker CLI (currently beta).

You can consider that Python-on-whales is in beta. Some small API changes are still possible. 

We encourage the community to try it out and give feedback in the issues!

To learn more about Python-on-whales:

DocumentationGithub repository
The post Guest Post: Calling the Docker CLI from Python with Python-on-whales appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

AWS Glue DataBrew optimiert sein Dashboard zur Datenqualität mit einer visuellen Vergleichsmatrix

Wenn Sie Datenqualitätsprofile zu Ihren Datensätzen generieren, veröffentlicht DataBrew jetzt ein visuelles Dashboard mit über 40 Statistiken und Visualisierungen im Tabellenfornat in der AWS Glue DataBrew-Konsole, um einfache Vergleiche zu ermöglichen. Das Verständnis der Datenqualität ist der Schlüssel zum Erfolg Ihrer Analyse- und Machine Learning-Projekte. Mit dieser neuen Funktion in DataBrew ist es leicht möglich, in Datensätzen mit wenigen Tausend bis zu mehreren Millionen Zeilen und in unterschiedlichen Dateiformaten Anomalien in Datenverteilungen und Ausreißer zu erkennen, Versatz zu verstehen  
Quelle: aws.amazon.com