Cloud Native 5 Minutes at a Time: Volumes and Persistent Storage

One of the biggest challenges for implementing cloud native technologies is learning the fundamentals—especially when you need to fit your learning into a busy schedule. In this series, we’ll break down core cloud native concepts, challenges, and best practices into short, manageable exercises and explainers, so you can learn five minutes at a time. These … Continued
Quelle: Mirantis

Microsoft DDoS protection response guide

Receiving Distributed Denial of Service (DDoS) attack threats?

DDoS threats have seen a significant rise in frequency lately, and Microsoft stopped numerous large-scale DDoS attacks last year. This guide provides an overview of what Microsoft provides at the platform level, information on recent mitigations, and best practices.

Microsoft DDoS platform

Microsoft provides robust protection against layer three (L3) and layer four (L4) DDoS attacks, which include TCP SYN, new connections, and UDP/ICMP/TCP floods.
Microsoft DDoS Protection utilizes Azure’s global deployment scale, is distributed in nature, and offers 60Tbps of global attack mitigation capacity.
All Microsoft services (including Microsoft365, Azure, and Xbox) are protected by platform level DDoS protection. Microsoft's cloud services are intentionally built to support high loads, which help to protect against application-level DDoS attacks.
All Azure public endpoint VIPs (Virtual IP Address) are guarded at platform safe thresholds. The protection extends to traffic flows inbound from the internet, outbound to the internet, and from region to region.
Microsoft uses standard detection and mitigation techniques such as SYN cookies, rate limiting, and connection limits to protect against DDoS attacks. To support automated protections, a cross-workload DDoS incident response team identifies the roles and responsibilities across teams, the criteria for escalations, and the protocols for incident handling across affected teams.
Microsoft also takes a proactive approach to DDoS defense. Botnets are a common source of command and control for conducting DDoS attacks to amplify attacks and maintain anonymity. The Microsoft Digital Crimes Unit (DCU) focuses on identifying, investigating, and disrupting malware distribution and communications infrastructure to reduce the scale and impact of botnets.

Recent incidents1

At Microsoft, despite the evolving challenges in the cyber landscape, the Azure DDoS Protection team was able to successfully mitigate some of the largest DDoS attacks ever, both in Azure and in the course of history.

Last October 2021, Microsoft reported on a 2.4 terabit per second (Tbps) DDoS attack in Azure that we successfully mitigated. Since then, we have mitigated three larger attacks.
In November 2021, Microsoft mitigated a DDoS attack with a throughput of 3.47 Tbps and a packet rate of 340 million packets per second (pps), targeting an Azure customer in Asia. As of February 2022, this is believed to be the largest attack ever reported in history. It was a distributed attack originating from approximately 10,000 sources and from multiple countries across the globe, including the United States, China, South Korea, Russia, Thailand, India, Vietnam, Iran, Indonesia, and Taiwan.

Protect your applications in Azure against DDoS attacks in three steps:

Customers can protect their Azure workloads by onboarding to Azure DDoS Protection Standard. For web workloads it is recommended to use web application firewall in conjunction with DDoS Protection Standard for extensive L3-L7 protection.

1. Evaluate risks for your Azure applications. This is the time to understand the scope of your risk from a DDoS attack if you haven’t done so already.

a. If there are virtual networks with applications exposed over the public internet, we strongly recommend enabling DDoS Protection on those virtual networks. Resources in a virtual network that requires protection against DDoS attacks are Azure Application Gateway and Azure Web Application Firewall (WAF), Azure Load Balancer, virtual machines, Bastion, Kubernetes, and Azure Firewall. Review “DDoS Protection reference architectures” to get more details on reference architectures to protect resources in virtual networks against DDoS attacks.

2. Validate your assumptions. Planning and preparation are crucial to understanding how a system will perform during a DDoS attack. You should be proactive to defend against DDoS attacks and not wait for an attack to happen and then act.

a. It is essential that you understand the normal behavior of an application and prepare to act if the application is not behaving as expected during a DDoS attack. Have monitors configured for your business-critical applications that mimic client behavior and notify you when relevant anomalies are detected. Refer to monitoring and diagnostics best practices to gain insights on the health of your application.

b. Azure Application Insights is an extensible application performance management (APM) service for web developers on multiple platforms. Use Application Insights to monitor your live web application. It automatically detects performance anomalies. It includes analytics tools to help you diagnose issues and to understand what users do with your app. It's designed to help you continuously improve performance and usability.

c. Finally, test your assumptions about how your services will respond to an attack by generating traffic against your applications to simulate DDoS attack. Don’t wait for an actual attack to happen! We have partnered with Ixia, a Keysight company, to provide a self-service traffic generator (BreakingPoint Cloud) that allows Azure DDoS Protection customers to simulate DDoS test traffic against their Azure public endpoints.

3. Configure alerts and attack analytics. Azure DDoS Protection identifies and mitigates DDoS attacks without any user intervention.

a. To get notified when there’s an active mitigation for a protected public IP, we recommend configuring an alert on the metric under DDoS attack or not. DDoS attack mitigation alerts are automatically sent to Microsoft Defender for Cloud.

b. You should also configure attack analytics to understand the scale of the attack, traffic being dropped, and other details.

Best practices to be followed

Provision enough service capacity and enable auto-scaling to absorb the initial burst of a DDoS attack.
Reduce attack surfaces; reevaluate the public endpoints and decide whether they need to be publicly accessible.
If applicable, configure Network Security Group to further lock-down surfaces.
If IIS (Internet Information Services) is used, leverage IIS Dynamic IP Address Restrictions to control traffic from malicious IPs.
Setup monitoring and alerting if you have not done so already.
Some of the counters to monitor:

TCP connection established
Web current connections
Web connection attempts

Optionally, use third-party security offerings, such as web application firewalls or inline virtual appliances, from the Azure Marketplace for additional L7 protection that is not covered via Azure DDoS Protection and Azure WAF (Azure Web Application Firewall).

When to contact Microsoft support

During a DDoS attack if you find that the performance of the protected resource is severely degraded, or the resource is not available. Review step two above on configuring monitors to detect resource availability and performance issues.
You think your resource is under DDoS attack, but DDoS Protection service is not mitigating the attack effectively.
You're planning a viral event that will significantly increase your network traffic.

For attacks that have a critical business impact, create a severity-A support ticket to engage DDoS Rapid Response team.

References

1Azure DDoS Protection—2021 Q3 and Q4 DDoS attack trends
Quelle: Azure

How Kubernetes works under the hood with Docker Desktop

Docker Desktop makes developing applications for Kubernetes easy. It provides a smooth Kubernetes setup experience by hiding the complexity of the installation and wiring with the host. Developers can focus entirely on their work rather than dealing with the Kubernetes setup details. 

This blog post covers development use cases and what happens under the hood for each one of them. We analyze how Kubernetes is set up to facilitate the deployment of applications, whether they are built locally or not, and the ease of access to deployed applications.

1. Kubernetes setup

Kubernetes can be enabled from the Kubernetes settings panel as shown below.

Checking the Enable Kubernetes box and then pressing Apply & Restart triggers the installation of a single-node Kubernetes cluster. This is all a developer needs to do.

What exactly is happening under the hood? 

Internally, the following actions are triggered in the Docker Desktop Backend and VM:

Generation of certificates and cluster configurationDownload and installation of Kubernetes internal componentsCluster bootupInstallation of additional controllers for networking and storage

The diagram below shows the interactions between the internal components of Docker Desktop for the cluster setup.

Generating cluster certs, keys and config files

Kubernetes requires certificates and keys for authenticated connections between its internal components, and with the outside. Docker Desktop takes care of generating these server and client certificates for the main internal services: kubelet (node manager), service account management, frontproxy, api server, and etcd components.

Docker Desktop installs Kubernetes using kubeadm, therefore it needs to create the kubeadm runtime and cluster-wide configuration. This includes configuration for the cluster’s network topology, certificates, control plane endpoint etc.  It uses Docker Desktop-specific naming and is not customizable by the user. The current-context, user and cluster names are always set to docker-desktop while the global endpoint of the cluster is using the DNS name https://kubernetes.docker.internal:6443. Port 6443 is the default port the Kubernetes control plane is bound to. Docker Desktop forwards this port on the host which facilitates the communication with the control plane as it would be installed directly on the host.

Download and installation of Kubernetes components 

Inside the Docker Desktop VM, a management process named Lifecycle service takes care of deploying and starting services such as Docker daemon and notifying their state change.

Once the Kubernetes certificates and configuration have been generated, a request is made to the Lifecycle service to install and start Kubernetes. The request contains the required certificates (Kubernetes PKI) for the setup.

The lifecycle service then starts pulling all the images of the Kubernetes internal components from Docker Hub. These images contain binaries such as kubelet, kubeadm, kubectl, crictl etc which are extracted and placed in `/usr/bin`.

Cluster bootup

Once these binaries are in place and the configuration files have been written to the right paths, the Lifecycle service runs `kubeadm init` to initialize the cluster and then start the kubelet process. As this is a single-node cluster setup, only one kubelet instance is being run.

The Lifecycle service then waits for the following system pods to be running in order to notify Docker Desktop host service that Kubernetes is started: coredns, kube-controller-manager and the kube-apiserver. 

Install additional controllers

Once Kubernetes internal services have started, Docker Desktop triggers the installation of additional controllers such as storage-provisioner and vpnkit-controller. Their roles concern persisting application state between reboots/upgrades and how to access applications once deployed.

Once these controllers are up and running, the Kubernetes cluster is fully operational and the Docker Dashboard is notified of its state.

We can now run kubectl commands and deploy applications.

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1m

Checking system pods at this state should return the following:

$ kubectl get pods -n kube-systemNAME                                     READY   STATUS    RESTARTS       AGEcoredns-78fcd69978-7m52k                 1/1     Running   0              99mcoredns-78fcd69978-mm22t                 1/1     Running   0              99metcd-docker-desktop                      1/1     Running   1              99mkube-apiserver-docker-desktop            1/1     Running   1              99mkube-controller-manager-docker-desktop   1/1     Running   1              99mkube-proxy-zctsm                         1/1     Running   0              99mkube-scheduler-docker-desktop            1/1     Running   1              99mstorage-provisioner                      1/1     Running   0              98mvpnkit-controller                        1/1     Running   0              98m

2. Deploying and accessing applications

Let’s take as an example a Kubernetes yaml for the deployment of docker/getting-started, the Docker Desktop tutorial. This is a generic Kubernetes yaml deployable anywhere, it does not contain any Docker Desktop-specific configuration.


apiVersion: v1
kind: Service
metadata:
name: tutorial
spec:
ports:
– name: 80-tcp
port: 80
protocol: TCP
targetPort: 80
selector:
com.docker.project: tutorial
type: LoadBalancer
status:
loadBalancer: {}


apiVersion: apps/v1
kind: Deployment
metadata:
labels:
com.docker.project: tutorial
name: tutorial
spec:
replicas: 1
selector:
matchLabels:
com.docker.project: tutorial
strategy:
type: Recreate
template:
metadata:
labels:
com.docker.project: tutorial
spec:
containers:
– image: docker/getting-started
name: tutorial
ports:
– containerPort: 80
protocol: TCP
resources: {}
restartPolicy: Always
status: {}

On the host of Docker Desktop, open a terminal and run:

$ kubectl apply -f tutorial.yaml
service/tutorial created
deployment.apps/tutorial created

Check services:

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 118m
tutorial LoadBalancer 10.98.217.243 localhost 80:31575/TCP 12m

Services of type LoadBalancer are exposed outside the Kubernetes cluster. Opening a browser and navigating to localhost:80 displays the Docker tutorial.

What needs to be noticed here is that service access is trivial as if running directly on the host. Developers do not need to concern themselves with any additional configurations. 

This is due to Docker Desktop taking care of exposing service ports on the host to make them directly accessible on it. This is done via the additional controller installed previously.

Vpnkit-controller is a port forwarding service which opens ports on the host and forwards

connections transparently to the pods inside the VM. It is being used for forwarding connections

to LoadBalancer type services deployed in Kubernetes.

3. Speed up the develop-test inner loop

We have seen how to deploy and access an application in the cluster. However, the development cycle consists of developers modifying the code of an application and testing it continuously. 

Let’s take as an example an application we are developing locally. 

$ cat main.go
package main

import (
“fmt”
“log”
“net/http”
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Println(r.URL.RawQuery)
fmt.Fprintf(w, `
## .
## ## ## ==
## ## ## ## ## ===
/”””””””””””””””””___/ ===
{ / ===-
______ O __/
__/
___________/

Hello from Docker!

`)
}
func main() {
http.HandleFunc(“/”, handler)
log.Fatal(http.ListenAndServe(“:80″, nil))
}

The Dockerfile to build and package the application as a Docker image:

$ cat Dockerfile
FROM golang:1.16 AS build

WORKDIR /compose/hello-docker
COPY main.go main.go
RUN CGO_ENABLED=0 go build -o hello main.go

FROM scratch
COPY –from=build /compose/hello-docker /usr/local/bin/hello
CMD [”/usr/local/bin/hello”]

To build the application, we run docker build as usual:

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE

$ docker build -t hellodocker .
[+] Building 0.9s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 38B 0.0s
. . .
=> => naming to docker.io/library/hellodocker 0.0s

We can see the image resulting from the build stored in the Docker engine cache.

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hellodocker latest 903fe47400c8 4 hours ago 6.13MB

But now we have a problem!

Kubernetes normally pulls images from a registry, which would mean we would have to push and pull the image we have built after every change. Docker Desktop removes this friction by using  dockershim to share the image cache between the Docker engine and Kubernetes. Dockershim is an internal component of Kubernetes that acts like a translation layer between kubelet and Docker Engine.

For development, this provides an essential advantage: Kubernetes can create containers from images stored in the Docker Engine image cache. We can build images locally and test them right away without having to push them to a registry first. 

In the kubernetes yaml from the tutorial example, update the image name to hellodocker and set the image pull policy to IfNotPresent. This ensures that the image from the local cache is going to be used.


containers:
– name: hello
image: hellodocker
ports:
– containerPort: 80
protocol: TCP
resources: {}
imagePullPolicy: IfNotPresent
restartPolicy: Always

Re-deploying applies the new updates:

$ kubectl apply -f tutorial.yaml
service/tutorial configured
deployment.apps/tutorial configured

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tutorial LoadBalancer 10.109.236.243 localhost 80:31371/TCP 4s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6h56m

$ curl localhost:80

## .
## ## ## ==
## ## ## ## ## ===
/”””””””””””””””””___/ ===
{ / ===-
______ O __/
__/
___________/

Hello from Docker!

To delete the application from the cluster run:

$ kubectl delete -f tutorial.yaml

4. Updating Kubernetes

When this is the case, the Kubernetes version can be upgraded after a Docker Desktop update. However, when a new Kubernetes version is added to Docker Desktop, the user needs to reset its current cluster in order to use the newest version.

As pods are designed to be ephemeral, deployed applications usually save state to persistent volumes. This is where the storage-provisioner helps in persisting the local storage data.

Conclusion

Docker Desktop offers a Kubernetes installation with a solid host integration aiming to work without any user intervention. Developers in need of a Kubernetes cluster without concerning themselves about its setup can simply install Docker Desktop and enable the Kubernetes cluster to have everything in place in a matter of a few minutes. 

To get Docker Desktop, follow the instructions in the Docker documentation. It also contains a dedicated guide on how to enable Kubernetes.

Join us at DockerCon 2022

DockerCon is the world’s largest development conference of its kind and it’s coming to you virtually and completely free on May 10th, 2022. DockerCon 2022 is an amazing opportunity for you and your developers to learn directly from the community, get tips, tricks, and best practices that will elevate your Docker knowledge, and to learn about what’s coming up on the Docker Roadmap. You can register for DockerCon now, pre-registration is free and open.
The post How Kubernetes works under the hood with Docker Desktop appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

EC2 Hibernation ist jetzt in den AWS-Regionen Asien-Pazifik (Jakarta) und (Osaka) verfügbar

Die EC2 Hibernation-Funktion ist jetzt in den AWS-Regionen Asien-Pazifik (Jakarta) und (Osaka) verfügbar. Mit der Hibernation-Funktion können Sie EC2-Instances starten, sie wie gewünscht einrichten und sie dann bei Bedarf anhalten und wieder fortsetzen. Ihre Instances und Anwendungen werden genau dort fortgesetzt, wo sie gestoppt wurden, anstatt ihren Speicherbedarf bei einem Kaltstart neu aufzubauen. Im Ruhezustand können Sie eine Flotte „vorgewärmter“ Instances verwalten, mit der Sie schneller einen produktiven Zustand erreichen können, ohne Ihre vorhandenen Anwendungen zu ändern. Die Hibernation-Funktion ist für Instances verfügbar, auf denen Linux- und Windows-Betriebssysteme ausgeführt werden.
Quelle: aws.amazon.com

Amazon Aurora unterstützt PostgreSQL 13.5, 12.9, 11.14 und 10.19

Nach der Ankündigung von Updates an der PostgreSQL-Datenbank von der Open-Source-Community hat AWS die Kompatibilität von Amazon Aurora PostgreSQL aktualisiert, um die PostgreSQL-Versionen 13.5, 12.9, 11.14 und 10.19 in der Region AWS GovCloud (USA) kommerziell zu unterstützen. Diese Releases enthalten Fehlerbehebungen und Verbesserungen durch die PostgreSQL-Community. 
Quelle: aws.amazon.com

Amazon MQ unterstützt jetzt RabbitMQ Version 3.8.27

Amazon MQ bietet jetzt Unterstützung für RabbitMQ Version 3.8.27. Diese Version enthält mehrere Fixes für Version 3.8.26. Amazon MQ ist ein verwalteter Message-Broker-Service für Apache ActiveMQ und RabbitMQ, der die Einrichtung und Bedienung von Message-Brokern in AWS vereinfacht. Sie können Ihren betriebliche Belastung reduzieren, indem Sie Amazon MQ verwenden, um die Bereitstellung, Einrichtung und Wartung von Message Brokern zu verwalten. Da Amazon MQ über APIs und Protokolle nach Branchenstandard eine Verbindung zu Ihren aktuellen Anwendungen herstellt, können Sie noch einfacher auf AWS migrieren, ohne den Code neu schreiben zu müssen.
Quelle: aws.amazon.com

AWS QnABot fügt Unterstützung für das Genesys Cloud-Kontaktcenter, Client-Filter und das Schwärzen von Protokollen vertraulicher Informationen hinzu

AWS-Lösungen hat den AWS QnABot aktualisiert, ein auf Amazon Lex basierender Open-Source-, Multikanal- und mehrsprachiger Konversations-Chatbot, der auf die Fragen, Antworten und das Feedback Ihrer Kunden antwortet. Ohne Programmierung ermöglicht die AWS QnABot-Lösung den Kunden die schnelle Bereitstellung von Self-Service-Konversations-KI auf mehreren Kanälen, einschließlich ihrer Kontaktcenter, Websites, Social-Media-Kanäle, SMS-Textnachrichten oder Amazon Alexa.
Quelle: aws.amazon.com