Configure OpenShift Metrics with Prometheus backed by OpenShift Container Storage

Note: This scenario assumes you already have an OpenShift cluster or have followed the instructions in the Deploying OpenShift Container Storage 4 to OpenShift 4 Blog to set up an OpenShift Container Platform 4.2.14+ cluster using OpenShift Container Storage 4.

1. This Blog will cover:
What Prometheus is
Checking the current storage backend of your Prometheus environment
Make your monitoring and data persistent
2. What is Prometheus?
From the official Prometheus website:
“Prometheus was started in 2012 by Soundcloud and is an open-source monitoring and alerting toolkit. Nowadays it is a stand-alone project and independent of any single company. Due to its design, it doesn’t rely itself on a cluster or distributed database, but all its nodes are autonomous. All communication happens through HTTP and Prometheus pulls information from its’ nodes rather than receiving them like Nagios, for example.”
3. Modify your Prometheus environment
By default, Prometheus in Red Hat OpenShift Container Platform 4 is deployed on ephemeral storage so it is now time to talk about adjusting the environment to your needs. Every supported configuration change is controlled through a central ConfigMap, which needs to be created before we can make changes.
3.1. Create the ConfigMap
When you start off with a clean installation of Openshift, the ConfigMap to configure the Prometheus environment may not be present. To check if your ConfigMap is present, execute this:
oc -n openshift-monitoring get configmap cluster-monitoring-config

Output if the ConfigMap is not yet created:
Error from server (NotFound): configmaps “cluster-monitoring-config” not found

If you are missing the ConfigMap, create it:
oc -n openshift-monitoring create configmap cluster-monitoring-config

You can edit the ConfigMap with the following command. Do this now and ensure that the ConfigMap looks like below – especially the data section should be present:
oc -n openshift-monitoring edit configmap cluster-monitoring-config

ConfigMap content
apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-monitoring-config
namespace: openshift-monitoring
data:
config.yaml: |

3.2. Configuring persistent storage for the Prometheus stack
The Prometheus stack consists of the Prometheus database and the alertmanager data. Persisting the data from both is a best practice since data loss on any of these will cause you to lose your collected metrics and alerting data. View the official Openshift 4.2 documentation about this topic for further information.
While the documentation recommends using the local-storage provider, we will set up the Prometheus stack to use OpenShift Container Storage. By doing so, we will ensure that the Prometheus Pods can move freely between Nodes. Watch out for our performance briefs where we will show what this means for performance, by comparing the performance of the default EmptyDir, the recommended local-storage and OpenShift Container Storage-backed Prometheus.
To configure the Prometheus stack to use OpenShift Container Storage, edit the ConfigMap that was created in the previous section:
oc -n openshift-monitoring edit configmap cluster-monitoring-config

ConfigMap content
apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-monitoring-config
namespace: openshift-monitoring
data:
config.yaml: |
prometheusK8s:
volumeClaimTemplate:
metadata:
name: prometheusdb
spec:
storageClassName: ocs-storagecluster-ceph-rbd
resources:
requests:
storage: 40Gi
alertmanagerMain:
volumeClaimTemplate:
metadata:
name: alertmanager
spec:
storageClassName: ocs-storagecluster-ceph-rbd
resources:
requests:
storage: 40Gi

Once you save and exit the editor, the affected Pods will automatically be restarted and the new storage will be applied.

Note: It is not possible to retain data that was written on the default EmptyDir-based installation. Thus you will start with an empty database after changing the backend storage.
After a couple of minutes, the Alertmanager and Prometheus Pods will have restarted and you will see new PVCs in the openshift-monitoring namespace:

oc get -n openshift-monitoring pvc

Example output:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
alertmanager-alertmanager-main-0 Bound pvc-2f6714f7-feff-11e9-9bdd-005056818b15 40Gi RWO ocs-storagecluster-ceph-rbd 102m
alertmanager-alertmanager-main-1 Bound pvc-2f6dd091-feff-11e9-9bdd-005056818b15 40Gi RWO ocs-storagecluster-ceph-rbd 102m
alertmanager-alertmanager-main-2 Bound pvc-2f74e00d-feff-11e9-9bdd-005056818b15 40Gi RWO ocs-storagecluster-ceph-rbd 102m
prometheusdb-prometheus-k8s-0 Bound pvc-e0f7b201-ff0c-11e9-9bdd-005056818b15 40Gi RWO ocs-storagecluster-ceph-rbd 4m34s
prometheusdb-prometheus-k8s-1 Bound pvc-e101b1db-ff0c-11e9-9bdd-005056818b15 40Gi RWO ocs-storagecluster-ceph-rbd 4m34s

3.3. Configure even more
You can configure a lot more inside the cluster-monitoring-config ConfigMap. Since this Blog is focused on Storage, the other options have been omitted. A great way to learn more is to go to the official OpenShift Container Platform documentation for configuring the Prometheus Cluster Monitoring stack.
One thing you want to check out in the documentation is how you can set up the alertmanager.yml and how to define the retention time of Prometheus. By default, Prometheus only retains the last 15 days worth of data.
Resources and Feedback
To find out more about OpenShift Container Storage or to take a test drive, visit https://www.openshift.com/products/container-storage/.
If you would like to learn more about what the OpenShift Container Storage team is up to or provide feedback on any of the new 4.2 features, take this brief 3-minute survey.
The post Configure OpenShift Metrics with Prometheus backed by OpenShift Container Storage appeared first on Red Hat OpenShift Blog.
Quelle: OpenShift

OpenShift 4.3: Quay Container Security Integration

Overview
In the Red Hat OpenShift 4.2 Web UI Console, we introduced a new Cluster Overview Dashboard as the landing page when users first log in. The dashboard is there to help users resolve issues more efficiently and maintain a healthy cluster. With the latest 4.3 release, we added an image security section to the cluster health dashboard card. This section will appear on the dashboard when the Container Security Operator gets installed.
Container Security Operator
The Container Security Operator brings Quay and Clair metadata into OpenShift.

Installing this operator enables cluster administrators to monitor known container image vulnerabilities in pods running on their Kubernetes cluster.

After deployment, there is no additional configuration for the operator to start querying the container registry for vulnerability data. If the registry supports image scanning, like Quay, then the vulnerabilities found will be exposed to the user by populating the ImageManifestVuln resource list and surfacing it on the cluster dashboard.
Vulnerability Details
Users will see the new “Image Security” section on the cluster health dashboard card. The card will list the number of vulnerabilities found and will provide a link to access more detailed information on the type of vulnerabilities.

Clicking on the link opens a popover with the breakdown of vulnerability by severity. Critical and high vulnerabilities will be listed first, followed by medium and lower risk. Fixable vulnerabilities are listed with two links. One link is the vulnerability name that goes out to the vulnerability view for the Quay instance the image is hosted on (for example, Quay.io), and the other is a link to view the Image Manifest Vulnerability resource details in the affected namespace.

Only the top five most severe vulnerabilities will be listed in the popover. For cases where there are more than five, users can navigate to the custom resource definition list to view all Image Manifest Vulnerability instances under the ImageManifestVuln resource.
Learn More
We are focused on helping users resolve issues rapidly to maintain healthy clusters. The Container Security Operator can now help by exposing vulnerability information on the cluster dashboard. Be on the lookout for additional security information to be surfaced in upcoming releases.
If you’d like to learn more about what the OpenShift team is up to or provide feedback on any of the new 4.3 features, please take this brief 3-minute survey.
The post OpenShift 4.3: Quay Container Security Integration appeared first on Red Hat OpenShift Blog.
Quelle: OpenShift