How InterSystems Builds an Enterprise Database at Scale with Docker Enterprise

We sat down recently with InterSystems, our partner and customer, to talk about how they deliver an enterprise database at scale to their customers. InterSystems’s software powers mission-critical applications at hospitals, banks, government agencies and other organizations.
We spoke with Joe Carroll, Product Specialist, and Todd Winey, Director of Partner Programs at InterSystems about how containerization and Docker are helping transform their business.
Here’s what they told us. You can also catch the highlights in this 2 minute video:

On InterSystems and Enterprise Databases…
Joe Carroll: InterSystems is a 41 year old database and data platform company. We’ve been in data storage for a very long time and our customers tend to be traditional enterprises — healthcare, finance, shipping and logistics as well as government agencies. Anywhere that there’s mission critical data we tend to be around. Our customers have really important systems that impact people’s lives, and the mission critical nature of that data characterizes who our customers are and who we are.
On Digital Transformation in Established Industries…
Todd Winey: Many of those organizations and industries have been traditionally seen as laggards in terms of their technology adoption in the past, so the speed with which they’re moving to digital transformation is a key theme for everyone involved. Our customers are really seeing the benefits of being able to adopt technology faster and with a higher degree of confidence.
Our goal is to support them on that journey with our software. To do that, we’ve had to transform our business at InterSystems.
Why Docker…
Todd: From a software delivery standpoint, Docker has provided some truly amazing capabilities. Quality development was probably one of the biggest bottlenecks we faced, along with creating the processes we needed to ensure good quality software was going out the door.
And so rather than solve it by trying to throw more people at the problem, Docker’s platform allows us to do a lot of automation in that process so that we’re getting orders of magnitude improvements without additional staff.
On Scaling Software Testing for an Enterprise Database…
Joe: The InterSystems IRIS platform is the latest iteration of our database platform, and is built container and cloud first. We modernized the database with Docker Enterprise is our database software. 
The scalability the Docker Enterprise gives us in terms of our testing infrastructure allows us to go from what was before a tens of tests a day to thousands of tests every night, to eventually tens of thousands of tests every night. We’re able to provide higher quality of software four our customers because of this testing infrastructure. 
We’re able to test at scale, and do that quickly without sacrificing quality. And when we kick off our testing suite we can be confident the software will run on whatever cloud provider we need, so we can ensure portability of our application. 
On How Docker has Helped…
Todd: Adopting Docker Enterprise really provided three key benefits. We’re getting quarterly releases out the door whereas before we’re we’re looking at maybe one major release, and really starting to push the envelope on continuous delivery. So our customers who are ready to take software with a new build on a daily basis, we can meet those demands.
Staffing of the software quality development process is much easier. We’re able to do more with less, which is what every business wants and what every business wants out a digital transformation.
We’ve always delivered our software across a large number of operating systems. Docker allows us to treat cloud as one more operating system, so we can provide flexibility to our customers to run our software confidently where they want.
The Future for InterSystems
InterSystems expects to continue improving its software release cycle, delivering software updates daily to its enterprise database.
To learn more about InterSystems and how Docker helps enterprises build better software:

Watch the InterSystems DockerCon 2019 session
See what’s new in Docker Enterprise 3.0

How @InterSystems builds their enterprise database software at scale with Docker EnterpriseClick To Tweet

The post How InterSystems Builds an Enterprise Database at Scale with Docker Enterprise appeared first on Docker Blog.
Quelle: https://blog.docker.com/feed/

Kubeflow + OpenShift Container Platform + Dell EMC Hardware: A Complete Machine Learning Stack

Kubeflow is an open source machine learning toolkit for Kubernetes. It bundles popular ML/DL frameworks such as TensorFlow, MXNet, Pytorch, and Katib with a single deployment binary. By running Kubeflow on Red Hat OpenShift Container Platform, you can quickly operationalize a robust machine learning pipeline. However, the software stack is only part of the picture. You also need high performance servers, storage, and accelerators to deliver the stack’s full capability. To that end, Dell EMC and Red Hat’s Artificial Intelligence Center of Excellence recently collaborated on two white papers about sizing hardware for Kubeflow on OpenShift.
The first whitepaper is called “Machine Learning Using the Dell EMC Ready Architecture for Red Hat OpenShift Container  Platform.”  It describes how to deploy Kubeflow 0.5 and OpenShift Container Platform 3.11 on Dell PowerEdge servers. The paper builds on Dell’s Ready Architecture for OpenShift Container Platform 3.11 — a prescriptive architecture for running OpenShift Container Platform on Dell hardware. It includes a bill of materials for ordering the exact servers, storage and switches used in the architecture. The machine learning whitepaper extends the ready architecture to include workload-specific recommendations and settings. It also includes instructions for configuring OpenShift and validating Kubeflow with a distributed TensorFlow training job.
Kubeflow is developed on upstream Kubernetes, which lacks many of the security features enabled in OpenShift Container Platform by default. Several of OpenShift Container Platform default security controls are relaxed in this whitepaper to get Kubeflow up and running. Additional steps might be required to meet your organization’s security standards for running Kubeflow on OpenShift Container Platform in production. These steps may include defining cluster roles for the Kubeflow services with appropriate permissions, adding finalizers to Kubeflow resources for reconciliation, and/or creating liveness probes for Kubeflow pods.
The second whitepaper is called “Executing ML/DL Workloads Using Red Hat OpenShift Container Platform v3.11.” It explains how to leverage Nvidia GPUs with Kubeflow for best performance on inferencing and training jobs. The hardware profile used in this whitepaper is similar to the ready architecture used in the first paper except the servers are outfitted with Nvidia Tesla GPUs. The architecture uses two GPU models. The OpenShift worker nodes have Nvidia Tesla T4 GPUs. Based on the Turing architecture, the T4s deliver excellent inference performance in a 70-Watt power profile. The storage nodes have Nvidia Tesla V100 GPUs. The V100 is a state of the art data center GPU. Based on the Volta architecture, the V100s are deep learning workhorses for both training and inference. 

The researchers compared the GPU models when training the Resnet50 TensorFlow benchmark. This is shown in the figure above. Not surprisingly, the Tesla V100s outperformed the T4s when training. They have double the compute capability — both in terms of FP64 and TensorCores — along with higher memory bandwidth due to the HBM2 memory subsystem. But the T4s should give better performance per Watt than the V100s when running less floating-point intensive tasks, particularly inferencing in mixed precision.
These whitepapers make it easier for you to select hardware for running Kubeflow on premises. Dell and Red Hat are continuing to collaborate on updating these documents to the latest version of Kubeflow and OpenShift Container Platform 4.
The post Kubeflow + OpenShift Container Platform + Dell EMC Hardware: A Complete Machine Learning Stack appeared first on Red Hat OpenShift Blog.
Quelle: OpenShift

Cloud Spanner amps up SLA, adds CSV support, and sharpens monitoring details

Providing reliable data services that you can trust to serve your data is the most important goal for our database team here at Google Cloud Platform (GCP). That’s why we put money on the table in the form of availability service-level agreements (SLAs).We’re pleased to announce that all Cloud Spanner instances (not just those of three nodes or more) are now covered under the SLA. Cloud Spanner now supports 99.99% monthly uptime percentage for all regional instances and 99.999% monthly uptime percentage for all multi-region instances under the Cloud Spanner SLA, regardless of instance size.How is this achieved? Each regional Cloud Spanner node is backed by three replicas (each in a different availability zone), and each multi-region Cloud Spanner node has five or more replicas behind it. Cloud Spanner replication allows the service to deliver high availability for each node, and Cloud Spanner’s industry-leading architecture allows all of these replicas to stay in sync and provide up-to-date data. Cloud Spanner provides “scale insurance”—you can start in production with a small instance and not have to re-architect as your application grows. All of this is backed by the SLA.What else is new with Cloud SpannerAdditionally, Cloud Spanner continues to launch multiple features to improve your experience developing applications on GCP, wherever you are in the world. Other recent highlights include:Open source JDBC driver.Written and supported by Google and available under the Apache-based EULA, our JDBC driver implements best practices to aid Java developers using Cloud Spanner. Get started here.Import and export data in CSV format. To help you move data in and out of Cloud Spanner using open and popular formats, the service now supports importing CSV (comma-separated values) files into Cloud Spanner, as well as exporting data from Cloud Spanner to CSV files, in addition to the already supported Apache Avro format. Using Cloud Dataflow, customers can import data into Cloud Spanner from a Cloud Storage bucket that contains a JSON manifest file and a set of CSV files, or export data from Cloud Spanner to a Cloud Storage bucket. To learn more, check out the documentation.Sao Paulo region. Cloud Spanner is now available in Sao Paulo, Brazil, benefiting those of you who need regional instances in South America.Introspection. One of the major differences of using a managed database service like Cloud Spanner instead of running your own database is the ability to peek under the hood when something doesn’t go as expected. To help you better understand how your Cloud Spanner instance is behaving, we have introduced improvements to the fidelity of monitoring data you can get from the system.Latency graphs. If you’re using Cloud Spanner, you can use latency metrics to understand the overall health of your instance and diagnose latency-related issues. The Cloud Spanner console now has graphs for 50th percentile and 99th percentile latency at the database and instance level, including breakdowns for read and write latency. These graphs are also available in Stackdriver.Finer-grained CPU utilization graphs. These enable you to see how Cloud Spanner CPU resources are used and get better insight into system operations vs. user-initiated work. CPU utilization graphs help customers diagnose CPU-related issues and allocate nodes more effectively. Cloud Spanner console now has graphs for rolling average and high-priority CPU utilization, as well as including the database and user/system breakdowns in the total CPU utilization graph.We hope these updates make developing on Cloud Spanner an even more reliable and productive experience. We can’t wait to hear about what you build. Check out our Cloud Spanner YouTube playlist to learn more.
Quelle: Google Cloud Platform