da Agency

8 essential criteria for your enterprise integration tooling

IBM just received the highest scores for “technology” and “execution and market impact” in a Middleware Ovum Decision Matrix. I’ll tell you what that might mean for your business in a minute. But first, a quick story you’ll probably find familiar.
I had to call a plumber the other day. An underground pipe for my sprinkler system burst and it started flooding my yard. While the pipe was being replaced the plumber asked me if I cared what type of pipe he used. Nope—it’s all the same, right? Apparently, not, and that’s what got me into trouble in the first place. It’s funny that you don’t think about how complex and critical your plumbing is until it breaks. Then you have a huge mess on your hands, or in this case, in my yard.
Integration is no different. Less visible areas tend not to get the attention they deserve. Integration might not be as cool as cognitive computing, or getting the buzz of blockchain. But without the right underlying connections your data doesn’t flow. And even the coolest innovations cannot deliver on its potential for your business without the ability to access all relevant data and services. For years, middleware has been the technology plumbing, the critical information infrastructure, that brings all of the data and services in an organization together. And it still is.
But now with the expansion of cloud computing, the state of integration has been refactored to meet the growing demands of new digital business and the agility requirements that come with it. Some of the big, multiyear projects are going the way of the iron plumber’s wrench. (That is a plumbing joke.)
Enter middleware as a service.
Middleware as a service (MWaaS) might not be a term you are familiar with. The analyst firm Ovum defines it as:
“a cohesive suite of cloud-based integration services aimed at hybrid integration use cases, including cloud service integration, API-led integration and API management, B2B integration, mobile application/backend integration, and IoT application and data integration.”
Today’s businesses don’t leverage only one form of integration. Moreover, they can’t throw out years of IT investment and completely start from scratch. And they can’t give up the security of an on-premises or private cloud environment. A multicloud approach is critical to success in the digital world. And integration and middleware are what powers it.
But where do you start? Ovum recently released their Decision Matrix: Middleware-as-a-service suites, 2017-2018—download it here. They outlined some of the critical characteristics of a leading solution. Considerations they mention include:

An API-centric, agile approach to application, data and process integration, reducing development effort and costs
The flexibility to provision various combinations of cloud-based integration services based on specific requirements
Uniformity in underlying infrastructure resources and enabling technologies
Flexible integration between various components of the middleware suite
Openness to federation with traditional, on-premises middleware platforms
Uniform user experience across different integration scenarios and middleware components
Support for embedding integration capabilities through APIs into a range of applications and solutions
An extension of developer productivity features and tools to a broader set of integration capabilities. For example, a drag-and-drop approach to integration flow development and prebuilt connectors and templates to a broader set of integration capabilities

Success depends on selecting the right partner for your journey. That is why I am so pleased to announce that IBM received the highest scores for “technology” and “execution and market impact” categories in the recent Ovum Decision Matrix. The recognition that integration is no longer the sole domain of the IT department or tooling that only runs in the central data center has helped drive us to create a solution with comprehensive integration capabilities.
The plumbing matters. Access to the right data and services matters. Flexibility, security and scalability matters. Your digital success is reliant on something that almost no one thinks about. That is, until it breaks your business.
Download the Ovum Decision Matrix: Middleware-as-a-service suites, 2017-2018.
The post 8 essential criteria for your enterprise integration tooling appeared first on Cloud computing news.
Quelle: Thoughts on Cloud

da Agency

Preventing log waste with Stackdriver Logging

By Aja Hammerly, Developer Advocate

If you work with web applications, you probably know they can generate a lot of log messages. There are often multiple log messages for each request, log messages for database queries, and log messages from a monitoring system. Analyzing and understanding all that data can take up precious time and energy, especially if your logs are full of “normal” noise that’s not relevant to the the issue you’re currently facing.

A few years ago, I gave a talk about how we, as a community, need to do a better job managing our data collection and retention. Even with sophisticated tools, searching several terabytes of data takes longer than searching a few gigabytes. Luckily, the solution is simple: stop logging everything. Instead, selectively log what is likely to be important and don’t log the noise.

Stackdriver Logging has recently released a new feature, Log Exclusion Filtering, that helps you be more selective about what is included in your log aggregation. Exclusion filters let you completely exclude log messages from a specific product or messages that match a certain query. You can also choose to sample certain messages so that only a percentage of the messages appear in Stackdriver Logs Viewer. You can learn more about getting started with Log Exclusions here.

Deciding what should always be logged and what you can safely sample or exclude depends on the details of your application. However, we thought we’d share some types of messages you can consider filtering out.

Logs from monitoring systems 
Most web applications have some kind of uptime monitoring in place, and I use Stackdriver Monitoring to monitor mine. It verifies that my application is up every minute from more than five locations. My application logs every request, and so my logs grow by five messages a minute. These messages do not have much value for me; if the uptime check fails, I can already see that in Stackdriver Monitoring. So I created a filter to exclude all messages from Stackdriver Uptime checks.

If your application is running on App Engine, or you’re using host health checking with Container Engine or Compute Engine, you might consider excluding those messages as well. If you run into an issue with your health check, you can choose to re-enable those log messages while you debug the issue.

Logs that indicate success
Logs that indicate everything is fine are another category of messages that are often safe to exclude. HTTP requests with status codes in the 200 range are one example. Log messages for redirects can also be safely excluded in most situations. You may also be able to exclude, or at least only sample, log messages from successful database queries.

These are just a few examples. Looking over your application logs will likely reveal several other messages that are basically “success spam.” Since success messages are some of the most common messages in our logs, reducing them can result in significantly fewer logs overall. This can reduce both actual and cognitive costs associated with log waste.

Logs from non-production systems 

Most folks know that staging and production logs should be clearly separated. But sometimes you’re only occasionally using a tool in production, or perhaps trying out a new product and the logs aren’t yet critical. In cases like these, you can turn off logs for an entire resource type. For example, if you only use BigQuery for ad-hoc analysis, turning off Stackdriver ingestion of BigQuery logs can help reduce the amount of logs that you need to sort through.

Logs from high throughput endpoints 

Logs from high throughput endpoints is another category to consider reducing. One of the applications I worked on early in my career drove 80% of the traffic through a single endpoint. We were generating several gigabytes of data a day for just that URL. Because there was so much data, we could have safely reduced our logging of that traffic from 100% to 50%, or possibly lower. There were enough requests that we would likely get an example of any errors even if we only logged one out of every two messages.

Static traffic is often high throughput, too. If your application is logging, each time someone downloads a stylesheet or favicon you may be able to reduce waste by only logging these messages occasionally.

The what ifs 
These are just a few examples of what can be reduced to help get your logging under control. Looking at your application logs and thinking about the types of errors you often see can yield even more ideas for reducing log volume.

So why don’t more of us reduce our logging? The most common reason I hear is: “What if we need it?” With Stackdriver Log Exclusions, you can always turn off an exclusion and see all the future traffic in the Logs Viewer. Once you’re aware of an issue, you can adjust your logging to help debug it. Additionally, you can export all the logs, even the excluded ones, to BigQuery or Google Cloud Storage if you need the full historical logs for debugging or other purposes.

Stackdriver Logging and Stackdriver Log Exclusions are powerful, and I encourage you to try them out to see if it can help you reduce costs and use resources more efficiently. To learn more, visit Cloud.google.com/logging/.
Quelle: Google Cloud Platform