How you can use IoT to power Industry 4.0 innovation

IoT is ushering in an exciting—and sometimes exasperating—time of innovation. Adoption isn’t easy, so it’s important to hold a vision of the promise of Industry 4.0 in mind as you get ready for this next wave of business.

IoT can serve as an onramp to continual transformation, providing companies with the ability to capitalize more fully on automation, AI, and machine learning. As companies harness the power of IoT, cloud services, robotics, and other emerging technologies, they’ll discover new ways of working, creating, and living. They’ll test and learn more swiftly, and scale results in the most promising areas. And this innovation will find form in smart buildings, more efficient factories, connected cities, fully autonomous vehicles, a healthier environment, and better lives.

Between now and that digital world, there are years of trial and error and dozens of applications ahead. But companies across the spectrum are embedding IoT to attain data and analytics mastery, optimize processes, create new services, and rethink products right now. Their leaders are positioning themselves and their companies to take advantage of the promise of digitization across industries.

This post is the fourth in a four-part series designed to help companies maximize their ROI on IoT. In the first post, we discussed how IoT can transform businesses. In the second, we shared insights on how to create a successful strategy that yields desired ROI. In the third post, we discussed how companies can fill capability gaps. Now let’s offer some fresh thinking on what innovation could look like for your company.

IoT innovation is not one size fits all. What it means for a process manufacturing firm is necessarily different than what it will mean for a healthcare company. To help you understand how you might apply IoT to your business—and learn from companies that have gone before you—here are four different innovation plays.

Push service optimization to new levels

With almost all companies competing on the customer experience, it makes sense to optimize service levels to trim cost, error, and delay from customer-facing processes. Better service can be a key differentiator in the marketplace. And when it’s paired with continual optimization enabled by IoT, your customers start seeing the benefit in their businesses.

Jabil is one of the world’s largest and most innovative providers of manufacturing, design engineering, and supply-chain-management technologies. Jabil was quick to recognize that keeping and increasing its competitive edge required the company to accelerate production cycles and personalize products. Its customers might order a product only once, meaning that they couldn’t afford the time delays and waste of traditional inspection processes. “We have many products that customers expect to [have] in their shops within a week,” says Matt Behringer, chief information officer for enterprise operations and quality systems at Jabil. “And that is including transit.”

Jabil used an IoT approach based on the Microsoft Azure Cortana Intelligence Suite to connect systems, gain predictive intelligence, and increase its flexibility and scalability. In a pilot project that connected an electronics manufacturing production line to the cloud, Jabil was able to anticipate and avoid more than half of circuit board failures at the second step in the process, and the remaining 45 percent at the sixth step. By using AI and machine learning, Jabil can correct board errors even earlier in the process, reducing scrapped materials, product failures, and warranty issues. Now, the IoT platform monitors all individual production lines and collects data from every Jabil factory and product worldwide. Jabil is pushing optimization further by using deep neural networks to refine its automated optical inspection process, increasing speed and accuracy to new levels.

“One of the things we’re able to do with predictive analytics in Azure is reduce waste, whether it’s from a process or design issue, or as a result of maintaining enough excess inventory to ensure we have enough for shipment. We’re confident we can produce a good-quality product all the way through the line,” says Behringer.

Leverage data from a digital ecosystem

As companies build IoT-enabled systems of intelligence, they’re creating ecosystems where partners work together seamlessly in a fluid and ever-changing digital supply chain. Participants gain access to a centralized view of real-time data they can use to fine-tune processes, and analytics to enable predictive decision-making. In addition, automation can help customers reduce sources of waste such as unnecessary resource use.

PCL Construction comprises a group of independent construction companies that perform work in the United States, the Caribbean, and Australia. Recognizing that smart buildings are the future of construction, PCL is partnering with Microsoft to drive smart building innovation and focus implementation efforts.

The company is using the full range of Azure solutions—Power BI, Azure IoT, advanced analytics, and AI—to develop smart building solutions for multiple use cases, including increasing construction efficiency and workplace safety, improving building efficiency by turning off power and heat in unused rooms, analyzing room utilization to create a more comfortable and productive work environment, and collecting usage information from multiple systems to optimize services at an enterprise level. PCL’s customers benefit with greater control, more efficient buildings, and lower energy consumption and costs.

However, the path forward wasn’t easy. “Cultural transformation was a necessary and a driving factor in PCL’s IoT journey. To drive product, P&L, and a change in approach to partnering, we had to first embrace this change as a leadership team,” says PCL manager of advanced technology services Chris Palmer.

Develop a managed-services business

Essen, Germany-based thyssenkrupp Elevator is one of the world’s leading providers of elevators, escalators, and other passenger transportation solutions. The company uses a wide range of Azure services to improve usage of its solutions and streamline maintenance at customers’ sites around the globe.

With business partner Willow, thyssenkrupp has used the Azure Digital Twin platform to create a virtual replica of its Innovation Test Tower, an 800-foot-tall test laboratory in Rottweill, Germany. The lab is also an active commercial building, with nearly 200,000 square feet of occupied space and IoT sensors that transmit data 24 hours a day. Willow and thyssenkrupp are using IoT to gain new insights into building operations and how space is used to refine products and services.

In addition, thyssenkrupp has developed MAX, a solution built on the Azure platform that uses IoT, AI, and machine learning to help service more than 120,000 elevators worldwide. Using MAX, building operators can reduce elevator downtime by half and cut the average length of service calls by up to four times, while improving user satisfaction.

The company’s MULTI system uses IoT and AI to make better decisions about where elevators go, providing faster travel times or even scheduling elevator arrival to align with routine passenger arrivals.

“We constantly reconfigure the space to test different usage scenarios and see what works best for the people in the [Innovation Test Tower] building. We don’t have to install massive new physical assets for testing because we do it all through the digital replica—with keystrokes rather than sledgehammers. We have this flexibility thanks to Willow Twin and its Azure infrastructure,” says professor Michael Cesarz, chief executive officer for MULTI at thyssenkrupp.

Rethink products and services for the digital era

Kohler, a leading manufacturer, is embedding IoT in its products to create smart kitchens and bathrooms, meeting consumer demand for personalization, convenience, and control. Built with the Microsoft Azure IoT platform, the platform responds to voice commands, hand motions, weather, and consumer preset options.

And Kohler innovated fast, using Azure to demo, develop, test, and scale the new solutions. “From zero to demo in two months is incredible. We easily cut our development cycle in half by using Azure platform services while also significantly lowering our startup investment,” says Fei Shen, associate director of IoT engineering at Kohler.

The smart bathroom and kitchen products can start a user’s shower, adjust the water temperature to a predetermined level, turn on mirror lights to preferred brightness and color, and share the day’s weather and traffic. They also warn users if water floods their kitchen and bathroom. The smart fixtures provide Kohler with critical insights into how consumers are using their products, which they can use to develop new products and fine-tune existing features.

Kohler is betting that consumer adoption of smart home technology will grow and is pivoting its business to meet new demand. “We’ve been making intelligent products for about 10 years, things like digital faucets and showers, but none have had IoT capability. We want to help people live more graciously, and digitally enabling our products is the next step in doing that,” said Jane Yun, Associate Marketing Manager in Smart Kitchens and Baths at Kohler.

As these examples show, the possibilities for IoT are boundless and success is different for every company. Some firms will leverage IoT only for internal processes, while others will use analytics and automation to empower all the partners in their digital ecosystems. Some companies will wrap data services around physical product offerings to optimize the customer experience and deepen relationships, while still others will rethink their products and services to tap emerging market demand and out-position competitors.

How will you apply IoT insights to transform your businesses and processes? Get help crafting your IoT strategy and maximizing your opportunities for ROI.

Download the Unlocking ROI white paper to learn how to get more value from the Internet of Things.
Quelle: Azure

Visual interface for Azure Machine Learning service

During Microsoft Build we announced the preview of the visual interface for Azure Machine Learning service. This new drag-and-drop workflow capability in Azure Machine Learning service simplifies the process of building, testing, and deploying machine learning models for customers who prefer a visual experience to a coding experience.
Quelle: Azure

Securing the pharmaceutical supply chain with Azure IoT

You’re responsible for overseeing the transportation of a pallet of medicine halfway around the world. Drugs will travel from your pharmaceutical company’s manufacturing outbound warehouse in central New Jersey to third-party logistics firms, distributors, pharmacies, and ultimately, patients. Each box in that pallet – no bigger than the box that holds the business cards on your desk – contains very costly medicine, the product of 10 years of research and R&D spending.

Oh, and there’s a catch – actually several. You will need to ensure compliance with a long list of requirements from temperature and vibration to whether the box has been opened. The box must be kept at a stable temperature of between 2-8 degrees Celsius the whole journey. Additionally, the box is as vulnerable to shock as a Faberge egg. And the contents of each box can easily be faked. And another catch: your company isn’t in the global logistics business, and you lose oversight of those boxes of precious medicine as soon as they leave your freight bay in New Jersey.

IoT opens a new era for secure, smart cold chain asset management

It used to be that the only solution available for you to monitor and manage your cold chain was for your freight technicians to toss a data logger in the center of each outbound pallet and hope for the best. The shipment was passed from the third-party logistics firm to distributors, to warehouses, past freight forwarders, onto last-mile distribution, and finally on to the pharmacy and patients. Your visibility was minimal while your exposure to drug waste or potential counterfeiting was high.

Microsoft and Wipro envisioned a better solution. One that that would help ensure the cold chain was maintained from production to delivery to customers. And one that would limit issues like counterfeiting.

We worked with a top 20 global pharmaceutical company to develop Titan Secure, a digital supply chain and anti-counterfeiting platform. The platform was built with Microsoft Azure Internet of Things (IoT) technologies. See the Titan Secure reference architecture below to learn more.

“Azure IoT technology enabled us to develop a real-time IoT solution that provided the alerts and analytics needed to maintain the cold chain and decrease counterfeiting costs for pharmaceutical customers,” explained Sujan Thanjavuru, Head of Life Sciences Strategy & Transformation, Wipro, Ltd. “We worked with our customer to customize the sensors and develop a user interface that made it easy for managers to understand the state of their pharma shipments in real time. The result was an easy-to-use dashboard that provided valuable insights.”

“Azure IoT brings greater efficiency and reliability to customer value chains with world-class IoT and location intelligence services,” added Tony Shakib, IoT Business Acceleration Leader, Microsoft Azure.

Imagine a future with reduced counterfeit drugs and cold chain product wastage

Fast forward: imagine you’ve implemented Titan Secure from Wipro. Now, your outbound freight technician slaps a small, flexible bluetooth low energy (BLE) beacon sensor onto each box of medication, which is paired with the FDA and EMA-compliant serial number and barcode. The sensors measure temperature, humidity, shock, vibration, and tamper data. They generate geospatial alerts in real time in the event of a temperature excursion or potential counterfeiting attempts. The information is stored in and displayed from Azure. Data is transferred on the backend using Microsoft blockchain, but shipping operators don’t need to know what that means to use it. On an easy-to-use, interactive map and dashboard, technicians can easily track each individual box of your company’s product as it’s shipped from your outbound warehouse all the way to the pharmacy. Your managers receive an alert when a shipment is predicted to get too hot, so that you can call the third party and fix the problem before the shipment has to be destroyed. Once you notice tampering within one of your shipments, you’ll find out quickly what’s happened and how many boxes have been affected.

Manage your cold chain in real-time

What does this mean for your company? Wipro’s Thanjavuru explained, “Pharmaceutical companies can now digitally transform their cold chain management. They can monitor temperature and telemetry data through the entire product journey, view analytics and alerts within the Titan Secure dashboard for visibility including anti-counterfeiting support, and – with cloud connectivity – information about the shipment is available in near real-time.”
Quelle: Azure

Drive higher utilization of Azure HDInsight clusters with Autoscale

We are excited to share the preview release of the Autoscale feature for Azure HDInsight. This feature enables enterprises to become more productive and cost-efficient by automatically scaling clusters up or down based on the load or a customized schedule. 

Let’s consider the scenario of a U.S. based health provider who is using Azure HDInsight to build a unified big data platform at corporate level to process various data for trend prediction or usage pattern analysis. To achieve their business goals, they operate multiple HDInsight clusters in production for real-time data ingestion, batch and interactive analysis.

Some clusters are customized to exact requirements, such as ISV/line of business applications and access control policies, which are subject to rigorous SLA requirements. Sizing such clusters is a hard problem by itself and operating them 24/7 at peak capacity is expensive. So once the clusters are created, IT admins either need to manually monitor the dynamic capacity requirements, scale the clusters up and down, or develop custom tools to do the same. These challenges prevent IT admins from being as productive as possible when building and operating cost-efficient big data analytics workloads.

With the new cluster Autoscaling feature, IT admins can have the Azure HDInsight service automatically monitor and scale the cluster up or down between a admin specified minimum and maximum number of nodes based on either actual load on the cluster or a customized schedule. IT admins can flexibly adjust the cluster size range or the schedule as the unique requirements of their workloads change. The Autoscale feature releases IT admins from having to build complex monitoring tools or worrying about wasted resources and high costs.

Benefits

Automatically make scaling decisions

Once Autoscale is enabled, you can rest assured that the service will take care of your cluster size.

In the load based mode: The cluster size will be scaled up exactly to how much more resources is needed by your applications, but never goes beyond the maximum number you set. Similarly, the cluster size will be scaled down to the minimum to meet your current resource requirements, but never goes below the minimum number of worker nodes you set.
In the schedule based mode: Cluster size will be scaled up and down based on the predefined schedule.  

All the above benefits release IT admins from worrying about wasted resources and allow enterprise to be cost effective and productive.

Pay for only what you need

Autoscale helps you achieve the balance between performance and cost efficiency. Scaling up the cluster lets you derive the business insight you need on time while scaling down the cluster removes the excess resources. Ultimately, Autoscale leads to higher utilization enabling you to pay for only what you need.

Customize to your own scenario

HDInsight Autoscale allows you to customize the scaling strategy based on your own scenario. In the load based mode, you can define the maximum and minimum based on your cost requirements. In the schedule based mode, you can define a schedule for each weekday to meet your own business objectives.

Monitor scaling history easily

The Autoscale feature gives you full visibility in to how the cluster has been scaled up or down. This enables you to further optimize the Autoscale configuration for higher utilization and workload performance.

Using the Azure portal, you can zoom in and out to check the cluster size over the past 90 days.

All the scaling events are also available in Azure Log Analytics. You can run queries to get all the details including when the scaling operation took place, how much resources were needed and how many worker nodes it scaled to. 

Get started

Read the HDInsight Autoscale documentation.
Learn the best practices for Autoscale and tune the settings to become more cost efficient.
Read this developer guide and follow the quick start guide to learn more about implementing open source analytics pipelines on Azure HDInsight.
Stay up-to-date on the latest Azure HDInsight news and features coming up in the near future by following us on Twitter #HDInsight and @AzureHDInsight.
For questions and feedback, please reach out to AskHDInsight@microsoft.com.

Quelle: Azure

Azure.Source – Volume 83

News and updates

Azure SQL Database Edge: Enabling intelligent data at the edge

At Microsoft Build 2019, we announced Azure SQL Database Edge, available in preview, to help address the requirements of data and analytics at the edge using the performant, highly available and secure SQL engine. Developers will now be able to adopt a consistent programming surface area to develop on a SQL database and run the same code on-premises, in the cloud, or at the edge.

Microsoft Azure portal May 2019 update

This month is packed with updates on the Azure portal, including enhancements to the user experience, resource configuration, management tools, and more. Sign in to the Azure portal now and see everything that’s new for yourself. Download the Azure mobile app to stay connected to your Azure resources anytime, anywhere.

A Cosmonaut’s guide to the latest Azure Cosmos DB announcements

At Microsoft Build 2019 we announced exciting new capabilities, including the introduction of real-time operational analytics using new built in support for Apache Spark and a new Jupyter notebook experience for all Azure Cosmos DB APIs. We believe these capabilities will help our customers easily build globally distributed apps at Cosmos scale. But there is even more! This blog lists additional enhancements to the developer experience, announced at Microsoft Build.

Azure Updates

Learn about important Azure product updates, roadmap, and announcements. Subscribe to notifications to stay informed.

Generally available

Premium files redefine limits for Azure Files

Azure Premium Files preview is now available to everyone! Premium files is a new performance tier that unlocks the next level of performance for fully managed file services in the cloud. Premium tier is optimized to deliver consistent performance for IO-intensive workloads that require high-throughput and low latency. Premium shares store data on the latest solid-state drives (SSDs) making it suitable for a wide variety of workloads like file services, databases, shared cache storage, home directories, content and collaboration repositories, persistent storage for containers, media and analytics, high variable and batch workloads, and many more.

Technical content

Azure Firewall and network virtual appliances

Network security solutions can be delivered as appliances on premises, as network virtual appliances (NVAs) that run in the cloud or as a cloud native offering (known as firewall-as-a-service). Customers often ask us how Azure Firewall is different from Network Virtual Appliances, whether it can coexist with these solutions, where it excels, what’s missing, and the total cost of ownership (TCO) benefits expected. We answer these questions in this blog post.

Operationalizing your PostgreSQL database health checks using SQL Notebooks

Most Postgres database administrators and community members would usually bookmark or save such articles so they can revisit them and reuse the queries shared in the article to run checks against their databases. The common challenge with this approach is, you end up with many saved archives, and searching through them when you need it is time consuming and less productive. A better way to operationalize your health check runbooks and database scripts is by creating SQL Notebooks in Azure Data Studio. This blog explains how to do that.

The Urlist — An application study in Serverless and Azure

The Urlist is an application that lets you create lists of URL's that you can share with others. Get it? A list of URL’s? The Urlist? Listen, naming things is hard and all the good domains are already taken. This project was born out of the author’s realization that I was ending my presentations with a slide full of links to additional resources. That’s crazy! What exactly is the audience supposed to do with that? Take a picture with their phone and then go back and manually type it all in later? What decade is this!?

How to Migrate Windows Server 2008 R2 FSMO roles to Windows Server 2019

With the "end of support" on the horizon for Windows Server 2008 R2 coming January 2020, folks are looking around for resources to help them check off some high ticket items from their "to do" list. While coming back from my last Microsoft Ignite The Tour stop, the author had some time to kill waiting for a connection. So he dusted off some of his Active Directory admin skills and document the quick and dirty process of upgrading your Active Directory from 2008 R2 over to the latest version of Windows Server 2019.

AI Search Algorithms Every Data Scientist Should Know

While in recent years, search and planning algorithms have taken a back seat to machine and deep learning methods, better understanding these algorithms can boost the performance of your models. Additionally as more powerful computational technologies such as quantum computing emerge it is very likely that search based AI will make a comeback. This TL;DR post outlines a few of the key search algorithms in AI, why they are important, what and what they are used for.

Azure shows

Next-level maps with ArcGIS for .NET

This week, James is joined by friend of the show & Microsoft MVP Morten Nielsen who introduces us to the world of advanced mapping with ArcGIS for .NET and Xamarin. Morten walks us through what ArcGIS is, how developers can build and use custom maps and data in mobile apps, and awesome 3D visualizations on maps.

Deep Dive: Deploying IoT Edge workloads on Kubernetes

Azure IoT Edge now features support for running natively on the Kubernetes orchestrator. This video goes into how the integration works and caps off with a demo showing what the experience is like for deploying a workload on an on-premise Kubernetes cluster.

Howden: How they built a knowledge-mining solution with Azure Search

Customers across industries including healthcare, legal, media, and manufacturing are looking for new solutions to solve business challenges with AI, including knowledge mining with Azure Search. Howden, a global engineering company, focuses on providing quality solutions for air and gas handling. With over a century of engineering experience, Howden creates industrial products that help multiple sectors improve their everyday processes; from mine ventilation and waste water treatment to heating and cooling. Watch a video to see how they implemented a knowledge-mining solution with Azure Search.

How to enable and use soft delete in a storage account | Azure Portal Series

In this video of the Azure Portal “how to” Series, you will learn how to enable and use “soft delete” in an Azure storage account.
Quelle: Azure

Optimize price-performance with compute auto-scaling in Azure SQL Database serverless

Optimizing compute resource allocation to achieve performance goals while controlling costs can be a challenging balance to strike especially for database workloads with complex usage patterns. To help address these challenges, we are pleased to announce the preview of Azure SQL Database serverless. SQL Database serverless (preview) is a new compute tier that optimizes price-performance and simplifies performance management for databases with intermittent and unpredictable usage. Line-of-business applications, dev/test databases, content management, and e-commerce systems are just some examples across a range of applications that often fit the usage pattern ideal for SQL Database serverless. SQL Database serverless (preview) is also well-suited for new applications with compute sizing uncertainty or workloads requiring frequent rescaling in order to reduce costs. The serverless compute tier enjoys all the fully managed, built-in intelligence benefits of SQL Database and helps accelerate application development, minimize operational complexity, and lower total costs. 

Compute auto-scaling

SQL Database serverless (preview) automatically scales compute for single databases based on workload demand and bills for compute used per second. Serverless contrasts with the provisioned compute tier in SQL Database which allocates a fixed amount of compute resources for a fixed price and is billed per hour. Over short time scales, provisioned compute databases must either over-provision resources at a cost in order to accommodate peak usage or under-provision and risk poor performance. Over longer time scales, provisioned compute databases can be rescaled, but this solution may require predicting usage patterns or writing custom logic to trigger rescaling operations based on a schedule or performance metrics. This adds to development and operational complexity. In serverless, compute scaling within configurable limits is managed by the service to continuously right-size resources. Serverless also provides an option to automatically pause the database during inactive usage periods and automatically resume when activity returns.

Pay only for compute used

In SQL Database serverless (preview), compute is only billed based on the amount of CPU and memory used per second.  While the database is paused only storage is billed, providing additional price optimization benefit. 

Consider a line-of-business application or a dev/test database that is idle at night, but needs multi-core bursting headroom throughout the day. In this example, the application is using SQL Database serverless (preview) configured to allow auto-pausing and auto-scaling up to four vcores and has the following usage pattern over a 24 hour period:

As can be seen, database usage corresponds to the amount of compute billed which is measured in units of vcore seconds and sums to around 46k vcore seconds over the 24 hour period. Suppose the compute unit price for the serverless database is around $0.000073/vcore/second. Then the compute bill for this one day period is just under $3.40. This is calculated by multiplying the compute unit price by the total number of vcore seconds accumulated. During this time period the database was auto-paused while idle and enjoyed the benefit of bursting episodes up to 80 percent of four vcores without customer intervention. In this example, the price savings using serverless is significant compared to a provisioned compute database configured with the same four vcore limit.   

Note that pricing is discounted for preview. In this example, pricing is based on the East US region in May 2019 and subject to change. For the most up-to-date pricing, please visit the Azure SQL Database pricing page.

Price-performance trade-offs

When using SQL Database serverless (preview) there are price-performance trade-offs to consider. These trade-offs are related to the compute unit price and the impact on application performance due to compute warm-up after periods of low or idle usage.

Compute unit price

The compute unit price is higher for a serverless database than for a provisioned compute database since serverless is optimized for workloads with intermittent usage patterns. If CPU or memory usage is high enough and sustained for long enough, then the provisioned compute tier may be less expensive.

Compute warm-up after low usage

While a serverless database is online, memory is gradually reclaimed if CPU or memory usage is low enough or long enough. When workload activity returns, disk IO may be required to rehydrate data pages into the SQL buffer pool or query plans may need to be recompiled. This memory management policy to reclaim cache based on low usage is unique to serverless and done to control customer costs, but can impact performance. Memory reclamation based on low usage does not occur in the provisioned compute tier for single databases or elastic pools where this kind of impact can be avoided.

Compute warm-up after pausing

The latency to pause and resume a database is usually around one minute or less during which time the database is offline. After the database is resumed, memory caches need to be rehydrated which adds additional latency before optimal performance conditions return. The idle period that must elapse before auto-pausing occurs can be configured to compensate for this performance impact. Alternatively, auto-pausing can be disabled for workloads sensitive to this impact and still benefit from auto-scaling. Compute minimums are billed while the database is online regardless of usage, and so disabling auto-pausing can increase costs.

Learn more

Azure SQL Database serverless (preview) is supported in the general purpose tier for single databases. 

Learn more about Azure SQL Database serverless (preview).
Learn more about Azure SQL Database pricing for the serverless compute tier (preview).

Quelle: Azure

Azure Marketplace new offers – Volume 36

We continue to expand the Azure Marketplace ecosystem. For this volume, 22 new offers successfully met the onboarding criteria and went live. See details of the new offers below:

Applications

Bluefish Editor on Windows Server 2016: Apps4Rent helps you deploy Bluefish Editor on Azure. Bluefish, a free software editor with advanced tools for building dynamic websites, is targeted as a middle path between simple editors and fully integrated development environments.

BOSH Stemcell for Windows Server 2019: This offer from Pivotal Software provides Windows Server 2019-based Stemcell for the Pivotal Cloud Foundry platform.

Corda Opensource VM: R3’s Corda is an open-source blockchain platform that removes costly friction in business transactions by enabling institutions to transact directly using smart contracts and ensures privacy and security.

DataStax Distribution of Apache Cassandra: DataStax offers a simple, cost-effective way to run the Apache Cassandra database in the cloud. DDAC addresses common challenges with adoption, maintenance, and support by streamlining operations and controlling costs.

DataStax Enterprise: DataStax delivers the always-on, active-everywhere, distributed hybrid cloud NoSQL database built on Apache Cassandra. DataStax Enterprise (DSE) makes it easy for enterprises to exploit hybrid and multi-cloud environments via a seamless data layer.

FatPipe WAN Optimization for Azure: Significantly boost wide area network performance with FatPipe WAN optimization, which appreciably increases utilization, providing effective use of bandwidth by caching/compressing that sharply reduces redundant data.

Flexbby One RU Edition: Get a comprehensive solution for complex workflow automation in sales, marketing, service, HR, and legal. Flexbby One is powerful software to help you manage the contract lifecycle, document archiving, procurement, customer service, and more.

Flowmon Collector for Azure: Flowmon Collector serves for collection, storage, and analysis of flow data (NetFlow, IPFIX). Flowmon is a comprehensive platform that includes everything you need to get absolute control over your network through network visibility.

Innofactor QualityFirst: Get QualityFirst by Innofactor for healthcare, patient, and care instructions.

Keycloak Gatekeeper Container Image: Keycloak Gatekeeper is an adapter that integrates with Keycloak authentication supporting access tokens in browser cookie or bearer tokens. This Bitnami Container Image is secure, up-to-date, and packaged using industry best practices.

MIKE Zero: This MIKE modeling suite from DHI A/S helps engineers and scientists who want to model water environments, and includes most of MIKE Powered by DHI's inland and marine software.

System Integrity Management Platform (SIMP) 6.3: SIMP is an open-source framework that can either enhance your existing infrastructure or allow you to quickly build one from scratch. Built on the Puppet product suite, SIMP is designed around scalability, flexibility, and compliance.

Consulting services

2 Hr Workshop: Windows in the Cloud: The planning and knowledge transfer workshop from Steeves gives an overview of the Windows 10 Servicing Model and Lifecycle and should be presented to key stakeholders such as IT management, IT staff, and IT decision makers.

Azure Accelerate: Determine the ROI of moving your workloads into Azure. Azure Accelerate from Blue Chip Consulting will deliver insights into server inventory, financial models, target-state architecture drawings, and detailed cloud roadmaps.

Azure Storage for Archive: 2-Day Implementation: CDW will assist you in enabling an archival solution in Azure, sharing industry-leading practices as well as identifying requirements. CDW will implement and pilot the solution in the production environment.

Azure Tiered Storage: 1-Day Implementation: A highly skilled CDW engineer will assist you in creating storage accounts in Azure for use in conjunction with an on-premises, cloud-enabled storage appliance, resulting in a hybrid cloud storage solution.

CSP Migration: 3-Week Assessment: SHI offers a rapid assessment and migration path for any existing Azure customer to its SHI Cloud Service Provider (CSP) offering. SHI keeps you up and running while ensuring best practices around security and manageability.

CSP Migration: 6-Week Assessment and Migration: Need more time to move? Get this six-week assessment and migration for existing Azure customers to the SHI Cloud Service Provider (CSP) offering. SHI keeps you up and running while ensuring best practices.

Domain Controller in Azure: 1-Day Implementation: CDW will configure up to two Azure IaaS virtual machines with the Microsoft AD DS domain controller role to connect to your existing single forest/single domain AD DS on-premises infrastructure.

Microsoft Azure AI Chatbot Development: This consultation with Cynoteck Technology Solutions will provide suggestions and solutions to help your company identify how to best use chatbots depending on your line of business.

SSO Using ADFS: 2-Day Implementation: CDW’s engineers will install and configure up to two Active Directory Federation Services servers and two ADFS web application proxy servers in a single location, simplifying things for your end users.

Windows Server Migration: 5-Day Implementation: This offering from CDW will assist your organization in planning a pilot migration of up to five on-premises supported, non-mission-critical virtualized Windows Servers to Azure.

Quelle: Azure

Accelerate bot development with Bot Framework SDK and other updates

Conversational experiences have become the norm, whether you’re looking to track a package or to find out a store’s hours of operation. At Microsoft Build 2019, we highlighted a few customers who are building such conversational experiences using the Microsoft Bot Framework and Azure Bot Service to transform their customer experience.
Quelle: Azure

Microsoft 365 boosts usage analytics with Azure Cosmos DB

This post is part of a 2-part series about how organizations are using Azure Cosmos DB to meet real-world needs, and the difference it’s making for them. In this first post we explore the challenges that led the Microsoft 365 usage analytics team to take action, the architecture of the new solution, and migration of the production workload. In part 2, we’ll examine additional implementation details and the outcomes resulting from the team’s efforts.

The challenge: Understanding the behavior of more than 150 million active users

Office 365 is a flagship service within the Microsoft 365 Enterprise solution, with millions of commercial customers and more than 150 million active commercial users each month. Office 365 provides extensive reporting for administrators within each company on how the service is being used including license assignment, product-level usage, user-level activity, site activity, group activity, storage consumption, and more. The Microsoft 365 usage analytics team incrementally adds new reports to cover more Office 365 services.

Previous architecture

The telemetry data needed to generate such reports was collected in a system called usage analytics, that until recently ran on the community version of MongoDB. The image below shows the data flow, with an importer web service used to write log streams collected in Azure Blob storage to MongoDB. An OData web service exposes APIs to extract the stored data for both reporting within the Microsoft 365 admin center and for access through Microsoft Graph. Every day, as part of a full daily refresh, several billion rows of data were added to the system.

Each of the primary geographies served by Office 365 has an independent usage analytics repository, all employing a similar architecture. In each geography, data was stored on two MongoDB clusters, with each cluster consisting of up to 50 virtual machines (VMs) hosted in Azure Virtual Machines and running MongoDB. The two clusters in each geography functioned in a primary/backup configuration. Data was written separately to both clusters and under normal operation, all reads were performed on the primary cluster.

Each cluster was designed for a write-heavy workload. To speed writes, sharding of data across individual cluster nodes was done using a random globally unique identifier (GUID) such as a MongoDB shard key. Every day for a few hours, new data from Azure Blob storage was written using a multithreaded importer. Each thread wrote batches of 2,000 records at a time to all cluster nodes and waited for all records to finish before starting on the next batch of 2,000.

Problems and pains

This architecture presented several problems for the Microsoft 365 usage analytics team, ranging from excessive administrative effort and costs to limited performance, reliability, availability, and scalability. Some specific pains included:

Poor performance. Reads were inefficient and reports sometimes timed out because of the use of a random GUID as a shard key required querying all nodes. In addition, during the few hours each day when new data was imported, with writes and reads hitting the primary cluster node during the same time, performance was poor. To make matters worse, if anything failed during a batch write, which often happened due to internal database errors, all 2,000 records had to be written again.
Full-time administration. Maintenance of the MongoDB clusters was manual and time-consuming, requiring human resources to dedicate time towards managing the clusters. This put an unnecessary resource constraint on the team, which would rather use its bandwidth to bring new reports to market. Plus, bugs in MongoDB 3.2 required all servers to be restarted weekly. And renewing the security certificates on each cluster node within the virtual network had to be completed annually, and required an additional two weeks of effort per cluster. During such routine administrative tasks, if an operation failed on one cluster node, the entire cluster was down until the issue was resolved.
High costs. Significant costs were incurred to run the MongoDB backup clusters, which remained idle most of the time. Those costs continued to increase as Office 365 usage grew.
Limited scalability. Less than three years after MongoDB was initially deployed, the largest repository was almost at maximum capacity. Any spare capacity was forecast to run out within six months as more products and reports were added, with no easy way to scale.

While the team was dealing with the architectural limitations of its existing solution, they were looking ahead to a lineup of new, high-scale capabilities that they wanted to enable for customers in the usage analytics space. The team started looking for a new, cost-effective, and low-maintenance solution that would let them move from self-maintained VMs running MongoDB to a fully managed database service.

Geo-distribution on Azure Cosmos DB: The key to an improved architecture

After exploring their options, the team decided to replace MongoDB with Azure Cosmos DB, a fully managed globally-distributed, multi-model database service designed for global distribution and virtually unlimited elastic scalability. The first step was to deploy the needed infrastructure.

In contrast to the primary/backup, two-cluster configuration that it had used with MongoDB, the team took advantage of turnkey global distribution of active data in Azure Cosmos DB. Using multiple Azure regions for data replication provided an easy way to write to any region, read from any region, and better balance the workload across the database instances—all while relying on Azure Cosmos DB to transparently handle active data replication and data consistency.

“True geo-replication had been deemed too hard to do with MongoDB, which is why the previous architecture separately wrote data to both the primary and backup clusters,” says Xiaodong Wang, a Software Engineer on the Microsoft 365 usage analytics team. “With Azure Cosmos DB, implementing transparent geo-distribution literally took minutes—just a few mouse clicks.”

The image below shows the internal architecture of the usage analytics system today. Each of the primary geographies served by Office 365 is served by Cosmos databases geo-replicated across two Azure regions within that geography. Under normal operating conditions, writes are sent to one region within each geography while reads are routed to both. If for some reason a region is prevented from serving reads, those reads are automatically routed to the other region serving that same geography.

Migrating a production workload to Azure Cosmos DB

Developers began writing a new data access layer on the new infrastructure to accommodate reads and writes, using the Azure Cosmos DB SQL (Core) API. After bringing the new system online, the team began to write new production data to both old and new systems, while continuing to serve production reports from the old one.

Developers began to address the reports that they would need to duplicate for the new solution, working through them one at a time. Separate Cosmos containers were created within the database for most reports, so that each collection would be separately scalable after the system came online. The largest reports were addressed first to ensure that Azure Cosmos DB could handle them, and after each new report was verified, the team began serving it from the new environment.

After all functionality and reports were being served by Azure Cosmos DB, and everything was running as it should, the team stopped writing new data to the old system and decommissioned the MongoDB environment. The development team was able to move to Azure Cosmos DB, rewrite the data access layer, and migrate all reports for all geographies without any service interruptions to end users.

In part 2 of this series, we'll cover additional implementation details and the outcomes resulting from the Microsoft 365 usage analytics team’s implementation of Azure Cosmos DB.
Quelle: Azure