Microsoft Receives 2020 SAP® Pinnacle Award: Public and Private Cloud Provider Partner of the Year

I’m pleased to share that SAP recently named Microsoft its Partner of the Year for the 2020 SAP® Pinnacle Award category of Public and Private Cloud Provider. SAP presents these awards annually to the top partners that have excelled in developing and growing their partnership with SAP and helping customers run better. Winners and finalists in multiple categories were chosen based on recommendations from the SAP field, customer feedback and performance indicators.

Microsoft and SAP have a long history of partnership to serve our mutual customers with enterprise-class products, service, and support they rely on to run their most mission-critical business processes. Customers like CONA Services have increased agility and performance to handle over 160,000 sales orders a day by running a 28 TB SAP HANA® system on Azure. Daimler AG reduced operational costs by 50 percent and increased agility by spinning up resources on-demand in 30 minutes with SAP S/4HANA® and Azure, empowering 400,000 global suppliers. Carlsberg modernized its infrastructure and optimized its SAP production landscape by migrating everything, 1,600 TB of data, to Azure within six months.

In October 2019 we deepened this relationship even more by announcing the Embrace initiative, whereby Microsoft and SAP signed a unique go-to-market partnership. As part of this initiative, SAP, Microsoft and our joint partner ecosystem have been working together to give our customers a well-defined path to migrate their SAP ERP to SAP S/4HANA® to the cloud, where they can harness the power of their applications to truly drive innovation. Specifically, our teams have been working on:

A simplified migration from on-premises editions of SAP ERP to SAP S/4HANA® for customers with integrated product and industry solutions. Industry market bundles will create a roadmap to the cloud for customers in focused industries, with a singular reference architecture and path to streamline implementation.
Collaborative support model for simplified resolution. In response to customer feedback, a combined support model for Azure and SAP Cloud Platform will help ease migration and improve communication.
Jointly developed market journeys to support customer needs. Designed in collaboration with SAP, Microsoft and system integrator partners will provide roadmaps to the digital enterprise with recommended solutions and reference architectures for customers. These offer a harmonized approach by industry for products, services, and practices across Microsoft, SAP and system integrators.

The past few months I had the privilege of working even closer with my colleagues in SAP marketing to really make the promise of the Embrace initiative real for our customers. This promise is all about bringing value to accelerate customers’ transformation to the intelligent enterprise. But I am not alone! Sales, marketing, engineering and support teams across the two organizations have been teaming up to make it easier and clearer for customers to build and run their mission-critical SAP solutions on Azure. In these current times of global disruptions and uncertainty, I believe both companies feel a bigger responsibility to help our customers reduce complexity, empower their employees, drive innovation, and run their businesses more efficiently.

SAP and Microsoft have been partners for more than 25 years. In fact, Microsoft is the only cloud provider that’s been running SAP for its own finance, HR and supply chains for the last 20+ years, including SAP S/4HANA®. Our Microsoft IT team will share their experience of migrating and running our SAP business applications in the cloud in this upcoming virtual session on April 22, 2020. Likewise, SAP has chosen Azure to run a growing number of its own internal system landscapes, also including those based on SAP S/4HANA®. Our commitment to work together to deliver a world-class experience for our customers has grown stronger over the years. I truly believe in and see every day how this partnership is taking a more unified approach to accelerate the value customers get in the cloud and open up new opportunities for growth, innovation and business transformation.

Learn more about SAP solutions on Azure.
Quelle: Azure

Next Generation SAP HANA Large Instances with Intel® Optane™ drive lower TCO

At Microsoft Ignite 2019, we announced general availability of the new SAP HANA Large Instances powered by the 2nd Generation Intel Xeon Scalable processors, formally Cascade Lake, supporting Intel® Optane™ persistent memory (PMem).

Microsoft’s largest SAP customers are continuing to consolidate their business functions and growing their footprint. S/4 HANA workloads demand increasingly larger nodes as they scale up. Some scenarios for high availability/disaster recovery (HA/DR) and multi-tier data needs are adding to the complexity of operations.

In partnership with Intel and SAP, we have worked to develop the new HANA Large Instances with Intel Optane PMem offering higher memory density and in-memory data persistence capabilities. Coupled with 2nd Generation Intel Xeon Scalable processors, these instances provide higher performance and higher memory to processor ratio.

For SAP HANA solutions, these new offerings help lower total cost of ownership (TCO), simplify the complex architectures for HA/DR and multi-tier data, and offer 22 times faster reload times. The new HANA large instances extend the broad array of the existing large instances offering with the purpose built capabilities critical for running SAP HANA workloads.

Available now

The new S224 HANA Large Instances support 3 TB to 9 TB of memory with four socket 224 vCPUs. The new instances support both DRAM-only and DRAM plus Intel® Optane™ persistent memory combinations.

The variety of SKUs gives our customers the ability to choose the best solution for their SAP HANA in-memory workload needs, with higher memory capacity and lower cost as compared to DRAM-only instances. S224 SKUs with a higher core to working memory ratio are performance-optimized for OLAP while higher working memory to core ratio are better priced for OLTP.

The S224 instances with Intel Optane PMem come in 1:1, 1:2, and 1:4 ratios. Each ratio indicates the size of DRAM memory paired with Intel Optane memory. The architecture options available with these offerings are discussed in the next section. The new HANA Large Instances are available in several Azure regions where HANA Large Instances are available.

Key benefits of deploying S224 instances

Platform consolidation

SAP HANA is an in-memory data platform and its hybrid structure for processing both OLTP and OLAP workloads in real-time with low latency is a major benefit for enterprises using SAP HANA. The 2nd Generation Intel Xeon Scalable processors offer 50 percent higher performance and higher memory to processor ratioi  compared to the previous generation processors. Coupled with Intel Optane, the new instances offer even higher memory densities with >3TB per socket.

SAP HANA uses Intel Optane PMem as an extension to DRAM memory by selectively placing the data structures in persistent memory, in a mode called app direct. The column store data which attributes for majority of the data in most HANA systems is enabled for placement in Intel Optane persistent memory where-as working DRAM memory is used for delta merges, row store and cache data.

For organizations with growing data needs, the higher memory densities enable a deployment to scale up or scale out with fewer of the S224 SKU’s (seen in Figure 1) as compared to a larger number of DRAM-only nodes on previous generation processors. This enables organizations to consolidate their platform footprint and reduce operational complexity, realizing reduced TCO.

Figure 1: Platform consolidation with higher memory density nodes from larger scale out to fewer scale up.

Faster reload times

The data stored in the Intel Optane PMem is persistent. This means for SAP HANA deployments using the new instances with Optane PMem, there is no need to load data from disks or slower storage tiers in the event of system reboot. As mentioned previously, SAP HANA leverages app direct mode to store most of the database into Optane persistent memory. When system reboot occurs during upgrades as an example, the data reload time is cut down dramatically, enabling a faster return to normal operations as compared to DRAM-only systems.

In recent testing conducted using two S224 instances, a DRAM-only system running 6 TB of memory, and a system with 9 TB of memory consisting of 3 TB DRAM and 6 TB of Optane PMem in a 1:2 ratio, the data reload time on the Optane system was 22 times faster as compared to the reload time on the DRAM-only system. The load time on the DRAM system post system reboot is around 44 minutes versus 2 minutes on the Optane node.

Figure 2: Internal testing using 3 TB HANA dataset Shows 22x improvement in DB restart times on the new SAP HANA large instances using Intel Optane.

The faster reload and recovery times may help some deployments to run without HA for non-production workloads with reduced service windows, and remove clustering complexity and downtimes needed for upgrades and/or patches. Each SAP HANA large instance region comes with hot spares to cover the scenario of complete system failure and recover the DB using hot spares.

Lower TCO for HA/DR

The higher memory density offered with the new instances also enable new deployment options available to enterprises for business continuity purposes. The smaller DRAM-only node at the primary site can replicate the data into a larger Intel Optane node offered in 1:2 and 1:4 ratios, with the data preloaded in persistent memory. Higher density Optane node can be used as a dual-purpose node (as seen in Figure 3) for QA testing and also act as primary node in the event of a failover at the primary site, thereby lowering cost by eliminating the need for standalone instances for QA and DR. The data on the larger Optane node is pre-loaded into Optane PMem, which eliminates the need to load the data from disks and cuts the downtime, thus achieving better RTO and RPO times.

Figure 3: Lower TCO with a dual-purpose node at DR site serving the needs for QA/Dev test and DR.

Similarly, HSR replicated configurations in a scale out S/4 HANA setup can be replicated into a single shared HA Optane node in a 1:4 ratio (as seen in Figure 4), reducing the complexity of managing multiple HA instances, thereby lowering TCO and achieving reduced service windows.

Figure 4: Lower TCO for HA and DR using shared higher memory node for scale out deployments.

Enabling SAP HANA on Intel Optane

Supported OS versions

Below is the guidance on the supported OS and HANA versions for using Intel Optane persistent memory technology (PMem).

Following OS versions support Intel Optane in App direct mode:

RHEL 7.6 or later
SLES* 12 SP4 or later
SLES 15 or later

SAP HANA support

SAP HANA 2.0 SPS 03 is the first SAP HANA version to support Intel Optane in app direct mode. The recommended version is SAP HANA 2.0 SPS 04 (or a later version) for customers using Optane nodes. SAP HANA can leverage Intel Optane in app direct mode by configuring PMem regions, namespaces, and file system. The HANA large instance operations team will drive the configuration setup before handing over the Optane node to customers.

SAP HANA configuration

SAP HANA needs to recognize the new Intel Optane PMem DIMMs. The directory that SAP HANA uses as base path must mount onto the file system that were created for PMem. SAP HANA SPS04 or a later version is a requirement for Optane usage. Below is the specific command to set up the base path for the PMem volumes:

In the [persistence]section of the global.ini file, provide a line with a comma-separated list of all mounted PMem volumes by running the following command. Following this, SAP HANA recognizes the PMem devices and loads column store data into the modules.

[persistence]
basepath_persistent_memory_volumes=/hana/pmem/nvmem0; /hana/pmem/nvmem1; /hana/pmem/nvmem2; /hana/pmem/nvmem3

Learn more

If you are interested in learning more about the S224 SKUs, please contact your Microsoft account team. To learn more about running SAP solutions on Azure, visit SAP on Azure or download a free SAP on Azure implementation guide.

i Intel Shows 1.59x Performance Improvement in Upcoming Intel Xeon Processor Scalable Family
Quelle: Azure

Accelerating digital transformation in manufacturing

Digital transformation in manufacturing has the potential to increase annual global economic value by $4.5 trillion according to the IDC MarketScape.i With so much upside, manufacturers are looking at how technologies like IoT, machine learning, and artificial intelligence (AI) can be used to optimize supply chains, improve factory performance, accelerate product innovation, and enhance service offerings.

Digital transformation starts by collecting data from machines on the plant floor, assets in the supply chain, or products being used by customers. This data can be combined with other business data and then modeled and analyzed to gain actionable insights.

Let’s take a look at three manufacturers—Festo, Kao, and AkzoNobel—and see how each one is using technologies like IoT, machine learning, and AI to accelerate their digital transformation.

Providing predictive maintenance as a service

Based in Germany, Festo sells electric and pneumatic drive solutions to 300,000 customers in 176 countries. The company’s goal is to increase uptime for customers by providing predictive maintenance offerings as software as a service (SaaS) offerings. Festo’s strategy is to connect machines to the cloud with Azure IoT and then enable customers to visualize data along the entire value chain.

One of the first SaaS offerings is Festo Dashboards built on Azure. Festo Dashboards provides a clear and intuitive status of equipment like sensor temperatures and valve switches. With Festo Dashboards, manufacturers can more easily monitor energy consumption, quickly diagnose faults, and optimize production availability.

Anticipating consumer trends for better manufacturing forecasting

Kao, one of Japan’s leading consumer brands, sees the consumer market evolving. Today, consumers prioritize their product experience over product quality. They also look to social media for purchasing guidance. These behaviors lead to forecasting challenges. To keep up with these changes, Kao sought to better understand individual customers and categorize trends into micro-segments. The company terms this approach “small mass marketing.” Kao designed a data analysis platform using Microsoft Azure Synapse Analytics and Microsoft Power BI to predict consumer trends for their detergent, cosmetic, and toiletry products. The Kao team combined data from real-time purchases, social media, and historical sales. Kao competes more effectively using predictive models, and chain store employees are empowered with real-time information for selling.

Reducing the development time of new paint colors

Dutch paint and coatings leader, AkzoNobel, is active in more than 100 countries. The company has honed the art of color matching for two centuries for cars, buildings, and interiors. One of the company’s businesses is developing the paint to repair cars when drivers have an accident. Manufacturers in the car and other industries constantly dream up new finishes to give their models an edge on the competition.

To keep up with rapid rate of change, AkzoNobel introduced Azure Machine Learning into its color prediction process. Previously, scientists labored painstakingly in labs to adjust, recalibrate, and tweak a color until it was just right. The company worked with its scientist and technicians to integrate machine mearning into their development process. The main impact is seen in the lab, where teams are now able to create more color recipes, more accurately, in less time. Previously, it could take up to two years to get a car color ready. Now AkzoNobel is seeing new paint colors ready in one month.

Next steps

For ideas on accelerating your digital transformation journey download, The Road to Intelligent Manufacturing: Leveraging a Platform, co-authored by Microsoft and Capgemini.

i IDC MarketScape: Worldwide Industrial IoT Platforms in Manufacturing 2019 Vendor Assessment
Quelle: Azure

Solutions and guidance to help content producers and creators work remotely

The global health pandemic has impacted every organization on the planet—no matter the size—their employees, and the customers they serve. The emphasis on social distancing and shelter in place orders have disrupted virtually every industry and form of business. The Media & Entertainment (M&E) industry is no exception. Most physical productions have been shut down for the foreseeable future. Remote access to post-production tools and content is theoretically possible, but in practice is fraught with numerous issues, given the historically evolved, fragmented nature of the available toolsets, vendor landscape, and the overall structure of the business

At the same time, more so today than ever before, people are turning to stories, content, and information to connect us with each other. If you need help or assistance with general remote work and collaboration, please visit this blog.

If you’d like to learn more about best practices and solutions for M&E workloads, such as VFX, editorial, and other post-production workflows—which are more sensitive to network latency, require specialized high-performance hardware and software in custom pipelines, and where assets are mostly stored on-premises (sometimes in air-gapped environments)—read on.

First, leveraging existing on-premises hardware can be a quick solution to get your creative teams up and running. This works when you have devices inside the perimeter firewall, tied to specific hardware and network configurations that can be hard to replicate in the cloud. It also enables cloud as a next step rather than a first step, helping you fully leverage existing assets and only pay for cloud as you need it. Solutions such as Teradici Cloud Access Software running on your artists’ machines enables full utilization of desktop computing power, while your networking teams provide a secure tunnel to that machine. No data movement is necessary, and latency impacts between storage and machine are minimized, making this a simple, fast solution to get your creatives working again. For more information, read Teradici’s Work-From-Home Rapid Response Guide and specific guidance for standalone computers with Consumer Grade NVIDIA GPUs.

Customers who need to enable remote artists with cloud workstations, while maintaining data on-premises, can also try out an experimental way to use Avere vFXT for Azure caching policies to further reduce latency. This new approach optimizes creation, deletion, and listing of files on remote NFS shares often impacted by increased latency. 

Second, several Azure partners have accelerated work already in progress to provide customers with new remote options, starting with editorial.

Avid has made their new Avid Edit on Demand solution immediately available through their Early Access Program. This is a great solution for broadcasters and studios who want to spin up editorial workgroups of up to 30 users. While the solution will work for customers anywhere in the world, it is currently deployed in US West 2, East US 2, North Europe, and Japan East so customers closest to those regions will have the best user experience. You can apply to the Early Access Program here, and applications take about two days to process. Avid is also working to create a standardized Bring Your Own License (BYOL) and Software as a Service (SaaS) that addresses enterprise post-production requirements.
Adobe customers who purchase Creative Cloud for individuals or teams can use Adobe Premiere Pro for editing in a variety of remote work scenarios. Adobe has also extended existing subscriptions for an additional two months. For qualified  Enterprise customers who would like to virtualize and deploy Creative Cloud applications in their environments, Adobe wanted us to let you know, “it is permitted as outlined in the Creative Cloud Enterprise Terms of Use.” Customers can contact their Adobe Enterprise representative for more details and guidance on best practices and eligibility.
BeBop, powered by Microsoft Azure, enables visual effects artists, editors, animators, and post-production professionals to create and collaborate from any corner of the globe, with high security, using just a modest internet connection. Customers can remotely access Adobe Creative Cloud applications, Foundry software, and Autodesk products and subscriptions including Over the Shoulder capabilities and BeBop Rocket File Transfer. You can sign up at Bebop’s website.
StratusCore provides a comprehensive platform for the remote content creation workforce including industry leading software tools through StratusCore’s marketplace; virtual workstation, render nodes and fast storage; project management, budget and analytics for a variety of scenarios. Individuals and small teams can sign up here and enterprises can email them here.

Third, while these solutions work well for small to medium projects, teams, and creative workflows, we know major studios, enterprise broadcasters, advertisers, and publishers have unique needs. If you are in this segment and need help enabling creative—or other Media and Entertainment specific workflows for remote work—please reach out to your Microsoft sales, support, or product group contacts so we can help

I know that we all want to get people in this industry back to work, while keeping everyone as healthy and safe as possible!

We’ll keep you updated as more guidance becomes available, but until then thank you for everything everyone is doing as we manage through an unprecedented time, together.
Quelle: Azure

Using Azure Monitor source map support to debug JavaScript errors

Azure Monitor’s new source map support expands a growing list of tools that empower developers to observe, diagnose, and debug their JavaScript applications.

Difficult to debug

As organizations rapidly adopt modern JavaScript frontend frameworks such as React, Angular, and Vue, they are left with an observability challenge. Developers frequently minify/uglify/bundle their JavaScript application upon deployment to make their pages more performant and lightweight which obfuscates the telemetry collected from uncaught errors and makes those errors difficult to discern.

Source maps help solve this challenge. However, it’s difficult to associate the captured stack trace with the correct source map. Add in the need to support multiple versions of a page, A/B testing, and safe-deploy flighting, and it’s nearly impossible to quickly troubleshoot and fix production errors.

Unminify with one-click

Azure Monitor’s new source map integration enables users to link an Azure Monitor Application Insights Resource to an Azure Blob Services Container and unminify their call stacks from the Azure Portal with a single click. Configure continuous integration and continuous delivery (CI/CD) pipelines to automatically upload your source maps to Blob storage for a seamless end-to-end experience.

Microsoft Cloud App Security’s story

The Microsoft Cloud App Security (MCAS) Team at Microsoft manages a highly scalable service with a React JavaScript frontend and uses Azure Monitor Application Insights for clientside observability.

Over the last five years, they’ve grown in their agility to deploying multiple versions per day. Each deployment results in hundreds of source map files, which are automatically uploaded to Azure Blob container folders according to version and type and stored for 30 days.

Daniel Goltz, Senior Software Engineering Manager, on the MCAS Team explains, “The Source Map Integration is a game-changer for our team. Before it was very hard and sometimes impossible to debug and resolve JavaScript based on the unminified stack trace of exceptions. Now with the integration enabled, we are able to track errors to the exact line that faulted and fix the bug within minutes.”

Debugging JavaScript demo

Here’s an example scenario from a demo application:

Get started

Configure source map support once, and all users of the Application Insights Resource benefit. Here are three steps to get started:

Enable web monitoring using our JavaScript SDK.
Configure a Source Map storage account.

End-to-end transaction details blade.
Properties blade.

Configure CI/CD pipeline.

Note: Add an Azure File Copy task to your Azure DevOps Build pipeline to upload source map files to Blob each time a new version of your application deploys to ensure relevant source map files are available.

 

Manually drag source map

If source map storage is not yet configured or if your source map file is missing from the configured Azure Blob storage container, it’s still possible to manually drag and drop a source map file onto the call stack in the Azure Portal.

 

Submit your feedback

Finally, this feature is only possible because our Azure Monitor community spoke out on GitHub. Please keep talking, and we’ll keep listening. Join the conversation by entering an idea on UserVoice, creating a new issue on GitHub, asking a question on StackOverflow, or posting a comment below.
Quelle: Azure

Detect large-scale cryptocurrency mining attack against Kubernetes clusters

Azure Security Center's threat protection enables you to detect and prevent threats across a wide variety of services from Infrastructure as a Service (IaaS) layer to Platform as a Service (PaaS) resources in Azure, such as IoT, App Service, and on-premises virtual machines.

At Ignite 2019 we announced new threat protection capabilities to counter sophisticated threats on cloud platforms, including preview for threat protection for Azure Kubernetes Service (AKS) Support in Security Center and preview for vulnerability assessment for Azure Container Registry (ACR) images.

Azure Security Center and Kubernetes clusters 

In this blog, we will describe a recent large-scale cryptocurrency mining attack against Kubernetes clusters that was recently discovered by Azure Security Center. This is one of the many examples Azure Security Center can help you protect your Kubernetes clusters from threats.

Crypto mining attacks in containerized environments aren’t new. In Azure Security Center, we regularly detect a wide range of mining activities that run inside containers. Usually, those activities are running inside vulnerable containers, such as web applications, with known vulnerabilities that are exploited.

Recently, Azure Security Center detected a new crypto mining campaign that targets specifically Kubernetes environments. What differs this attack from other crypto mining attacks is its scale: within only two hours a malicious container was deployed on tens of Kubernetes clusters.

The containers ran an image from a public repository: kannix/monero-miner. This image runs XMRig, a very popular open source Monero miner.

The telemetries showed that container was deployed by a Kubernetes Deployment named kube-control.

As can be shown in the Deployment configuration below, the Deployment, in this case, ensures that 10 replicas of the pod would run on each cluster:

In addition, the same actor that deployed the crypto mining containers also enumerated the cluster resources including Kubernetes secrets. This might lead to exposure of connection strings, passwords, and other secrets which might enable lateral movement.

The interesting part is that the identity in this activity is system:serviceaccount:kube-system:kubernetes-dashboard which is the dashboard’s service account.
This fact indicates that the malicious container was deployed by the Kubernetes dashboard. The resources enumeration was also initiated by the dashboard’s service account.

There are three options for how an attacker can take advantage of the Kubernetes dashboard:

Exposed dashboard: The cluster owner exposed the dashboard to the internet, and the attacker found it by scanning.
The attacker gained access to a single container in the cluster and used the internal networking of the cluster for accessing the dashboard (which is possible by the default behavior of Kubernetes).
Legitimate browsing to the dashboard using cloud or cluster credentials.

The question is which one of the three options above was involved in this attack? To answer this question, we can use a hint that Azure Security Center gives, security alerts on the exposure of the Kubernetes dashboard. Azure Security Center alerts when the Kubernetes dashboard is exposed to the Internet. The fact that this security alert was triggered on some of the attacked clusters implies that the access vector here is an exposed dashboard to the Internet.

A representation of this attack on the Kubernetes attack matrix would look like:

 

Avoiding cryptocurrency mining attacks

How could this be avoided?

Do not expose the Kubernetes dashboard to the Internet: Exposing the dashboard to the Internet means exposing a management interface.
Apply RBAC in the cluster: When RBAC is enabled, the dashboard’s service account has by default very limited permissions which won’t allow any functionality, including deploying new containers.
Grant only necessary permissions to the service accounts: If the dashboard is used, make sure to apply only necessary permissions to the dashboard’s service account. For example, if the dashboard is used for monitoring only, grant only “get” permissions to the service account.
Allow only trusted images: Enforce deployment of only trusted containers, from trusted registries.

Learn more

Kubernetes is quickly becoming the new standard for deploying and managing software in the cloud. Few people have extensive experience with Kubernetes and many only focuses on general engineering and administration and overlook the security aspect. Kubernetes environment needs to be configured carefully to be secure, making sure no container focused attack surface doors are not left open is exposed for attackers. Azure Security Center provides:

Discovery and Visibility: Continuous discovery of managed AKS instances within Security Center’s registered subscriptions.
Secure Score recommendations: Actionable items to help customers comply with security best practices in AKS as part of the customer’s Secure Score, such as "Role-Based Access Control should be used to restrict access to a Kubernetes Service Cluster."
Threat Detection: Host and cluster-based analytics, such as “A privileged container detected."

To learn more about AKS Support in Azure Security Center, please visit the documentation here.
Quelle: Azure

Introducing incremental enrichment in Azure Cognitive Search

Incremental enrichment is a new feature of Azure Cognitive Search that brings a declarative approach to indexing your data. When incremental enrichment is turned on, document enrichment is performed at the least cost, even as your skills continue to evolve. Indexers in Azure Cognitive Search add documents to your search index from a data source. Indexers track updates to the documents in your data sources and update the index with the new or updated documents from the data source.

Incremental enrichment is a new feature that extends change tracking from document changes in the data source to all aspects of the enrichment pipeline. With incremental enrichment, the indexer will drive your documents to eventual consistency with your data source, the current version of your skillset, and the indexer.

Indexers have a few key characteristics:

Data source specific.
State aware.
Can be configured to drive eventual consistency between your data source and index.

In the past, editing your skillset by adding, deleting, or updating skills left you with a sub-optimal choice. Either rerun all the skills on the entire corpus, essentially a reset on your indexer, or tolerate version drift where documents in your index are enriched with different versions of your skillset.

With the latest update to the preview release of the API, the indexer state management is being expanded from only the data source and indexer field mappings to also include the skillset, output field mappings knowledge store, and projections.

Incremental enrichment vastly improves the efficiency of your enrichment pipeline. It eliminates the choice of accepting the potentially large cost of re-enriching the entire corpus of documents when a skill is added or updated, or dealing with the version drift where documents created/updated with different versions of the skillset and are very different in shape and/or quality of enrichments.

Indexers now track and respond to changes across your enrichment pipeline by determining which skills have changed and selectively execute only the updated skills and any downstream or dependent skills when invoked. By configuring incremental enrichment, you will be able to ensure that all documents in your index are always processed with the most current version of your enrichment pipeline, all while performing the least amount of work required. Incremental enrichment also gives you the granular controls to deal with scenarios where you want full control over determining how a change is handled.

Indexer cache

Incremental indexing is made possible with the addition of an indexer cache to the enrichment pipeline. The indexer caches the results from each skill for every document. When a data source needs to be re-indexed due to a skillset update (new or updated skill), each of the previously enriched documents is read from the cache and only the affected skills, changed and downstream of the changes are re-run. The updated results are written to the cache, the document is updated in the index and optionally, the knowledge store. Physically, the cache is a storage account. All indexes within a search service may share the same storage account for the indexer cache. Each indexer is assigned a unique cache id that is immutable.

Granular controls over indexing

Incremental enrichment provides a host of granular controls from ensuring the indexer is performing the highest priority task first to overriding the change detection.

Change detection override: Incremental enrichment gives you granular control over all aspects of the enrichment pipeline. This allows you to deal with situations where a change might have unintended consequences. For example, editing a skillset and updating the URL for a custom skill will result in the indexer invalidating the cached results for that skill. If you are only moving the endpoint to a different virtual machine (VM) or redeploying your skill with a new access key, you really don’t want any existing documents reprocessed.

To ensure that that the indexer only performs enrichments you explicitly require, updates to the skillset can optionally set disableCacheReprocessingChangeDetection query string parameter to true. When set, this parameter will ensure that only updates to the skillset are committed and the change is not evaluated for effects on the existing corpus.

Cache invalidation: The converse of that scenario is one where you may deploy a new version of a custom skill, nothing within the enrichment pipeline changes, but you need a specific skill invalidated and all affected documents re-processed to reflect the benefits of an updated model. In these instances, you can call the invalidate skills operation on the skillset. The reset skills API accepts a POST request with the list of skill outputs in the cache that should be invalidated. For more information on the reset skills API, see the documentation.

Updates to existing APIs

Introducing incremental enrichment will result in an update to some existing APIs.

Indexers

Indexers will now expose a new property:

Cache

StorageAccountConnectionString: The connection string to the storage account that will be used to cache the intermediate results.
CacheId: The cacheId is the identifier of the container within the annotationCache storage account that is used as the cache for this indexer. This cache is unique to this indexer and if the indexer is deleted and recreated with the same name, the cacheid will be regenerated. The cacheId cannot be set, it is always generated by the service.
EnableReprocessing: Set to true by default, when set to false, documents will continue to be written to the cache, but no existing documents will be reprocessed based on the cache data.

Indexers will also support a new querystring parameter:

ignoreResetRequirement set to true allows the commit to go through, without triggering a reset condition.

Skillsets

Skillsets will not support any new operations, but will support new querystring parameter:

disableCacheReprocessingChangeDetection set to true when you want no updates to on existing documents based on the current action.

Datasources

Datasources will not support any new operations, but will support new querystring parameter:

ignoreResetRequirement set to true allows the commit to go through without triggering a reset condition.

Best practices

The recommended approach to using incremental enrichment is to configure the cache property on a new indexer or reset an existing indexer and set the cache property. Use the ignoreResetRequirement sparingly as it could lead to unintended inconsistency in your data that will not be detected easily.

Takeaways

Incremental enrichment is a powerful feature that allows you to declaratively ensure that your data from the datasource is always consistent with the data in your search index or knowledge store. As your skills, skillsets, or enrichments evolve the enrichment pipeline will ensure the least possible work is performed to drive your documents to eventual consistency.

Next steps

Get started with incremental enrichment by adding a cache to an existing indexer or add the cache when defining a new indexer.
Quelle: Azure

Accelerating innovation: Start with Azure Sphere to secure IoT solutions

From agriculture to healthcare, IoT unlocks opportunity across every industry, delivering profound returns, such as increased productivity and efficiency, reduced costs, and even new business models. And with a projected 41.6 billion IoT connected devices by 2025, momentum continues to build.

While IoT creates new opportunities, it also brings new cybersecurity challenges that could potentially result in stolen IP, loss of brand trust, downtime, and privacy breaches. In fact, 97 percent of enterprises rightfully call out security as a key concern when adopting IoT. But when organizations have a reliable foundation of security on which they can build from the start, they can realize durable innovation for their business versus having to figure out what IoT device security requires and how to achieve it.

Read on to learn how you can use Azure Sphere—now generally available—to create and accelerate secure IoT solutions for both new devices and existing equipment. As you look to transform your business, discover why IoT security is so important to build in from the start and see how the integration of Azure Sphere has enabled other companies to focus on innovation. For a more in-depth discussion, be sure to watch the Azure Sphere general availability webinar.

Defense in depth, silicon-to-cloud security

It’s important to understand on a high level how Azure Sphere delivers quick and cost-effective device security. Azure Sphere is designed around the seven properties of highly secure devices and builds on decades of Microsoft experience in delivering secure solutions. End-to-end security is baked into the core, spanning the hardware, operating system, and cloud, with ongoing service updates to keep everything current.

While other IoT device platforms must rely on costly manual practices to mitigate missing security properties and protect devices from evolving cybersecurity threats, Azure Sphere delivers defense-in-depth to guard against and respond to threats. Add in ongoing security and OS updates to help ensure security over time, and you have the tools you need to stay on top of the shifting digital landscape.

Propel innovation on a secure foundation

Azure Sphere removes the complexity of securing IoT devices and provides a secure foundation to build on. This means that IoT adopters spend less time and money focused on security and more time innovating solutions that solve key business problems, delivering a greater return on investment as well as faster time to market.

Connected coffee with Azure Sphere 

A great example is Starbucks, who partnered with Microsoft to connect its fleet of coffee machines using the guardian module with Azure Sphere. The guardian module helps businesses quickly securely connect existing equipment without any redesign, saving both time and money.

With IoT-enabled coffee machines, Starbucks collects more than a dozen data points such as type of beans, temperature, and water quality for every shot of espresso. They are also able to perform proactive maintenance on the machines to avoid costly breakdowns and service calls. Finally, they are using the solution to transmit new recipes directly to the machines, eliminating manual processes and reducing costs.

Azure Sphere innovation within Microsoft

Here at Microsoft, Azure Sphere is also being used by the cloud operations team in their own datacenters. With the aim of providing safe, fast and reliable cloud infrastructure to everyone, everywhere, it was an engineer’s discovery of Azure Sphere that started to make their goal of connecting the critical environment systems—the walls, the roof, the electrical system, and mechanical systems that house the datacenters—a reality.

Using the guardian module with Azure Sphere, they were able to move to a predictive maintenance model and better prevent issues from impacting servers and customers. Ultimately it is allowing them to deliver better outcomes for customers and utilize the datacenter more efficiently. And even better, Azure Sphere is giving them the freedom to innovate, create and explore—all on a secure, cost-effective platform.

Partner collaborations broaden opportunities

Throughout it all, enabling this innovation, is our global ecosystem of Microsoft partners that enable us to advance capabilities and bring Azure Sphere to a broad range of customers and applications.

Together, we can provide a more extensive range of options for businesses—from the single chip Wi-Fi solution from MediaTek that meets more traditional needs to other upcoming solutions from NXP and Qualcomm. NXP will provide an Azure Sphere certified chip that is optimized for performance power, and Qualcomm will offer the first cellular-native Azure Sphere chip.

Register today

Register for the Azure Sphere general availability webinar to explore how Azure Sphere works, how businesses are benefiting from it, and how you can use Azure Sphere to create secure, trustworthy IoT devices that enable true business transformation.
Quelle: Azure

New Azure RTOS collaborations with leaders in the semiconductor industry

IoT is reaching mainstream adoption across businesses in all market segments. Our vision is to enable Azure to be the world’s computer, giving businesses real-time visibility into every aspect of their operations, assets, and products. Businesses are harnessing signals from IoT devices of all shapes and sizes, from the very smallest microcontroller units (MCUs) to very capable microprocessor units (MPUs). This presents a great opportunity for collaboration between semiconductor manufacturers with extensive expertise in MCUs/MPUs and Azure IoT, an industry leader in IoT.

It has been nearly one year since we acquired Express Logic and their popular ThreadX RTOS, and last year we announced Azure RTOS that provides customers those capabilities with the leading real-time operating system (RTOS) in the industry.

Today, we’re announcing additional collaborations with industry leaders, which together represent the vast majority of the market for 32-bit MCUs. Their MCUs are embedded into billions of devices from sensors, streetlights, and shipping containers to smart home appliances, medical devices, and more.

STMicroelectronics, Renesas, NXP, Microchip, and Qualcomm will all offer embedded development kits featuring Azure RTOS ThreadX, one of the components of the Azure RTOS embedded application development suite. This allows embedded developers to access reliable, real-time performance for resource-constrained devices, and seamless integration with the power of Azure IoT to connect, monitor, and control a global fleet of IoT assets.

We will also be releasing the full source code for all Azure RTOS components on GitHub, allowing developers to freely explore, develop, test, and adapt Azure RTOS to suit their needs. When developers are ready to take their code into production, the production license will be included automatically if they deploy to any of the supported MCU devices from STMicroelectronics, Renesas, NXP, Microchip, or Qualcomm. If they prefer to use a different device in production, they may contact Microsoft for direct licensing details.

As we work with our semiconductor partners to implement best practices for connected devices, Azure RTOS will include easy-to-use reference projects and templates for connectivity to Azure IoT Hub, Azure IoT Central, Azure IoT Edge Gateways as well as first-class integration with Azure Security Center. Azure RTOS will soon ship with an Azure Security Center module for monitoring threats and vulnerabilities on IoT devices.

When combined with Azure Sphere, Azure RTOS enables embedded developers to quickly build real-time, highly-secured IoT devices for even the most demanding environments—robust devices that offer real-time performance and protection from evolving cybersecurity threats. For MCUs and system on chips (SoCs) that are smaller than what Azure Sphere supports, Azure RTOS and Azure IoT Hub Device Management enable secure communications for embedded developers and device operators who have the ability to implement best practices to protect devices from cybersecurity attacks.

For partners wishing to deliver reliable, real-time performance on highly-secured connected devices that stay secured against evolving cybersecurity threats over time, we recommend Azure RTOS and Azure Sphere together for the most demanding environments.

Here are more details on our collaboration with industry leaders.

STMicroelectronics (ST)

STMicroelectronics (ST) is a renowned world leader in ARM® Cortex®-M MCUs with its STM32 family, providing their OEM and mass-market customers with a wide portfolio of simple-to-use MCUs, coming with a complete development environment and best-in-class ecosystem.

“We are delighted to be collaborating with Microsoft to address even better our customers’ needs,” said Ricardo de Sa Earp, Group Vice-President, Microcontrollers Division General Manager, STMicroelectronics. “Leveraging our installed base of more than five billion STM32 MCUs shipped to date to the global embedded market, we see Azure RTOS ThreadX and middleware as a perfect match to both our mass-market and OEM IoT strategies, complementing our development environment with industry-proven, reliable, high-quality source code.” 

Renesas Electronics Corporation

Renesas Electronics Corporation is a premier supplier of advanced semiconductor solutions. Last October, we announced that Azure RTOS will be broadly available across Renesas' products, including the Synergy and RA MCU families. Renesas is also working to build Azure RTOS into their broader set of MCUs and MPUs.

“Our Synergy and RX cloud kits combined with Azure RTOS and other Azure IoT building blocks offer MCU customers a quick and secure end-to-end solution for cloud connectivity,” said Sailesh Chittipeddi, Executive Vice President, General Manager of Renesas’ IoT and Infrastructure business unit. “We are excited to expand our collaboration with Microsoft and look forward to bringing Microsoft Azure to our MCU and MPU customers, including solutions that will support Azure IoT Edge Runtime for Linux on our RZ MPUs.”

NXP Semiconductors 

NXP Semiconductors is a world leader in secure connectivity solutions for embedded applications, serving customers in the automotive, industrial and IoT, mobile, and communication infrastructure sectors. Microsoft has been collaborating with NXP to extend intelligent cloud computing to the intelligent edge, from adding voice control directly to devices to offering machine learning solutions for edge devices, to device security with Azure Sphere. They plan to integrate Azure RTOS into their evaluation kits and some of the most popular IoT processor families in the industry.

“Edge computing reduces the latency, bandwidth and privacy concerns of a cloud-only Internet of Things," said Jerome Schang, Head of Cloud Partnership programs at NXP. “Enabling Azure RTOS on NXP’s MCUs is yet another step to provide edge computing solutions that unlock the benefits of edge to Azure IoT cloud interaction.”

Microchip Technology, Inc.

Microchip Technology Inc. is a leading provider of smart, connected, and secure embedded control solutions. Their solutions serve customers across the industrial, automotive, consumer, aerospace and defense, communications, and computing markets. Microchip plans to incorporate support for Azure RTOS and Azure IoT Edge across their product families.

“Microchip is building on its already comprehensive portfolio of tools and solutions to enable quick, easy development of secure IoT applications across the full spectrum of embedded control devices and architectures,” said Greg Robinson, associate vice president of Microchip’s 8-bit microcontroller business unit. “Our partnership with Microsoft Azure extends our dedication to developing innovative solutions.”

Qualcomm Technologies, Inc.

Qualcomm is a pioneer of wireless technology and powers the cellular connection of smartphones and tablets all over the planet. Qualcomm will be offering a cellular-enabled Azure Sphere certified chip and will be bringing Azure RTOS to cellular-connected device solutions found inside asset trackers, health monitors, security systems, smart city sensors, and smart meters, as well as a range of wearables.

”Qualcomm is a leader in wireless compute and connectivity technologies – not just in mobile, but in emerging markets like the Internet of Things as well,” said Jeff Torrance, Vice President, IoT, Qualcomm. “We’re proud to continue to work closely with Microsoft on solutions like Azure RTOS and Azure Sphere to jointly advance the IoT industry around the world.”

Learn more

We continue to work diligently with industry-leaders to create a rich, robust ecosystem that serves the world’s unique and diverse needs. Our collective aim is to enable customers to easily bring their ideas to life and truly unlock the opportunities available on the intelligent edge and the intelligent cloud. Find out more about why so many IoT industry leaders are excited about the benefits that Azure RTOS brings to their device solutions.
Quelle: Azure

Announcing server-side encryption with customer-managed keys for Azure Managed Disks

Today, we're announcing the general availability for server-side encryption (SSE) with customer-managed keys (CMK) for Azure Managed Disks. Azure customers already benefit from SSE with platform-managed keys for Managed Disks enabled by default. SSE with CMK improves on platform-managed keys by giving you control of the encryption keys to meet your compliance need.

Today, customers can also use Azure Disk Encryption, which leverages the Windows BitLocker feature and the Linux dm-crypt feature to encrypt Managed Disks with CMK within the guest virtual machine (VM). SSE with CMK improves on Azure Disk encryption by enabling you to use any OS types and images, including custom images, for your VMs by encrypting data in the Azure Storage service.

SSE with CMK is integrated with Azure Key Vault, which provides highly available and scalable secure storage for your keys backed by Hardware Security Modules. You can either bring your own keys (BYOK) to your Key Vault or generate new keys in the Key Vault.

About the key management

Managed Disks are encrypted and decrypted transparently using 256-bit Advanced Encryption Standard (AES) encryption, one of the strongest block ciphers available. The Storage service handles the encryption and decryption in a fully transparent fashion using envelope encryption. It encrypts data using 256-bit AES-based data encryption keys, which are, in turn, protected using your keys stored in a Key Vault.

The Storage service generates data encryption keys and encrypts them with CMK using RSA encryption. The envelope encryption allows you to rotate (change) your keys periodically as per your compliance policies without impacting your VMs. When you rotate your keys, the Storage service re-encrypts the data encryption keys with the new CMK.

Full control of your keys

You are in full control of your keys in your Key Vault. Managed Disks uses system-assigned managed identity in your Azure Active Directory (Azure AD) for accessing keys in Key Vault. An administrator with required permissions in the Key Vault must first grant access to Managed Disks in Key Vault to use the keys for encrypting and decrypting the data encryption key. You can prevent Managed Disks from accessing your keys by either disabling your keys or by revoking access controls for your keys—doing so for disks attached to running VMs will cause the VMs to fail. Moreover, you can track the key usage through Key Vault monitoring to ensure that only Managed Disks or other trusted Azure services are accessing your keys.

Availability of SSE with CMK

SSE with CMK is available for Standard HDD, Standard SSD, and Premium SSD Managed Disks that can be attached to Azure Virtual Machines and VM scale sets. Ultra Disk Storage support will be announced separately. SSE with CMK is now enabled in all the public and Azure Government regions and will be available in the regions in Germany (Sovereign) and China in a few weeks.

You can use Azure Backup to back up your VMs using Managed Disks encrypted with SSE with CMK. Also, you can choose to encrypt the backup data in your Recovery Services vaults using your keys stored in your Key Vault instead of platform-managed keys available by default. Refer to documentation for more details on the encryption of backups using CMK.

You can use Azure Site Recovery to replicate your Azure virtual machines that have Managed Disks encrypted with SSE with CMK to other Azure regions for disaster recovery. You can also replicate your on-premises virtual machines to Managed Disks encrypted with SSE with CMK in Azure. Learn more about replicating your virtual machines using Managed Disks encrypted with SSE with CMK.

Get started

To enable the encryption with CMK for Managed Disks, you must first create an instance of a new resource type called DiskEncryptionSet and then grant the instance access to the key Vault. DiskEncryptionSet represents a key in your Key Vault and allows you to reuse the same key for encrypting many disks, snapshots, and images with the same key.

Let’s look at an example of creating an instance of DiskEncryptionSet:

1. Create an instance of DiskEncryptionSet by specifying a key in your Key Vault.

keyVaultId=$(az keyvault show –name yourKeyVaultName –query [id] -o tsv)

keyVaultKeyUrl=$(az keyvault key show –vault-name yourKeyVaultName –name yourKeyName –query [key.kid] -o tsv)

az disk-encryption-set create -n yourDiskEncryptionSetName -l WestCentralUS -g yourResourceGroupName –source-vault $keyVaultId –key-url $keyVaultKeyUrl

2. Grant the instance access to the Key Vault. When you created the instance, the system automatically created a system-assigned managed identity in your Azure AD and associated the identity with the instance. The identity must have access to the Key Vault to perform required operations such as wrapkey, unwrapkey and get.

desIdentity=$(az disk-encryption-set show -n yourDiskEncryptionSetName -g yourResourceGroupName –query [identity.principalId] -o tsv)

az keyvault set-policy -n yourKeyVaultName -g yourResourceGroupName –object-id $desIdentity –key-permissions wrapkey unwrapkey get

az role assignment create –assignee $desIdentity –role Reader –scope $keyVaultId

You are ready to enable the encryption for disks, snapshots, and images by associating them with the instance of DiskEncryptionSet. There is no restriction on the number of resources that can be associated with the same DiskEncryptionSet.

Let’s look at an example of enabling for an existing disk:

1. To enable the encryption for disks attached to a VM, you must stop(deallocate) a virtual machine.

az vm stop –resource-group MyResourceGroup –name MyVm

2. Enable the encryption for an attached disk by associating it with the instance of DiskEncryptionSet.

diskEncryptionSetId=$(az disk-encryption-set show -n yourDiskEncryptionSetName -g yourResourceGroupName –query [id] -o tsv)

az disk update -n yourDiskEncryptionSetName -g yourResourceGroupName –encryption-type EncryptionAtRestWithCustomerKey –disk-encryption-set $diskEncryptionSetId

3. Start the VM.

az vm start -g MyResourceGroup -n MyVm

Refer to the Managed Disks documentation for detailed instructions on enabling server side encryption with CMK for Managed Disks.

Send us your feedback

We look forward to hearing your feedback for SSE with CMK. Please email us here. 
Quelle: Azure