Microsoft Services is now a Kubernetes Certified Service Provider

Modern applications are increasingly built using containers, which are microservices packaged with their dependencies and configurations. For this reason, many companies are either containerizing their existing applications or creating new complex applications that are composed of multiple containers.

As applications grow to span multiple containers deployed across multiple servers, operating them becomes more complex. To manage this complexity, Kubernetes, an open-source software for deploying and managing those containers at scale, provides an open source API that controls how and where those containers will run.

Kubernetes Certified Service Provider

Microsoft Services is now a Kubernetes Certified Service Provider (KCSP). The KCSP program is a pre-qualified tier of vetted service providers who have deep experience helping enterprises successfully adopt Kubernetes. The KCSP partners offer Kubernetes support, consulting, professional services, and training for organizations embarking on their Kubernetes journey.

We have trained hundreds of consultants on Kubernetes, developed a comprehensive service offering around Kubernetes, and successfully delivered Kubernetes engagements to many customers in all industries, all over the world.

Using our global reach and ecosystem, we empower organizations to put innovation into practice to deliver strategic business outcomes, maximize the value of cloud technology, and drive success through continual support.

Microsoft Services is your partner to enable your organization to leverage container capabilities and frameworks, such as Kubernetes, to adopt modern technologies to increase speed and agility while also maintaining control and good governance.

The Azure Workloads for Containers offering

We recognize a need to help you address your secure infrastructure challenges and requirements. We envision the containers infrastructure to be more than just the containers orchestration layer to include networking, storage, secrets, and Infrastructure as Code (IaC).

Microsoft Services has a full Kubernetes offering, called Azure Workloads for Containers. This offering is composed of several workstreams that focus on the activities and outcomes that are most relevant to our customers. These workstreams provide full flexibility to our customers as each one of them can be selected independently and customized to meet the specific needs of a given project.

Below are the details of these workstreams.

Kubernetes foundation

Design and plan Azure Kubernetes Service (AKS) cluster and shared services.
Implement AKS cluster and shared services.
Deploy application on AKS.
Test application.
Rollout to production. ​

Containers migration

Assess, design, and plan migration.
Migrate the containers-based application(s).
Test the migrated application(s).
Rollout to production.

Kubernetes security hardening

Refactor your security controls for AKS.
Secure your CI/CD pipeline (DevSecOps).
Harden your AKS environment to meet your compliance obligations.
Assist with third-party security product integration.

Kubernetes threat modeling

Build a threat mo​​del based on the AKS cluster and the apps running on it.
Identify threats and mitigations.
Produce clear actions to mitigate the threats.

Application containerization

Create container image(s) for one or multiple applications.
Test the application(s) running as container.
Deploy the application to an AKS cluster in production​.

The offering is aligned to Microsoft’s Cloud Adoption Framework for Azure and focuses primarily on the Adopt: Innovate principle of your cloud journey for Kubernetes.

Learn more

To learn more, have a look at the Azure Workloads for Containers datasheet.
Quelle: Azure

DCsv2-series VM now generally available from Azure confidential computing

Security and privacy are critically important when storing and processing sensitive information in the cloud, from payment transactions, to financial records, personal health data, and more. With the general availability of DCsv2-series VMs, we are ushering in the start of a new level of data protection in Azure.

With more workloads moving to the cloud and more customers putting their trust in Microsoft, the Azure confidential computing team continues to innovate to provide offerings that keep and build upon that trust. Starting with our world-class security researchers, and working closely with industry partners, we are developing new ways to protect data while it’s in use with Azure confidential computing. DCsv2-series VMs can protect the confidentiality and integrity of your data even while it’s processed.

What is confidential computing?

There are ways to encrypt your data at rest and while in transit, but confidential computing protects the confidentiality and integrity of your data while it is in use. Azure is the first public cloud to offer virtualization infrastructure for confidential computing that uses hardware-based trusted execution environments (TEEs). Even cloud administrators and datacenter operators with physical access to the servers cannot access TEE-protected data.

By combining the scalability of the cloud and ability to encrypt data while in use, new scenarios are possible now in Azure, like confidential multi-party computation where different organizations combine their datasets for compute-intensive analysis without being able to access each other’s data. Examples include banks combining transaction data to detect fraud and money laundering, and hospitals combining patient records for analysis to improve disease diagnosis and prescription allocation.

Data protection powered by Intel hardware

Our DCsv2 confidential computing virtual machines run on servers that implement Intel Software Guard Extensions (Intel SGX). Because Intel SGX hardware protects your data and keeps it encrypted while the CPU is processing it, even the operating system and hypervisor cannot access it, nor can anyone with physical access to the server.

Microsoft and Intel are committed to providing best-in-class cloud data protection through our deep ongoing partnership:

“Customers are demanding the capability to reduce the attack surface and help protect sensitive data in the cloud by encrypting data in use. Our collaboration with Microsoft brings enterprise-ready confidential computing solutions to market and enables customers to take greater advantage of the benefits of cloud and multi-party compute paradigms using Intel® SGX technology.” —Anil Rao, VP Data Center Security and Systems Architecture, Intel

Partners in the Azure Marketplace

Microsoft works directly with platform partners to provide seamless solutions, development, and deployment experiences running on top of our Azure confidential computing infrastructure. Software offerings can be discovered through our Azure Marketplace including:

Fortanix—Offers a cloud-native data security solution including key management, HSM, tokenization, and secrets management built on Azure confidential computing.
Anjuna—Delivers secure Azure instances using end-to-end CPU hardware-level encryption without changing your application or operations.
Anqlave—A valued partner in Singapore, offers enterprise ready confidential computing solutions.

“Anqlave’s proprietary, institutional-grade modern key management and data encryption solution addresses the most critical security issues we face today. With Anqlave Data Vault (ADV) secret management allows users to securely create, store, transport and use its secrets. Leveraging Azure confidential computing, allows us to make this technology more accessible to our enterprise customers and easily support their scale. Providing a secure enclave that is portable in the cloud is one the key reasons why our enterprises will prefer to host their ADV on Azure confidential computing regardless of their other cloud infrastructure.” —Assaf Cohen, CEO, Anqlave

How customers are succeeding with Azure confidential computing

Customers are already using Azure confidential computing for production workloads. One customer is Signal:

“Signal develops open source technology for end-to-end encrypted communications, like messaging and calling. To meet the security and privacy expectations of millions of people every day, we utilize Azure confidential computing to provide scalable, secure environments for our services. Signal puts users first, and Azure helps us stay at the forefront of data protection with confidential computing.” —Jim O’Leary, VP of Engineering, Signal

While many applications and services can take advantage of data protection with confidential computing, we have seen particular benefits with regulated industries, such as financial, government, and healthcare. Companies can now take advantage of the cloud for processing sensitive customer data with reduced risk and higher confidence that their data can be protected, including when processing.

For example, MobileCoin, a new international cryptocurrency trusts Azure confidential computing to support digital currency transfers. Their network code is now available in open source, and a TestNet is available to tryout:

“MobileCoin partners with Azure because Microsoft has decided to invest in trustworthy systems. Confidential computing rides the edge between what we can imagine and what we can protect. The praxis we’ve experienced with Azure allows us to commit to systems that are integral, high trust, and performant.” —Joshua Goldbard, CEO, MobileCoin

Confidential computing has proven useful for enterprise-grade blockchain, enabling fast and secure transaction verification across a decentralized network. Fireblocks is yet another customer taking advantage of Azure confidential computing infrastructure:

“At Fireblocks, our mission is to secure blockchain-based assets and transactions for the financial industry. Once we realized the traditional tech stack was not suitable for this challenge, we turned to Azure confidential computing and Intel SGX to implement our patent-pending technology. Our customers trust Fireblocks to securely store and move their digital assets—over $6.5 billion of them each month—and Azure provides a backbone for us to deliver on that promise.” —Michael Shaulov, CEO and co-founder, Fireblocks

Industry leadership bringing confidential computing to the forefront

Microsoft is not alone in bringing confidential computing to the forefront of the cloud computing industry. In September 2019, we were a founding member of the Confidential Computing Consortium (CCC), which now consists of dozens of companies working to develop and open source technologies and best practices for protecting data while it’s in use. These companies include hardware, cloud, platform, and software providers.

Microsoft is also committed to the developer experience to ensure platform partners and application developers can build solutions that take advantage of confidential computing. We donated our Open Enclave SDK to the consortium, an open source SDK for developing platforms and applications on top of confidential computing infrastructure.

Get started today

Get started deploying your own DCsv2 virtual machine from the Azure Marketplace and install necessary tools. Then, run the Hello World sample using the Open Enclave SDK to begin building confidential workloads in the cloud.
Quelle: Azure

Update #3: Business continuity with Azure

Thank you for your response to our cloud continuity blogs; many of you have told us that this information is helpful. We’re committed to providing further posts when we have additional information.

Here at Microsoft, as most of our company starts the seventh week in changed professional and personal arrangements, we are learning new ways to live, work, learn and communicate. We are also learning from you—our customers and partners. We are all adjusting in this moment together and are appreciative of the feedback we receive and the confidence our customers have in our wide range of cloud services.

As a technology first responder serving first responders battling the global health crisis, as a trusted cloud provider to ensure your technology investment continues to deliver the value you expect, and as a company committed to assisting as organizations adapt to changing needs—we are relentlessly focused on providing the support needed to help the workforce operate as smoothly as possible during these changing times.

To ensure optimum focus, our efforts continue to be anchored in two key areas of action:

Help our customers address their most urgent needs.
Ensure Microsoft Azure continues to scale to meet new demand.

The rest of this post shares insights into the work we have done to support those two areas of continuity for organizations, businesses, and the people within them, around the world.

Helping our customers address their most urgent needs

Across our portfolio of cloud services, we work with a diverse group of global customers and organizations. Although their fields of work and customer needs are unique, there is consistency in what they’re looking for from cloud providers. Remote work, distance learning, real-time insights, and analytics have all been common themes of when it comes to the most pressing needs during this time.

Some examples of this work in action:

As businesses and schools around the world prioritize the safety and well-being of their employees and students, Microsoft Teams, which runs on Azure, is playing a critical role in helping them stay connected through video meetings, calls, and chats. We’ve seen a new daily record of 2.7 billion meeting minutes in one day. One of the organizations using Teams is St. Luke’s University Health Network. St. Luke’s University Health network serves approximately 1 million people across 10 counties in Pennsylvania and New Jersey. In a matter of weeks, they transformed the way they work and deliver patient care through Teams, and since mid-March have completed over 75,000 virtual patient visits. This allowed them to continue critical outpatient visits while protecting both patients and physicians from COVID-19 exposure and preserving valuable resources like masks and gloves. Tablets have also been installed in patient rooms so providers can engage with infected patients via Teams, minimizing exposure while still allowing for face-to-face connections between patients and caregivers.

HoloLens 2 and Dynamics 365 Remote Assist are being used on the front lines by nurses and doctors (like Dr. Thomas Gregory) to maintain social distancing and minimize interactions all while ensuring expert support of patients via remote participation of support staff and access to valuable patient data and health records. And for the first time ever, instead of working together on campus, all 185 first-year students from Case Western Reserve University’s School of Medicine are using HoloLens and the university’s signature HoloAnatomy mixed-reality software, in light of the need for physical separation during the pandemic.

Hundreds of healthcare providers have installed the Power Platform Emergency Response Solution for hospitals, which was developed with Swedish Health Services in the Seattle area to analyze and improve resource tracking and decision support tools for hospital administrators.

Our Nonprofit Data Warehouse Quickstart efforts are helping nonprofits easily deploy Azure analytics services such as Azure Synapse Analytics and with prebuilt Power BI templates by integrating sample datasets such as the World Health Organization Water and Sanitation data repository, data that is aligned to the International Aid Transparency Initiative (IATI) data standard, and the Common Data Model for Nonprofits.

We recently announced the Dynamics 365 Healthcare Accelerator Patient Scheduling and Screening Template—a tool designed to help healthcare organizations address large volumes of patient requests with higher efficiency. The template provides access to a portal with information about COVID-19, an easy-to-use self-assessment tool for patients to determine risk, and an automated process for booking and performing COVID-19 screening.

Emergency Medical Services Copenhagen provides emergency care for about one-third of Denmark’s population. Shortly after the COVID-19 outbreak calls to its emergency lines almost doubled, with around 2,000 calls daily by early March from worried people showing symptoms of COVID-19 or having questions about the disease. Emergency Medical Services Copenhagen is now one of many healthcare organizations in Europe and beyond using Microsoft’s Healthcare Bot service to help screen people for potential coronavirus infection and treatment.

Ensuring Azure continues to scale to meet new demand

The impact of the current pandemic is a great example of how cloud computing can rapidly meet new challenges. All of Microsoft’s cloud services including Teams and other Microsoft 365 products, Dynamics 365 and Azure were put to the test during these unprecedented and uncertain times. We are incredibly proud to be serving our customers, like those mentioned above, through this time and we also acknowledge that it hasn’t all been without issue. We look to continuously improve our design and operations to account for all circumstances. Before we share the improvements we’re making, here’s some background on how we build and operate Azure.

Azure has been designed to quickly scale to meet surges in demand when they occur. Over the past few years, we have seen phenomenal demand for Azure services. To keep up with this demand, we have continued to expand our datacenter footprint—with 58 datacenter regions around the world. To manage the normal high growth we have come to expect, we design and source our own infrastructure components, (and share our designs back to the community through the Open Compute Project), and closely manage our strategic demand and supply chain forecasting models. In general, in any particular Azure region we ensure a near-instant capacity buffer within the datacenters, and hold additional infrastructure buffer warehoused, ready to ship to regions with high demand.

Last month, the surging use of Teams for remote work and education due to the pandemic crossed into unprecedented territory. Although we had seen surges in specific datacenter regions or wider geographies before, such as in response to natural disasters, the substantial Teams demand increase from Asia and then quickly followed in Europe indicated that we were seeing something very different, and increasingly global. Without knowing the true scale of the new demand, we took a cautious approach and put in place temporary resource limits on new Azure subscriptions. (Existing customer subscriptions did not experience these restrictions as each Azure customer account has a defined quota of services they can access.) This allowed us to continue to meet the promised quota for all existing Azure customers, prioritize new needs for life and safety organizations on the front lines of the pandemic response and support the dramatic shift to remote work and education on Teams.

As this surge in Teams demand occurred, we quickly took steps towards managing increased cloud infrastructure and network demand including:

Optimized and load-balanced the Teams architecture and quickly rolled out these improvements worldwide (using Azure DevOps), without interrupting the customer experience. This work is durable such that we can manage Teams rapid growth moving forward without creating pressure on Azure customers’ capacity needs.
Expediting additional server capacity to the specific regions that faced constraints, while ensuring the safety and health of our datacenter staff and supply chain partners.
Approving the backlog of customer quota requests, which we are rapidly doing every day and are on track to complete over the next few weeks in almost all regions.
Removing restrictions for new free and benefit subscriptions in several regions, so that anyone can learn more about Azure’s capabilities and develop new skills.
Refining our Azure demand models. Our data science models are using what we’ve learned from this pandemic to better forecast future demands, including adding more support to handle future global events like a pandemic that drives simultaneous demand usage everywhere in the world.

We remain committed to operational excellence and we will continue to share what we are learning and doing to support everyone during this time.
Quelle: Azure

Azure Migrate now available in Azure Government

Microsoft’s service for datacenter migration, Azure Migrate, is now available in Azure Government—unlocking the whole range of functionality for government customers. Previously, Azure Migrate V1 was available to US Azure Government customers, which performed limited scale assessment for VMware workloads. Azure Migrate V2 for Azure Government, now available, includes a one-stop shop for discovery, assessment, and migration of largescale datacenters.Why migrate to Azure GovernmentWe know how important security is for Government customers. Fortunately, Azure Government, Microsoft’s government cloud offering, provides industry-leading security with more compliance certifications than any other cloud provider. By using a cloud government solution, your organization can meet high compliance certifications that aren’t available on-premises. Azure Government has six government-exclusive datacenter regions across the US, with an Impact Level 5 Provisional Authorization. This means Azure Government can host workloads for the most sensitive organizations, like the US Department of Defense. Azure Government also offers hybrid flexibility, which allows you to customize your digital transformation by keeping select data and functionality on-premises. Leading-edge innovations in Azure ensure your government organization is modernized and effective, with advanced data analytics, artificial intelligence (AI), IoT, and high-performance computing. Transform how your organization learns from and interacts with citizens. Analyze smart devices real-time to improve weather sensors and optimize emergency services. Take preemptive action against evolving security threats with predictive models. Learn more about Azure Government.Azure Migrate supports your migration to Azure GovernmentAzure Migrate provides a central hub of Microsoft and ISV migration tools. The hub helps identify the right tools for your migration scenario and features end-to-end progress tracking to help with largescale datacenter migrations and cloud transformation projects. Azure Migrate provides comprehensive coverage for a variety of migration scenarios, now all available for government customers, including: Windows and Linux servers—Largescale discovery, assessment, and migration for VMware, Hyper-V, and bare metal servers. Features include agentless discovery, application inventory mapping, dependency mapping, and cost analysis. You can also migrate VMware VMs (now generally available) to Azure with zero data loss and minimal downtime using an agentless migration, in addition to the agent-based migration capability.SQL and other databases—Assessment and migration for a variety of on-premises databases to Azure SQL database and Azure SQL Database managed instance. Web-apps—Assessment and migration of .NET and PHP web apps to Azure App Service.Virtual Desktop Infrastructure (VDI) migration—Migration of virtual desktop infrastructure to Windows Virtual Desktop in Azure.Data migration—Migration of block data to Azure using Data Box.Azure Migrate Hub.Agentless Discovery. Dependency Mapping. Learn more about Azure Migrate.Geographic and regional availability for Azure MigrateAzure Migrate is currently available in Asia Pacific, Australia, Canada, Europe, India, Japan, United Kingdom, and United States for public cloud. Now, Azure Migrate capabilities will be extended to US Gov Arizona and US Gov Virginia for government customers. Note the individual SKUs supported in the assessment and migration tools will depend on availability in these regions. See a comparison of Gov SKUs with respect to public cloud SKUs. Get started with Azure Migrate for GovernmentAs always, Azure Migrate is included in your Azure subscription without any additional licensing costs. To get started with Azure Government, request an Azure Government trial. If you already have an Azure Government subscription,  you can get started using Azure Migrate to discover, assess, and migrate your mission critical workloads to Azure. You can learn how to get started with Azure Migrate and access tutorials in the Azure Migrate documentation.We are thrilled to empower our customers to be future ready and leverage the continuous innovation of Azure. You can see the latest and greatest Azure Migrate capabilities in action in the videos below. Get started with Azure Migrate Migrate VMware VMs to Azure How to discover, assess, and migrate Hyper-V VMs to Azure
Quelle: Azure

Optimize cost and performance with Query Acceleration for Azure Data Lake Storage

The explosion of data-driven decision making is motivating businesses to have a data strategy to provide better customer experiences, improve operational efficiencies, and make real-time decisions based on data. As businesses become data driven, we see more customers build data lakes on Azure. We also hear that more cost optimization and more performance are two of the most important features of data lake architecture on Azure. Normally, these two qualities are traded off for each other—if you want more performance, you will need to pay more; if you want to save money, expect your performance curve to go down.

That’s why today, we’re announcing the preview of Query Acceleration for Azure Data Lake Storage—a new capability of Azure Data Lake Storage, which improves both performance and cost. The feature is now available for customers to start realizing these benefits and improving their data lake deployment on Azure.

How Query Acceleration for Azure Data Lake improves performance and cost

Big data analytics frameworks, such as Spark, Hive, and large-scale data processing applications, work by reading all of the data using a horizontally-scalable distributed computing platform with techniques such as MapReduce. However, a given query or transformation generally does not require all of the data to achieve its goal. Therefore, applications typically incur the costs of reading, transferring over the network, parsing into memory and finally filtering out the majority of the data that is not required. Given the scale of such data lake deployments, these costs become a major factor that impacts the design and how ambitious you can be. Improving cost and performance at the same time enhances how much valuable insight you can extract from your data.

Query Acceleration for Azure Data Lake Storage allows applications and frameworks to push-down predicates and column projections, so they may be applied at the time data is first read, meaning that all downstream data handling is saved from the cost of filtering and processing unrequired data.

The following diagram illustrates how a typical application uses Query Acceleration to process data:

The client application requests file data by specifying predicates and column projections.
Query Acceleration parses the specified query and distributes work to parse and filter data.
Processors read the data from the disk, parses the data by using the appropriate format, and then filters data by applying the specified predicates and column projections.
Query Acceleration combines the response shards to stream back to client application.
The client application receives and parses the streamed response. The application doesn't need to filter any additional data and can apply the desired calculation or transformation directly.

Azure offers powerful analytic services

Query Acceleration for Azure Data Lake Storage is yet another example of how we’re committed to making Azure the best place for organizations to unlock transformational insights from all data. Customers can benefit from tight integration with other Azure Services for building powerful cloud scale end-to-end analytics solutions. These solutions support modern data warehousing, advanced analytics, and real-time analytics easily and more economically.

We’re also committed to remaining an open platform where the best-in-breed open source solutions benefit equally from the innovations occurring at all points within the platform. With Azure Data Lake Storage underpinning an entire ecosystem of powerful analytics services, customers can extract transformational insights from all data assets.

Learn more

To find out more about Query Acceleration for Azure Data Lake Storage you can:

Sign up for the Azure Data Lake Storage preview program.
Read the Azure Data Lake Storage documentation.
Learn how to use Query Acceleration for Java and .NET.
Understand the pricing model for Query Acceleration.
Learn more about Azure Data Lake Storage.

Quelle: Azure

Azure GPUs with Riskfuel’s technology offer 20 million times faster valuation of derivatives

Exchange-traded financial products—like stocks, treasuries, and currencies—have had the benefit of a tremendous wave of technological innovation in the past 20 years, resulting in more efficient markets, lower transaction costs, and greater transparency to investors.

However, large parts of the capital markets have been left behind. Valuation of instruments composing the massive $500 trillion market in over-the-counter (OTC) derivatives—such as interest rate swaps, credit default swaps, and structured products—lack the same degree of immediate clarity that is enjoyed by their more straightforward siblings.

In times of increased volatility, traders and their managers need to know the impacts of market conditions on a given instrument as the day unfolds to be able to take appropriate action. Reports reflecting the conditions at the previous close of business are only valuable in calm markets and even then, firms with access to fast valuation and risk sensitivity calculations have a substantial edge in the marketplace.

Unlike exchange-traded instruments, where values can be observed each time the instrument trades, values for OTC derivatives need to be computed using complex financial models. The conventional means of accomplishing this is through traditional Monte Carlo—a simple but computationally expensive probabilistic sweep through a range of scenarios and resultant outcomes- or finite-difference analysis.

Banks spend tens of millions of dollars annually to calculate the values of their OTC derivatives portfolios in large, nightly batches. These embarrassingly parallel workloads have evolved directly from the mainframe days to run on on-premise clusters of conventional, CPU-bound workers—delivering a set of results good for a given day.

Using conventional algorithms, real-time pricing, and risk management is out of reach. But as the influence of machine learning extends into production workloads, a compelling pattern is emerging across scenarios and industries reliant on traditional simulation. Once computed, the output of traditional simulation can be used to train DNN models that can then be evaluated in near real-time with the introduction of GPU acceleration.

We recently collaborated with Riskfuel, a startup developing fast derivatives models based on artificial intelligence (AI), to measure the performance gained by running a Riskfuel-accelerated model on the now generally available Azure ND40rs_v2 (NDv2-Series) Virtual Machine instance powered by NVIDIA GPUs against traditional CPU-driven methods.

Riskfuel is pioneering the use of deep neural networks to learn the complex pricing functions used to value OTC derivatives. The financial instrument chosen for our study was the foreign exchange barrier option.

The first stage of this trial consisted of generating a large pool of samples to be used for training data. In this instance, we used conventional CPU-based workers to generate 100,000,000 training samples by repeatedly running the traditional model with inputs covering the entire domain to be approximated by the Riskfuel model. The traditional model took an average of 2250 milliseconds (ms) to generate each valuation. With the traditional model, the valuation time is dependent on the maturity of the trade.

The histogram in Figure 1 shows the distribution of valuation times for a traditional model:

 

Figure 1: Distribution of valuation times for traditional models.

Once the Riskfuel model is trained, valuing individual trades is much faster with a mean under 3 ms, and is no longer dependent on maturity of the trade:

Figure 2: Riskfuel model demonstrating valuation times with a mean under 3 ms.

These results are for individual valuations and don’t use the massive parallelism that the Azure ND40rs_v2 Virtual Machine can deliver when saturated in a batch inferencing scenario. When called upon to value portfolios of trades, like those found in a typical trading book, the benefits are even greater. In our study, the combination of a Riskfuel-accelerated version of the foreign exchange barrier option model and with an Azure ND40rs_v2 Virtual Machine showed a 20M+ times performance improvement over the traditional model.

In Figure 3 shows the throughput, as measured in valuations per second, of the traditional model running on a non-accelerated Azure Virtual Machine versus the Riskfuel model running on an Azure ND40rs_v2 Virtual Machine (in blue):

 

Figure 3: Model comparison of traditional model running versus the Riskfuel model.

For portfolios with 32,768 trades, the throughput on an Azure ND40rs_v2 Virtual Machine is 915,000,000 valuations per second, whereas the traditional model running on CPU-based VMs has a throughput of just 32 valuations per second. This is a demonstrated improvement of more than 28,000,000x.

It is critical to point out here that the speedup resulting from the Riskfuel model does not sacrifice accuracy. In addition to being extremely fast, the Riskfuel model effectively matches the results generated by the traditional model, as shown in Figure 4:

 

Figure 4: Accuracy of Riskfuel model.

These results clearly demonstrate the potential of supplanting traditional on-premises high-performance computing (HPC) simulation workloads with a hybrid approach: using traditional methods in the cloud as a methodology to produce datasets used to train DNNs that can then evaluate the same set of functions in near real-time.

The Azure ND40rs_v2 Virtual Machine is a new addition to the NVIDIA GPU-based family of Azure Virtual Machines. These instances are designed to meet the needs of the most demanding GPU-accelerated AI, machine learning, simulation, and HPC workloads, and the decision to use the Azure ND40rs_v2 Virtual Machine was to take full advantage of the massive floating point performance it offers to achieve the highest batch-oriented performance for inference steps, as well as the greatest possible throughput for model training.

The Azure ND40rs_v2 Virtual Machine is powered by eight NVIDIA V100 Tensor Core GPUs, each with 32 GB of GPU memory, and with NVLink high-speed interconnects. When combined, these GPUs deliver one petaFLOPS of FP16 compute.

Riskfuel’s Founder and CEO, Ryan Ferguson, predicts the combination of Riskfuel accelerated valuation models and NVIDIA GPU-powered VM instances on Azure will transform the OTC market:

“The current market volatility demonstrates the need for real-time valuation and risk management for OTC derivatives. The era of the nightly batch is ending. And it’s not just the blazing fast inferencing of the Azure ND40rs_v2 Virtual Machine that we value so much, but also the model training tasks as well. On this fast GPU instance, we have reduced our training time from 48 hours to under four! The reduced time to train the model coupled with on-demand availability maximizes the productivity of our AI engineering team.”

Scotiabank recently implemented Riskfuel models into their leading-edge derivatives platform already live on the Azure GPU platform with NVIDIA GPU-powered Azure Virtual Machine instances. Karin Bergeron, Managing Director and Head of XVA Trading at Scotiabank, sees the benefits of Scotia’s new platform:

“By migrating to the cloud, we are able to spin up extra VMs if something requires some additional scenario analysis. Previously we didn’t have access to this sort of compute on demand. And obviously the performance improvements are very welcome. This access to compute on demand helps my team deliver better pricing to our customers.”

Additional resources

Learn more about Azure NDv2-Series Virtual Machines.
Explore Azure HPC.
Learn more about Riskfuel solutions.

Quelle: Azure

Enhanced features in Azure Archive Storage now generally available

Since launching Azure Archive Storage, we've seen unprecedented interest and innovative usage from a variety of industries. Archive Storage is built as a scalable service for cost-effectively storing rarely accessed data for long periods of time. Cold data, including application backups, healthcare records, autonomous driving recordings, and other data sets that might have been previously deleted could be stored in Azure Storage’s Archive tier in an offline state, then rehydrated to an online tier when needed.

With your usage and feedback, we’ve made our archive improvements generally available, making our service even better.

Priority retrieval from Azure Archive

Priority retrieval allows you to flag the rehydration of your data from the offline archive tier back into an online hot or cool tier as a high priority action. By paying a little bit more for the priority rehydration operation, your archive retrieval request is placed in front of other requests and your offline data is expected to be returned online in less than one hour.

The two archive retrieval options are:

Standard priority is the default option for archive Set Blob Tier and Copy Blob requests, with retrievals taking up to 15 hours.
High priority fulfills the need for urgent data access from archive, with retrievals for blobs under 10 GB typically taking less than 1 hour.

Priority retrieval is recommended to be used for emergency requests for a subset of an archive dataset. For the majority of use cases, our customers plan for and utilize standard archive retrievals which complete in less than 15 hours. On rare occasions, a retrieval time of an hour or less is required for business continuity. Priority retrieval requests can deliver archive data in a fraction of the time of a standard retrieval operation, allowing our customers to quickly resume business as usual. For more information, please see the Azure Blob rehydration documentation.

Upload blob direct to access tier of choice (hot, cool, or archive)

You can upload your blob data using PutBlob or PutBlockList directly to the access tier of your choice using the optional parameter x-ms-access-tier. This allows you to upload your object directly into the hot, cool, or archive tier regardless of your account’s default access tier setting. This capability makes it simple for customers to upload objects directly to Azure Archive in a single transaction. Then, as data usage patterns change, you would change the access tier of the blob manually with the Set Blob Tier API or automate the process with blob lifecycle management rules. For more information, please see the Azure Blob storage access tiers documentation.

Copy Blob enhanced capabilities

In certain scenarios, you may want to keep your original data untouched but work on a temporary copy of the data. The Copy Blob API is now able to support the archive access tier; allowing you to copy data into and out of the archive access tier within the same storage account. With our access tier of choice enhancement, you can set the optional parameter x-ms-access-tier to specify which destination access tier you would like your data copy to inherit. If you are copying a blob from the archive tier, you can also specify the x-ms-rehydrate-priority of how quickly you want the copy created in the destination hot or cool tier. Please see the Azure Blog rehydration documentation for more information.

Getting started

All of the features discussed today (upload blob direct to access tier, priority retrieval from archive, and Copy Blob enhancements) are supported by the most recent releases of the Azure Portal, AzCopy, .NET Client Library, Java Client Library, Python Client Library, and Storage Services REST API (version 2019-02-02 or higher). In general, we always recommend using the latest version of our tools and SDKs.

In addition to our first party tools, Archive Storage has an extensive network of partners who can help you discover and retain value from your data. As we improve our service with new features, we're also working to build our ecosystem and onboard additional partners. Please visit the Azure update to see the latest additions to our partner network.

Build it, use it, and tell us about it!

We will continue to improve our Archive and Blob Storage services and are looking forward to hearing your feedback about these features through email. As a reminder, we love hearing all of your ideas and suggestions about Azure Storage, which you can post at Azure Storage feedback forum.
Quelle: Azure

Azure Maps updates offer new features and expanded availability

This blog post was co-authored by Chad Raynor, Principal Program Manager, Azure Maps.

Updates to Azure Maps services include new and recently added features, including the general availability of Azure Maps services on Microsoft Azure Government cloud. Here is a rundown of the new and recently added features for Azure Maps services:

Azure Maps is now generally available on Azure Government cloud

The general availability of Azure Maps for Azure Government cloud allows you to easily include geospatial and location intelligence capabilities in solutions deployed on Azure Government cloud with the quality, performance, and reliability required for enterprise grade applications. Microsoft Azure Government delivers a cloud platform built upon the foundational principles of security, privacy and control, compliance, and transparency. Public sector entities receive a physically isolated instance of Microsoft Azure that employs world-class security and compliance services critical to the US government for all systems and applications built on its architecture.

Azure Maps Batch services are generally available

Azure Maps Batch capabilities available through Search and Route services are now generally available. Batch services allows customers to send batches of queries using just a single API request.

Batch capabilities are supported by the following APIs:

Post Search Address Batch
Post Search Address Reverse Batch
Post Search Fuzzy Batch
Post Route Directions Batch

What’s new for the Azure Maps Batch services?

Users have now an option to submit synchronous (sync) request, which is designed for lightweight batch requests. When the service receives a request, it will respond as soon as the batch items are calculated instead of returning a 202 along with a redirect URL. With sync API there will be no possibility to retrieve the results later. When Azure Maps receives sync request, it responds as soon as the batch items are calculated. For large batches, we recommend continuing to use the Asynchronous API that is appropriate for processing big volumes of relatively complex route requests.

For Search APIs, the Asynchronous API allows developers to batch up to 10,000 queries and sync API up to 100 queries. For Route APIs, the Asynchronous API allows developers to batch up to 700 queries and sync API up to 100 queries.

Azure Maps Matrix Routing service is generally available

The Matrix Routing API is now generally available. The service allows calculation of a matrix of route summaries for a set of routes defined by origin and destination locations. For every given origin, the service calculates the travel time and distance of routing from that origin to every given destination.

For example, let's say a food delivery company has 20 drivers and they need to find the closest driver to pick up the delivery from the restaurant. To solve this use case, they can call Matrix Route API.

What’s new in the Azure Maps Matrix Routing service?

The team worked to improve the Matrix Routing performance and added support to submit synchronous request like for the batch services described above. The maximum size of a matrix for asynchronous request is 700 and for synchronous request it's 100 (the number of origins multiplied by the number of destinations).

For Asynchronous API calls we introduced new waitForResults parameter. If this parameter is set to be true, user will get a 200 response if the request is finished under 120 seconds. Otherwise, user will get a 202 response right away and async API will return users an URL to check the progress of async request in the location header of the response.

Updates for Render services

Introducing Get Map tile v2 API in preview

Like Azure Maps Get Map Tiles API v1, our new Get Map Tile version 2 API, in preview, allows users to request map tiles in vector or raster format typically to be integrated into a map control or SDK. The service allows to request various map tiles, such as Azure Maps road tiles or real-time Weather Radar tiles. By default, Azure Maps uses vector map tiles for its SDKs.

The new version will offer users more consistent way to request data. The new version introduces a concept of tileset, a collection of raster or vector data that are further broken up into a uniform grid of square tiles at preset zoom levels. Every tileset has a tilesetId to request a specific tileset. For example, microsoft.base.

Also, Get Map Tile v2now supports the option to call imagery data that was earlier only available through Get Map Imagery Tile API. In addition, Azure Maps Weather Service radar and infrared map tiles are only available through the version 2.

Dark grey map style available through Get Map Tile and Get Map Image APIs

In addition to serve the Azure Maps dark grey map style through our SDKs, customers can now also access it through Get Map Tile APIs (version 1 and version 2) and Get Map Image API in vector and raster format. This empowers customers to create rich map visualizations, such as embedding a map image into a web page.

Azure Maps dark grey map style.

Route service: Avoid border crossings, pass in custom areas to avoid

The Azure Maps team has continued to make improvements to the Routing APIs. We have added new parameter value avoid=borderCrossings to support routing scenarios where vehicles are required to avoid country/region border crossings, and keep the route within one country.

To offer more advanced vehicle routing capabilities, customers can now include areas to avoid in their POST Route Directions API request. For example, a customer might want to avoid sending their vehicles to a specific area because they are not allowed to operate in the area without a permission form the local authority. As a solution, users can now pass in the route request POST body polygons in GeoJSON format as a list of areas to avoid.

Cartographic and styling updates

Display building models

Through Azure Maps map control, users have now option to render 2.5D building models on the map. By default, all buildings are rendered as just their footprints. By setting showBuildingModels to true, buildings will be rendered with their 2.5D models. Try the feature now.

Display building models.

Islands, borders, and country/region polygons

To improve the user experience and give more detailed views, we reduced the boundary data simplification reduction to offer better visual experience at higher zoom levels. User can now see more detailed polygon boundary data.

Left: Before boundary data simplification reduction. Right: After boundary data simplification reduction.

National Park labeling and data rendering

Based on feedback from our users, we simplified labels for scatters polygons by reducing the number of labels. Also, National park and National Forest labels are displayed already on zoom level 6.

National Park and National Forest labels displayed on zoom level 6.

Send us your feedback

We always appreciate feedback from the community. Feel free to comment below, post questions to Stack Overflow, or submit feature requests to the Azure Maps Feedback UserVoice.
Quelle: Azure

Azure Security Center enhancements

At Microsoft Ignite 2019, we announced the preview of more than 15 new features. This blog provides an update for the features that are now generally available to our customers.

As the world comes together to combat COVID-19, and remote work becomes a critical capability for many companies, it’s extremely important to maintain the security posture of your cloud assets while enabling more remote workers to access them.

Azure Security Center can help prioritize the actions you need to take to protect your security posture and provide threat protection for all your cloud resources.

Enhanced threat protection for your cloud resources with Azure Security Center

Azure Security Center continues to extend its threat protection capabilities to counter sophisticated threats on cloud platforms:

Scan container images in Azure Container Registry for vulnerabilities generally available

Azure Security Center can scan container images in Azure Container Registry (ACR) for vulnerabilities.

The image scanning works by parsing through the packages or other dependencies defined in the container image file, then checking to see whether there are any known vulnerabilities in those packages or dependencies (powered by a Qualys vulnerability assessment database).

The scan is automatically triggered when pushing new container images to Azure Container Registry. Found vulnerabilities will surface as Security Center recommendations and be included in the Secure Score together with information on how to patch them to reduce the attack surface they allowed.

Since we launched the preview at Ignite 2019, registered subscriptions initiated over 1.5 million container image scans. We have carefully analyzed the feedback we received and incorporated it into this generally available version. We have added scanning status to reflect the progress of the scan (Unscanned, Scan in progress, Scan error, and Completed) and improved the user experience based on the feedback we've received from you.

Threat protection for Azure Kubernetes Service Support in Security Center generally available

The popular, open source, platform Kubernetes has been adopted so widely that it is now an industry standard for container orchestration. Despite this widespread implementation, there’s still a lack of understanding regarding how to secure a Kubernetes environment. Defending the attack surfaces of a containerized application requires expertise. You need to ensure the infrastructure is configured securely, and constantly monitor for potential threats. Support for Security Center Azure Kubernetes Service (AKS) is now generally available.
 
The capabilities include: 

Discovery and Visibility: Continuous discovery of managed AKS instances within Security Center’s registered subscriptions.
Secure Score recommendations: Actionable items to help customers comply with security best practices in AKS as part of the customer’s Secure Score, such as "Role-Based Access Control should be used to restrict access to a Kubernetes Service Cluster."
Threat Protection: Host and cluster-based analytics, such as “A privileged container detected.”

For the generally available release, we've added new alerts (for the full list please visit Alerts for Azure Kubernetes Service clusters and Alerts for containers – host level sections of the alerts reference table), and alert details were fine tuned to reduce false positives.
  

Cloud security posture management enhancements

Misconfiguration is the most common cause of security breaches for cloud workloads. Azure Security Center provides you with a bird’s eye security posture view across your Azure environment, enabling you to continuously monitor and improve your security posture using the Azure Secure Score. Security Center helps manage and enforce your security policies to identify and fix such misconfigurations across your different resources and maintain compliance. We continue to expand our resource coverage and the depth insights that are available in security posture management.

Support for custom policies generally available

Our customers have been wanting to extend their current security assessments coverage in Security Center with their own security assessments based on policies that they create in Azure Policy. With support for custom policies, this is now possible.

We're also announcing that Azure Security Center’s support for custom policies is generally available. These new policies will be part of the Azure Security Center recommendations experience, Secure Score, and the regulatory compliance standards dashboard. With the support for custom policies, you are now able to create a custom initiative in Azure policy and add it as a policy in Azure Security Center through a simple click-through onboarding experience and visualize them as recommendations.

For this release, we've added the ability to edit the custom recommendation metadata to include severity, remediation steps, threat information, and more.

Assessment API generally available

We are introducing a new API to get Azure Security Center recommendations with information and provide you the reason why assessments failed. The new API includes two APIS:

Assessments metadata API: Gets recommendation metadata.
Assessments API: Provides the assessment results of each recommendation on a resource.

We advise that our customers using the existing Tasks API should use the new Assessments API for their reporting.

Regulatory compliance dynamic compliance packages generally available  

You can now add ‘dynamic compliance packages,’ or additional standards beyond the ‘built-in’ compliance packages in regulatory compliance.

The regulatory compliance dashboard in Azure Security Center provides insights into your compliance posture relative to a set of industry standards, regulations, and benchmarks. Assessments continually monitor the security state of your resources and are used to analyze how well your environment is meeting the requirements for specific compliance controls. Those assessments also include actionable recommendations for how to remediate the state of your resources and thus improve your compliance status.

Initially, the compliance dashboard included a very limited set of standards that were ‘built-in’ to the dashboard and relied on a static set of rules included with Security Center. With the dynamic compliance packages feature, you can add new standards and benchmarks that are important to you to your dashboard. Compliance packages are essentially initiatives defined in Azure Policy. When you add a compliance package to your subscription or management group from the ASC Security Policy, that essentially assigns the regulatory initiative to your selected scope (subscription or management group). You can see that standard or benchmark appear in your compliance dashboard with all associated compliance data mapped as assessments.

In this way, you can track newly published regulatory initiatives as compliance standards in your Security Center regulatory compliance dashboard, Additionally, when Microsoft releases new content for the initiative (new policies that map to more controls in the standard), the additional content appears automatically in your dashboard. You are also able to download a summary report for any of the standards that have been onboarded to your dashboard.

There are several supported regulatory standards and benchmarks that can be onboarded to your dashboard. The newest one is the Azure Security Benchmark, which is the Microsoft-authored Azure-specific guidelines for security and compliance best practices based on common compliance frameworks. Additional standards will be supported by the dashboard as they become available.  

For more information about dynamic compliance packages, see the documentation here.

Workflow automation with Azure Logic Apps generally available 

Organizations with centrally managed security and IT operations implement internal workflow processes to drive required action within the organization when discrepancies are discovered in their environments. In many cases, these workflows are repeatable processes and automation can greatly reduce overhead streamline processes within the organization.

Workflow automation in Azure Security Center, now generally available, allows customers to create automation configurations leveraging Azure Logic Apps and to create policies that will automatically trigger them based on specific Security Center findings such as Recommendations or Alerts. Azure Logic App can be configured to do any custom action supported by the vast community of Logic App connectors or use one of the templates provided by Security Center such as sending an email. In addition, users are now able to manually trigger a Logic App on an individual alert or recommendation directly from the recommendation (with a ‘quick fix’ option) or alert page in Azure Security Center.

Advanced integrations with export of Security Center recommendations and alerts generally available

The continuous export feature of Azure Security Center, which supports the export of your security alerts and recommendations, is now generally available, also via policies. Use it to easily connect the security data from your Security Center environment to the monitoring tools used by your organization, by exporting to Azure Event Hubs or Azure Log Analytics workspaces.

This capability supports enterprise-scale scenarios, among others, via the following integrations:

Export to Azure Event Hubs enables integration with Azure Sentinel, third party SIEMs, Azure Data Explorer, and Azure Functions.
Export to Azure Log Analytics workspaces enables integration with Microsoft Power BI, custom dashboards, and Azure Monitor.

For more information, read about continuous export.

Building a secure foundation

With these additions, Azure continues to provide a secure foundation and gives you built-in native security tools and intelligent insights to help you rapidly improve your security posture in the cloud. Azure Security Center strengthens its role as the unified security management and advanced threat protection solution for your hybrid cloud.

Security can’t wait. Get started with Azure Security Center today and visit Azure Security Center Tech Community, where you can engage with other security-minded users like yourselves.
Quelle: Azure

Updates to Azure Maps Web SDK includes powerful new features

Today, we are announcing updates to the Azure Maps Web SDK, which adds support for common spatial file formats, introduces a new data driven template framework for popups, includes several OGC services, and much more.

Spatial IO module

 

With as little as three lines of code this module makes it easy to integrate spatial data with the Azure Maps Web SDK. The robust features in this module allow developers to:

Read and write common spatial data files to unlock great spatial data that already exists without having to manually convert between file types. Supported file formats include: KML, KMZ, GPX, GeoRSS, GML, GeoJSON, and CSV files containing columns with spatial information.
Use new tools for reading and writing Well-Known Text (WKT). Well-Known Text is a standard way to represent spatial geometries as a string and is supported by most GIS systems. (Docs)
Connect to Open Geospatial Consortium (OGC) services and integrate with Azure Maps web SDK.

Overlay Web Map Services (WMS) and Web Map Tile Services (WMTS) as layers on the map. (Docs)
Query data in a Web Feature Service (WFS). (Docs)

Overlay complex data sets that contain style information and have them render automatically using minimal code. For example, if your data aligns with the GitHub GeoJSON styling schema, many of these will automatically be used to customize how each shape is rendered. (Docs)
Leverage high-speed XML and delimited file reader and writer classes. (Docs)

Try out these features in the sample gallery.

WMS overlay of world geological survey.

Popup templates

Popup templates make it easy to create data driven layouts for popups. Templates allow you to define how data should be rendered in a popup. In the simplest case, passing a JSON object of data into a popup template will generate a key value table of the properties in the object. A string with placeholders for properties can be used as a template. Additionally, details about individual properties can be specified to alter how they are rendered. For example, URLs can be displayed as a string, an image, a link to a web page or as a mail-to link. (Docs | Samples)

A popup template displaying data using a template with multiple layouts.

Additional Web SDK enhancements

Popup auto-anchor—The popup now automatically repositions itself to try and stay within the map view. Previously the popup always opened centered above the position it was anchored to. Now, if the position it is anchored to is near a corner or edge, the popup will adjust the direction it opens so that is stays within the map view. For example, if the anchored position is in the top right corner of the map, the popup would open down and to the left of the position.
Drawing tools events and editing—The drawing tools module now exposes events and supports editing of shapes. This is great for triggering post draw scenarios, such as searching within the area the user just drew. Additionally, shapes also support being dragged as a whole. This is useful in several scenarios, such as copying and pasting a shape then dragging it to a new location. (Docs | Samples)
Style picker layout options—The style picker now has two layout options. The standard flyout of icons or a list view of all the styles. (Docs | Sample)

Style picker icon layout.

Code sample gallery

The Azure Maps code sample gallery has grown to well over 200 hundred samples. Nearly every single sample was created as a response to a technical query we had from a developer using Azure Maps.

An Azure Maps Government Cloud sample gallery has also been created and contains all the same samples as the commercial cloud sample gallery, ported over to the government cloud.

Here are a few of the more recently added samples:

The Route along GeoJSON network sample loads a GeoJSON file of line data that represent a network of paths and calculates the shortest path between two points. Drag the pins around on the map to calculate a new path. The network can be any GeoJSON file containing a feature collection of linestrings, such as a transit network, maritime trade routes, or transmission line network. Try the feature out.

Map showing shortest path between points along shipping routes.

The Census group block analysis sample uses census block group data to estimate the population within an area drawn by the user. Not only does it take into consideration the population of each census block group, but also the amount of overlap they have with the drawn area as well. Try the feature out.

Map showing aggregated population data for a drawn area.

The Get current weather at a location sample retrieves the current weather for anywhere the user clicks on the map and displays the details in a nicely formatted popup, complete with weather icon. Try the feature out.

Map showing weather information for Paris.

Send us your feedback

We always appreciate feedback from the community. Feel free to comment below, post questions to Stack Overflow, or submit feature requests to the Azure Maps Feedback UserVoice.
Quelle: Azure