Enhancing Microsoft's commercial marketplace for partners

As we head into the global partner conference Microsoft Inspire on July 14-18, 2019, a big focus is on rethinking how we make it easier for customers to discover, try, and buy cloud-based software and services from our partners. Today, we're excited to announce new tools, commerce options, and a rewards program through the Microsoft commercial marketplace that help partners leverage this important distribution channel.

Today, we're excited to announce new tools, commerce options, and a rewards program through the Microsoft commercial marketplace that makes it easier than ever for our partners to grow their business through this important distribution channel.

Commercial marketplace as a new distribution channel

Many people think of a commercial marketplace as a simple catalog of offer listings which are often difficult to navigate.  For customers, they are often linked off to a different experience for trial and purchase. Publishers and partner selling solutions are challenged by how to differentiate their solutions to stand out in the volume of offers.

We are working with our partner community to ensure the commercial marketplace experiences deliver a new distribution channel to drive their business growth. For example, Microsoft AppSource targets business decision makers while Azure Marketplace targets IT and developers. This includes having the commerce capabilities and solution supply to capture the rising customer demand in online enterprise software purchases.

Microsoft’s commercial marketplace has at its core, one product catalog, which includes both Microsoft cloud software and services as well as software and services from our partners built on top of and to connect with one or more cloud services offered by Microsoft (Microsoft 365, Dynamics 365, Microsoft Power Platform, and Azure) publishing as transactable offers. This is not just for independent software vendors (ISVs) creating repeatable intellectual property (IP). The commercial marketplace experiences also support offers from managed service providers (MSPs) and consulting services from systems integrators (SIs) such as one-day assessments, migration offers, and more.

Customers can discover, try, and buy solutions from the marketplace in one of three ways:

Direct from the publishers
Through our field sales teams who retire quota for selling eligible partner solutions, or
Through our global distribution channel, where we now also pay the channel a 10 percent incentive to sell marketplace publisher solutions with a transactable SaaS offer, and who participate in the IP co-sell program. 

Customers are looking for quicker buying experiences where they can purchase Microsoft products AND solutions from our partners – together in one place, with one transaction, on a unified invoice, which the commercial marketplace provides.

Using the commercial marketplace as a strategic distribution channel will require partners to think about their business model in new and different ways, which can provide significant new revenue streams. For instance, any publisher can continue to list or trial their solution in Microsoft AppSource or Azure Marketplace, but the impact will likely be similar to what they face today, where the customer discovery experience is crowded due to volume of offers and the publisher struggles to differentiate their solution. However, when a publisher chooses to transact in Microsoft’s commercial marketplace, they get access to a whole new set of benefits and ways to sell:

Gain access to a global reseller channel with over 70,000 cloud solution providers (CSP) in over 140 countries who receive an incentive directly from Microsoft when they resell publisher solutions.
Provides simplified deal-making with custom contract amendments.
Centralized partnership experience via Partner Center for the commercial marketplace onboarding, lead sharing, deal registration, benefits, incentives, sales analytics, and investments.
New go-to-market (GTM) benefits via marketplace rewards that unlocks GTM benefits for publishers as they reach various transaction thresholds.

A single onboarding and management experience

Whether a customer buys direct through Microsoft field sellers or through CSP, each of these channels is accessible and managed by partners through a single ingestion point known as Partner Center. Within Partner Center, publishers can publish marketplace offers and manage their engagements, while resellers can bundle Microsoft software and services with publisher’s software and services. This simplifies customer, publisher, and reseller engagement with one transaction and one invoice.

New commerce options

To accompany this new publisher experience, we’ve released new commerce capabilities that partners of all sizes are already starting to benefit from such as ESRI with site-based SaaS, Barracuda, and Trend Micro who use custom business models for their SaaS-based applications. Approved Contact, Crossware, and MongoDB are also using the per-seat SaaS capabilities and managed services from long-time Microsoft partners like Ingram Micro.

These new commerce enhancements allow publishers to customize their offers to meet customer needs and scale through the global reach of Microsoft’s customer and channel communities.

Marketplace rewards

We’re also sharing marketplace rewards, which is a new benefits program which will enhance the success of publishers with transactable offers in the commercial marketplace. Through the program publishers can unlock sales, marketing, and technical benefits to help accelerate their success. As a publisher’s business grows they’ll continue to unlock more benefits designed to provide support at every stage of their growth. This comes with a new badging program for Microsoft AppSource and Azure Marketplace that will quickly direct customers to partner solutions they can trust, which will work with cloud services from Microsoft. We will be publishing additional details on the program next week during Microsoft Inspire.

With these capabilities, publishers will be able to create new revenue streams, reach new customers in new markets, and grow their business faster than ever before.

Next steps

Learn more about how to onboard and publish your offers at Partner Center, how to list them on Microsoft AppSource and Azure Marketplace, and how to take advantage of the new go-to-market services and onboarding resources.

Visit the Microsoft Inspire site, which will be updated with materials, photos, and keynote replays for more highlights from the event.
Quelle: Azure

Two ways to share Azure Advisor recommendations

If your IT organization is like most, you probably work with many different people across many different teams. When it comes to common IT tasks like optimizing your cloud workloads, you might need to interact with several resource owners or even complete a formal review process.

That’s why with Azure Advisor, we’ve made it easy to share recommendations with other people across your teams so you can follow best practices that help you get the most out of Azure. Advisor is a free Azure service that helps you optimize your Azure resources for high availability, security, performance, and cost by providing personalized recommendations based on your usage and configurations.

Here are two ways you can share your Advisor best practice recommendations with your teams.

1. Export a PDF or CSV of your Advisor recommendations

Probably the simplest way to share your Advisor recommendations is by exporting an Advisor recommendation report as a PDF or CSV through the Advisor UI in the Azure portal.

This report shows a summary of your Advisor recommendations by category, subscription, and potential business impact. Then you can easily share it with other teams so the resource owners can take action and optimize their resources for high availability, security, performance, and cost.

If you want to provide a specific view of a subset of your recommendations, you can use the UI filters or drill down into specific categories and recommendations. The recommendation report will only contain what you see on the screen when you generate it, which can help you focus on the most critical optimizations.

2. Use the Advisor API to integrate with your ticketing system or dashboards

The other way to share your Advisor recommendations with other people in your organization is via the Advisor REST API. Using this API, you can connect Advisor with your organization’s ticketing system and assign remediation work, set up an internal working dashboard your teams can review and action, or leverage Advisor’s recommendation data any way you choose.

The visual above shows just one way you can use the Advisor API with your ticketing application to share Advisor recommendations with your teams. Some setup is required, but once this scenario is complete, you can start remediating your recommendations more programmatically which will save you time as you optimize your resources.

This more advanced approach tends to work best for larger organizations, organizations managing a large number of Azure subscriptions and resources that are generating a large number of recommendations, and organizations that have a fairly sophisticated IT practice in place, since it scales well with the size of your deployments.

Visit the Advisor API documentation to learn more.

Get started with Advisor

Visit Advisor in the Azure portal to get started reviewing, sharing, and remediating your recommendations. For more in-depth guidance, visit the documentation. Let us know if you have a suggestion for Advisor by submitting an idea to the Azure Advisor feedback forum.
Quelle: Azure

Analyze AI enriched content with Azure Search’s knowledge store

Through integration with Cognitive Services APIs, Azure Search has long had the ability to extract text and structure from images and unstructured content. Until recently, this capability was used exclusively in full text search scenarios, exemplified in demos like the JFK files which analyzes diverse content in JPEGs and makes it available for online search. The journey from visual unstructured content, to searchable structured content is enabled by a feature called cognitive search. This capability in Azure Search is now extended with the addition of a knowledge store that saves enrichments for further exploration and analysis beyond search itself.

The knowledge store feature of Azure Search, available in preview, refers to a persistence layer in cognitive search that describes a physical expression of documents created through AI enrichments. Enriched documents are projected into tables or hierarchical JSON, which you can explore using any client app that is able to access Azure Storage. In Azure Search itself, you define the physical expression or shape of the projections in the knowledge store settings within your skillset.

Customers are using a knowledge store (preview) in diverse ways, such as to validate the structure and accuracy of enrichments, generate training data for AI models, and ad-hoc analysis of their data.

For example, the Metropolitan Museum of Art opened access to all images of public domain works in its collection. Enriching the artworks with cognitive search and the knowledge store allowed us to explore the latent relationships within the artworks on different dimensions like time and geography. Questions like how have images of family groups changed over time, or when were domestic animals included in paintings, are now answerable when you are able to identify, extract, and save the information in a knowledge store (preview).

With the knowledge store, anyone with an Azure subscription can apply AI to find patterns, insights, or create dashboards over previously inaccessible content.

What is the knowledge store (preview)?

Cognitive search is the enrichment of documents with AI skills before they are added to your search index. The knowledge store allows you to project the already enriched documents as objects (blobs) in JSON format or tabular data in table storage.

As part of your projection, you can shape the enriched document to meet your needs. This ensures that the projected data aligns with your intended use.

When using tabular projections, a knowledge store (preview) can project your documents to multiple tables while preserving the relationships between the data projected across tables. The knowledge store has several other features like allowing you to save multiple unrelated projections of your data. You can find more information about a knowledge store (preview) in the overview documentation.

Data visualization and analytics

Search enables you to find relevant documents, but when you’re looking to explore your data for corpus wide aggregations or want to visualize changes over time you need your data represented in a form other than a search index.

Leveraging Power BI’s integration with Azure tables, gets your dashboard started with only a few clicks. To identify insights from the enriched documents over dimensions like time or space, simply project your enriched documents into tables, validate that Power BI recognizes the relationships and you should now have your data in a format that is ready to consume within the visuals.

When you create a visual, any filters work, even when your data spans related tables. As an example, the art dashboard was created on the open access data from the MET in the knowledge store and the Art Explorer site uses the search index generated from the same set of enrichments.

The art explorer site allows you to find art works and related works while the Power BI report gives you a visual representation of the corpus and allows you to slice your data along different dimensions. You now can answer questions like “How does body armor evolve over time?”

In this example, a knowledge store (preview) enabled us to analyze the data ad-hoc. In another example, we may for instance enrich invoices or business forms, project the structured data to a knowledge store (preview), and then create a business-critical report.

Improving AI models

A knowledge store (preview) can also help improve the cognitive search experience itself as a data source for training AI models deployed as a custom skill within the enrichment pipeline. Customers deploying an AI model as a custom skill can project a slice of the enriched data shaped to be the source of their machine learning (ML) pipelines. A knowledge store (preview) now serves as a validator of the custom skill as well as a source of new data that can be manually labeled to retrain the model. While the enrichment pipeline operates on each document individually, corpus level skills like clustering require a set of documents to act on. A knowledge store (preview) can operate on the entire corpus to further enrich documents with skills like clustering and save the results back in a knowledge store (preview) or update the documents in the index.

Getting started

To start using a knowledge store (preview) you will need to:

Add a knowledge store (preview) configuration to your skillset.
Optionally, add a shaper skill to the skillset to define the shape of the projected enrichment.
Add a projection for tables, objects, or both to a knowledge store (preview). You may project the output of the shaper skill, or elements from the enriched document directly.

A knowledge store (preview) enables the use of your enriched data in new or improved models, visualizing and exploring the data in tools like Power BI and app based experiences merging the raw and enriched data. We will continue to add more capabilities and updates over the coming months.

For a detailed walkthrough, see the knowledge store (preview) getting started guide.
Quelle: Azure

Reducing overall storage costs with Azure Premium Blob Storage

In this blog post, we will take a closer look at pricing for Azure Premium Blob Storage, and its potential to reduce overall storage costs for some applications.

Premium Blob Storage is Azure Blob Storage powered by solid-state drives (SSDs) for block blobs and append blobs. For more information see, “Azure Premium Blob Storage is now generally available.” It is ideal for workloads that require very fast storage response times and/or has a high rate of operations. For more details on performance see, “Premium Block Blob Storage – a new level of performance.”

Azure Premium Blob Storage utilizes the same ‘pay-as-you-go’ pricing model used by standard general-purpose V2 (GPv2) hot, cool, and archive. This means customers only pay for the volume of data stored per month and the quantity of operations performed.

The current blob pricing can be found on the Azure Storage pricing page. You will see, data storage gigabyte (GB) pricing decreases for colder tiers, while the inverse is true for operation prices where operations per 10,000 pricing decreases for hotter tiers. Premium data storage pricing is higher than hot data storage pricing. However, read and write operations pricing for premium are lower than hot read and write operations. This means premium blob storage is meant to store data that is transacted upon frequently and is not intended for storing infrequently or rarely accessed data.

Given the lower operations costs, is there a point where premium, not only provides better performance, but also costs less than standard (GPv2) hot?

To answer this question, I created the graph below, which shows the relative total monthly cost of storing 1 Terabytes (TiB) of data in standard (GPv2) hot and premium, varying the operations per second performed on this 1TiB of data using a 70/30 split between read and write operations.

As you can see in the graph above, the estimated total monthly cost for premium becomes less than standard (GPv2) hot between 40 to 50 operations per second for each 1TiB of data. This means customers will save money for workloads with high rate of operations by using premium even if they do not require the better performance provided by premium.

Next steps

To get started with Premium Blob Storage, you provision a ‘Block Blob’ storage account in your subscription, and start creating containers and blobs using the existing Blob Service REST API or any existing tools such as AzCopy or Azure Storage Explorer.

Conclusion

We are very excited about Azure Premium Blob Storage providing low and consistent latency, and the potential cost savings for applications with high rate of operations. We look forward to hearing your feedback at premiumblobfeedback@microsoft.com or feel free to share your ideas and suggestions for Azure Storage on our feedback forum. To learn more about Azure Blob Storage please visit our product page.
Quelle: Azure

What’s new in Azure Firewall

This post was co-authored by Anitha Adusumilli, Principal Program Manager, Azure Networking. 

Today we are happy to share several key Azure Firewall capabilities as well as update on recent important releases into general availability (GA) and preview.

Multiple public IPs soon to be generally available
Availability Zones now generally available
SQL FQDN filtering now in preview
Azure HDInsight (HDI) FQDN tag now in preview
Central management using partner solutions

Azure Firewall is a cloud native firewall-as-a-service offering which enables customers to centrally govern and log all their traffic flows using a DevOps approach. The service supports both application and network level filtering rules and is integrated with the Microsoft Threat Intelligence feed for filtering known malicious IP addresses and domains. Azure Firewall is highly available with built-in auto scaling.

Multiple public IPs soon to be generally available

You can now associate up to 100 public IP addresses with your firewall. This enables the following scenarios:

DNAT – You can translate multiple standard port instances to your backend servers. For example, if you have two public IP addresses, you can translate TCP port 3389 (RDP) for both IP addresses.
SNAT – Additional ports are available for outbound SNAT connections, reducing the potential for SNAT port exhaustion.

Figure one – Sample Azure Firewall Public IP configuration with multiple public IPs.

Currently, Azure Firewall randomly selects the source public IP address to use for a connection. If you have any downstream filtering on your network, you need to allow all public IP addresses associated with your firewall. Explicit SNAT configuration is on our roadmap. See our documentation "Deploy an Azure Firewall with multiple public IP addresses using Azure PowerShell" for more information.

Multiple public IPs GA will be available in all public regions by July 12, 2019. It is currently supported using REST APIs, templates, PowerShell and Azure CLI. Portal support will be available shortly.

Availability Zones now generally available

Azure Firewall can be configured during deployment to span multiple Availability Zones for increased availability. With Availability Zones, your availability increases to 99.99 percent uptime. For more information, see the Azure Firewall Service Level Agreement (SLA). The 99.99 percent uptime SLA is offered when two or more Availability Zones are selected.

You can also associate Azure Firewall to a specific zone just for proximity reasons, using the service standard 99.99 percent SLA.

There's no additional cost for a firewall deployed in an Availability Zone. However, there are additional costs for inbound and outbound data transfers associated with Availability Zones. For more information, see Bandwidth pricing details.

 
Figure two – Creating Azure Firewall with 99.99 percent SLA

 

SQL FQDN filtering now in preview

You can now configure SQL FQDNs in Azure Firewall application rules. This allows you to limit access from your VNets to only the specified SQL server instances. The capability is available as a preview in all Azure regions.

Using this capability, you can filter traffic from your virtual networks (VNets) to Azure SQL Database, Azure SQL Data Warehouse, Azure SQL Managed Instance, or SQL IaaS instances deployed in your VNets.

During preview, SQL FQDN filtering is supported in proxy-mode only, port 1433. If you are using non-default ports for SQL IaaS traffic, you can configure those ports in the Firewall application rules. If you are using SQL in redirect mode, which is default for clients connecting within Azure, you can filter access using the SQL service tag as part of Azure Firewall network rules.

SQL FQDN filtering is currently available using REST APIs, templates, and Azure CLI. The portal will be available shortly.

Figure three – Creating Azure Firewall Application rule for SQL FQDN

Azure HDInsight (HDI) FQDN tag now in preview

We recently announced the availability of a FQDN Tag for Azure HDInsight (HDI). This tag is in public preview in all Azure public regions.

VNet-deployed Azure services like HDI have outbound infrastructure dependencies on other Azure services, for example, Azure Storage. To protect your data from exfiltration risk, you might want to use Azure Firewall to restrict outbound access for HDI clusters and allow access to only your data.  In addition, you should also allow access to the HDI infrastructure traffic.

FQDN tags for Azure Firewall allow services like HDI to pre-configure their infrastructure dependencies, for example, Azure Storage account FQDNs used by HDI. Instead of using network level service tags in the Azure Firewall to allow HDI outbound dependencies, you can get much more granular control to restrict outbound traffic for HDI by using the FQDN tags.

Figure four – Creating Azure Firewall Application rule for HDI FQDN tag

Central management using partner solutions

Azure Firewall public REST APIs can be used by third party security policy management tools to provide a centralized management experience for Azure Firewalls, Network Security Groups (NSGs), and network virtual appliances (NVAs).

Barracuda Cloud Security Guardian, now generally available in Azure Marketplace, automatically deploys and configures Barracuda's Cloud Generation WAF/Firewall, or Microsoft's Azure Firewall.
AlgoSec CloudFlow central management capability for Azure Firewall and NSGs is now public preview. For more information you can watch this video.
Tufin Orca, now public preview, automates the discovery, development and enforcement of a unified security policy across Kubernetes and Azure Firewall. For more information you can watch this video.

Next steps

For more information on everything we covered above please see the following blogs, documentation, and videos.

Azure Firewall Documentation
May blog: Azure Firewall and network virtual appliances

Azure Firewall central management partners:

AlgoSec CloudFlow
Barracuda Cloud Security Guardian
Tufin Orca

Quelle: Azure

Migrate to Azure HDInsight in as little as 12 weeks

Recent announcements in the open source ecosystem have led customers of prominent open source analytics technology companies to explore options with Microsoft Azure and to this end, we now have new and compelling offers aimed at helping on-premises open source analytics workloads to migrate to Azure! We are always trying to find ways to make it easier for our customers to move to the cloud and with this offer, our customers can now realize even greater savings and accelerate migration.

Azure HDInsight is an easy, cost-effective, fully managed and fully supported open source cloud-scale analytics service that can process massive amounts of data securely. We are pleased to offer customers who want to migrate their on-premises analytics workloads to the cloud, the HDInsight migration accelerator program which offers the following benefits:

Mitigate risk – Engagement led by Microsoft engineering and bolstered by custom accelerators enabling rapid migration of on-premises workloads to Azure HDInsight with minimal risk.
Get to production fast – Accelerated timeline of 12-17 weeks due to a structured engagement starting with an assessment all the way till go-live.

3. Discounted – A limited-time offer to take advantage of end-of-financial year discounts!
Quelle: Azure

Previewing Azure SDKs following new Azure SDK API Standards

Today we’re happy to share a new set of libraries for working with Azure Storage,  Azure Cosmos DB, Azure Key Vault, and Azure Event Hubs in Java, Python, JavaScript or TypeScript, and .NET. These libraries provide access to new service features, and represent the first step towards applying a new set of standards across the Azure SDKs that we believe will make the libraries easier to learn and integrate into your software. You can get these libraries today from your favorite package manager, and we would love to hear your feedback on GitHub. To get started follow the instructions linked below:

Python Release Notes
Java Release Notes
JavaScript Release Notes
.NET Release Notes

Why are we doing this?

Much like moving software from the client or on-premises to the cloud is a paradigm shift, we too have been going through a period of rapid innovation in Azure’s capabilities and learning about how best to expose it to developers. Now that some Azure services have matured and been adopted into business-critical enterprise applications, we have been learning what patterns and practices were critical to developer productivity around these services. In addition, we’ve been listening to your feedback and we’ve made sure that our new effort has incorporated your suggestions and requests. Finally, we understand that consistency, ease of use, and discoverability are equally important to the identified patterns when it comes to working with the Azure SDKs.

What’s different?

The team will go into much of what I am about to outline in follow-up blog posts but to get started, the big changes came from a set of objectives we defined based on your feedback. Those were:

Create easy to use APIs with productivity on par with the best libraries of the language ecosystems.
Provide APIs that are idiomatic to the language and ecosystem they are used in.
Evolve over time in a very compatible fashion.
Focus as much on documentation and samples, as on APIs.
Change how we create the libraries at their core.

Ease of use and productivity

Productivity is a multifaceted topic on its own, but two main elements of it are consistency and usability.

To help reach consistency, we codified the things we learned while working with Azure developers into a set of API design guidelines. The guidelines themselves are built in the open on our GitHub repository and consist of a section of principles that goes into more detail on how we approached this space, a set of general guidelines, and language specific guidelines for Java, Python, .NET, and JavaScript. By applying the guidelines we believe that these libraries will be easier to use and easier to learn. When you learn a pattern or API shape in one library you should be able to count on it being the same in others.

To help drive usability, we tweaked how we gather user feedback. We continue to do many of the standard practices in the industry such as releasing previews, working directly with developers on their projects, and responding to issues in many different community forums but the next step for us was to usability test the libraries. For each of the libraries we are releasing today, we have brought developers into a lab and had them work through different use cases while we observed them. That feedback was instrumental to shape both the guidelines as well as the API shape of the libraries.

Idiomatic libraries

A key piece of feedback we heard while talking with developers was that our APIs didn’t always feel ergonomic in a language. To fix that, we explicitly established as one of our core principles that the libraries we author should follow the patterns of that language. In addition, as we update each service’s libraries to follow guidelines, we are ensuring that we always release libraries for those services in each of the following languages: Java, Python, JavaScript, and .NET.

Compatibility

Compatibility has always been a value at Microsoft. Developers put significant time and money into solutions and should be able to count on them continuing to work. There is a tension here. For some cases we have had to make breaking changes to get to a better foundation. We believe aligning on that foundation will help meet the productivity goals outlined above, and once it’s set we intend to provide a high degree of compatibility. As a final note on compatibility, we’ve looked at the dependencies that we took and tried to minimize them as much as possible to reduce future incompatibilities and versioning complexities which should make upgrading libraries and using other pieces of software alongside these libraries easier.

Documentation

Having good documentation and samples could be considered an aspect of productivity but we wanted to call it out as its own goal because many developers rate it as the top factor in choosing what technologies to use. Much like the usability studies we’re doing on the APIs themselves, we have been doing the same on Azure Quickstarts to ensure that new developers can begin to experiment with Azure Services quickly. We have also heard feedback that both API Reference code snippets and samples get out-of-date and we have been building the tooling to build API Reference code snippets and samples regularly from GitHub and publish those into documentation.

Change how we build

We also found we needed to change how we work with and engage the community and what we build on top of. To that end, we’ve begun work to restructure and centralize our development effort into a few key repositories:

Azure-sdk: As a central location to start from and a place for high level topics like the design guidelines.
Repositories for each language:

azure-sdk-for-python
azure-sdk-for-java
azure-sdk-for-js
azure-sdk-for-net

Finally, we built a new core library that is helping us provide common features like identity and authentication, both synchronous and asynchronous APIs, logging, error handling, networking retries and more across all libraries.

What should I do?

We are very pleased to be sharing the preview release of the new Azure libraries that conform to many of the principles outlined above. We would like to encourage you to download and try the new SDKs today. To help you along the way, we are providing release notes describing what’s new in each library, how to get the packages, and how to file GitHub issues specific to the previews.

Python Release Notes
Java Release Notes
JavaScript Release Notes
.NET Release Notes

In addition to filing GitHub issues, feel free to also follow and tweet us @AzureSDK. We look forward to receiving your feedback so we can improve the libraries, and make it easier for you to create great software and solve problems with Azure.
Quelle: Azure

CGG speeds geoscience insights on Azure HPC

A leader in geosciences, CGG has been imaging the earth’s subsurface for more than 85 years. Their products and solutions help clients locate natural resources. When a customer asked whether the cloud could speed the creation of the high-resolution reservoir models that they needed, CGG turned to Azure high performance computing (HPC).

CGG is a team of acknowledged imaging experts in the oil and gas industry. Their subsurface imaging centers are internationally acclaimed, and their researchers are regularly recognized with prestigious international awards for their outstanding technical contributions to the industry. CGG was an early innovator in seismic inversion, the technique of converting seismic survey reflection data into a quantitative description of the rock properties in an oil and gas reservoir. From this data, highly detailed models can be made, like the following image showing the classification of rock types in a 3D grid for an unconventional reservoir interval. These images are invaluable in oil and gas exploration. Accurate models help geoscientists reduce risk, drill the best wells, and forecast production better.

The challenge is to get the best possible look beneath the surface. The specialized software is compute-intensive, and complex models can take hours or even days to render.

Figure 1. An advanced seismic model, using CGG’s Jason RockMod.

Azure provides the elasticity of compute that enables our clients to rapidly generate multiple high-resolution rock property realizations with Jason RockMod for detailed reservoir models.”

– Joe Jacquot, Strategic Marketing Manager, CGG GeoSoftware

The challenge: prove it in the cloud

An oil and gas company in Southeast Asia asked CGG to work with them to find the optimum way to deploy reservoir characterization technology for their international team. The customer also wondered if they could take advantage of the cloud to save on hardware costs in their datacenter, where they already ran leading-edge CGG geoscience software solutions. They knew the theory of the cloud’s elasticity and on-demand scalability were familiar with its use in oil exploration, but they didn’t know if their applications would perform as well in the cloud as they did on-premises.

In a rapidly moving industry, performance gains translate to speedier decision making, accelerated exploration, and development timelines, so the company asked CGG to provide a demonstration. As geoscience experts, CGG wanted to show their software in the best possible light. They turned to the customer advisors at AzureCAT (led by Tony Wu) for help creating a proof of concept that demonstrated how the cloud capacity could help the company get insights faster.

Demo 1: CGG Jason RockMod

CGG Jason RockMod overcomes the limitations of conventional reservoir characterization solutions with a geostatistical approach to seismic inversion. The outcome is accurate reservoir models so in depth that geoscientists can use them to predict field reserves, fluid flow patterns, and future production.

The company wanted to better understand the scale of the RockMod simulation in the cloud, and wondered whether the cloud offered any real benefits compared to running the simulations on their own workstations.

Technical teams from CGG and AzureCAT looked at RockMod as a big-compute workload. In this type of architecture, computationally intensive operations such as simulations are split across CPUs in multiple computers (10 to 1,000s). A common pattern is to administer a cluster of virtual machines, then schedule and monitor the HPC jobs. This do-it-yourself approach would enable the company to set up their own cluster environment in Azure virtual machine scale sets. The number of VM instances can automatically scale per demand or on a defined schedule.

RockMod creates a family of multiple equi-probable simulation outputs, called realizations. In the proof of concept tests for their customer, CGG ran RockMod on a Microsoft Windows virtual machine on Azure. It served as the master node for job scheduling and distribution of multiple realizations across several Linux-based HPC nodes in a virtual machine scale set. Scale sets support up to 1,000 VM instances or 300 custom VM images, more than enough scale for multiple realizations.

An HPC task generates the realizations, and the speed depends on the number of CPU cores or non-hyperthreading more than the processor speed. AzureCAT recommended non-hyperthreading cores as the best choice for application performance. From a storage perspective, the realizations are not especially demanding. During testing, the storage IO rate went up to a few hundred megabytes per second (MB/s). Based on these considerations, CGG chose the cost-effective DS14V2 virtual machine with accelerated networking, which has 16 cores and 7 GB per core.

“Azure provided the ideal blend of HPC and storage performance and price, along with technical support that we needed to successfully demonstrate our high-end reservoir characterization technology on the cloud. It was a technical success and paved the way for full-scale commercial deployment.”

– Joe Jacquot, Strategic Marketing Manager, CGG GeoSoftware

Figure 2. Azure architecture of CGG Jason RockMod.

Benchmarks and benefits

A typical project is about 15 GB in size and includes more than three million seismic data traces. Some of the in-house workstations could render a single realization within a day or two, but the goal was to run 30 realizations by taking advantage of the cloud’s scalability. The initial small scale test ran one realization on one HPC node, which took just under 12 hours to complete. That was within the target range, and now the CGG team needed to see what would happen when they ran at scale.

To test the linear scalability of the job, they first ran eight realizations on eight nodes. The results were nearly identical to the small scale test run. The one realization to one node formula seemed to work. For the sake of comparison, they tried running 30 realizations on just eight nodes. That didn’t work so well, the tests were nearly four times slower.

The final test ran 30 realizations on 30 nodes, one realization to one node. The results were similar to the small scale test, and the job was completed in just over 12 hours. This scenario was tested several times to validate the results which were consistent. The test was a success.

Demo 2: CGG InsightEarth

In their exploration and development efforts, the company also used CGG InsightEarth, a software suite that accelerates 3D interpretation and visualization.

The interpretation teams at the company were using several InsightEarth applications to locate hydrocarbon deposits, faults and fractures, and salt bodies all of which can be difficult to interpret. They asked CGG to compare the performance of the InsightEarth suite on Azure to their workstations on premises. Their projects were typically 15 GB in size, with more than three million seismic data traces.

InsightEarth is a powerful, memory-intensive application. To get the best performance, the application must run on a GPU-based computer, and to get the best performance from GPUs, efficient memory access is critical. To meet this requirement on Azure, the team of engineers from CGG and AzureCAT looked at the specialized, GPU-optimized VM sizes that are available. The Azure NV-series virtual machines are powered by NVIDIA Tesla M60 GPUs and the NVIDIA GRID technology with Intel Broadwell CPUs, which are suited for compute-intensive visualizations and simulations.

The GRID license gives the company the flexibility to use an NV instance as a virtual workstation for a single user, or to give 25 users concurrent access to the VM running InsightEarth. The company wanted the collaborative benefits of the second model. Unlike a physical workstation, a cloud-based GPU virtual workstation would allow them all to view the data after running a pre-processing or interpretation step because all the data was in the cloud. This would streamline their workflow, eliminating the need to move the data back on premises for that task.

The initial performance tests ran InsightEarth on an NV24 VM on Azure. The storage IO demand was moderate, with rates around 100 MB/s. However, the VM’s memory size proved to be a bottleneck. The amount of data that could be loaded from the memory was limited, and the performance wasn’t as good as the more powerful GPU setup used on premises.

Next, the team ran a similar test using an ND-series VM. This type is specifically designed to offer excellent performance for AI and deep learning workloads, where huge data volumes are used to train models. They chose an ND24-size VM with double the amount of the memory and the newer NVIDIA Tesla P40 GPUs. This time, the results were considerably better than NV24.

From an infrastructure perspective, storage also matters when deploying high-performance applications on Azure. The team implemented SoftNAS, a type of storage that supports both the Windows and Linux VMs used in the overall solution. SoftNAS also suits the lower to mid-range storage IO demands for these simulation and interpretation workloads.

Figure 3. Azure Architecture for InsightEarth

Summary

GPUs and memory-optimized VMs provided the performance that the company needed for both Jason Rockmod and InsightEarth. Better yet, Azure gave them the scale they needed to run multiple realizations in parallel. In addition, they deployed GPUs to both software solutions, giving the CGG engineers a standard front end to work with. They set up access to the Azure resources through an easy-to-use portal based on Citrix Cloud, which also runs on Azure.

CGG’s customer, the oil and gas company, was delighted with the results. Based on the results of the benchmark tests, the oil and gas company decided to move all of their current CGG application workload to Azure. The cloud’s elasticity and Azure’s pay-per-use model were a compelling combination. Not only did the Azure solution perform, it proved to be more cost-effective compared to the limited scalability they could achieve with their computing power on-premises.

Company:
CGG

Microsoft Azure CAT Technical Lead:
Tony Wu

Quelle: Azure

Enable receipt understanding with Form Recognizer’s new capability

One of the newest members of the Azure AI portfolio, Form Recognizer, applies advanced machine learning to accurately extract text, key-value pairs, and tables from documents. With just a few samples, it tailors its understanding to supplied documents, both on-premises and in the cloud. 

Introducing the new pre-built receipt capability

Form Recognizer focuses on making it simpler for companies to utilize the information hiding latent in business documents such as forms. Now we are making it easier to handle one the most commonplace documents in a business, receipts, “out of the box.”  Form Recognizer’s new pre-built receipt API identifies and extracts key information on sales receipts, such as the time and date of the transaction, merchant information, the amounts of taxes, totals, and more, with no training required.

Streamlining expense reporting

Business expense reporting can be cumbersome for everyone involved in the process. Manually filling out and approving expense reports is a significant time sink for both employees and managers. Aside from productivity lost to expense reporting, there are also pain points around auditing expense reports. A solution to automatically extract merchant and transaction information from receipts can significantly reduce the manual effort of reporting and auditing expenses.

Given the proliferation of mobile cameras, modern expense reports often contain images of receipts that are faded, crumpled up, or taken in suboptimal lighting conditions. Existing receipt solutions often target high quality scanned images and are not robust enough to handle such real-world conditions.

Enhance your expense reporting process using the pre-built capability

Form Recognizer eases common pain points in expense reporting, delivering real value back to business. By using the receipt API to extract merchant and transaction information from receipts, developers can unlock new experiences in the workforce. And since the pre-built model for receipts works off the shelf without training, it reduces the speed to deployment.

For employees, expense applications leveraging Form Recognizer can pre-populate expense reports with key information extracted from receipts. This saves employees time in managing expenses and travel that they can focus on their core roles. For central teams like finance within a company, it also helps expense auditing by using the key data extracted from receipts for verification. The optical character recognition (OCR) technology behind the service can handle receipts that are captured in a wide variety of conditions,  including smartphone cameras, reducing the amount of manual searching and reading of transaction documents required by auditors.

Our customer: Microsoft’s internal finance operations

The pre-built receipt functionality of Form Recognizer has already been deployed by Microsoft’s internal expense reporting tool, MSExpense, to help auditors identify potential anomalies. Using the data extracted, receipts are sorted into low, medium, or high risk of potential anomalies. This enables the auditing team to focus on high risk receipts and reduce the number of potential anomalies that go unchecked.

MSExpense also plans to leverage receipt data extraction and risk scoring to modernize the expense reporting process. Instead of identifying risky expenses during auditing, such automated processing can flag potential issues earlier in the process during the reporting or approval of the expenses. This reduces the turnaround time for processing the expense and any reimbursement.

“The pre-built receipt feature of Form Recognizer enables our application not only to scale from sampling 5 percent of receipts to 100 percent, but more importantly to streamline employee expense report experience by auto-populating/creating expense transactions, creating happy path to payment, receipt data insights to approver managers, giving employees time back to be used in value-add activities for our company. The service was simple to integrate and start seeing value.“

—Luciana Siciliano, Microsoft FinOps (MSExpense)

Learn more

To learn more about Form Recognizer and the rest of the Azure AI ecosystem, please visit our website and read the documentation.

Get started by contacting us.

For additional questions please reach out to us at formrecog_contact@microsoft.com
Quelle: Azure

Larger, more powerful standard file shares for Azure Files now in preview

Better scale and more power for IT pros and developers.

Azure Files has always delivered secure, fully managed cloud file shares with a full range of data redundancy options. While customers love the simplicity of Azure Files and the hybrid capabilities of Azure File Sync, until now, scaling cloud file shares beyond 5 TiB required changing the paradigm for accessing data.

Today, we are excited to announce the preview of a larger and higher scale standard tier for Azure Files, now available to all Azure customers. This preview significantly improves your experience by increasing standard file shares’ capacity and performance limits. In select regions, standard file shares in general purpose accounts can support the following larger limits.

Azure Files standard storage scale limits

Azure Files
Before (standard tier)
New (standard tier)

Capacity per share
5 TiB
100 TiB (20x increase)

Max IOPS per share
1,000 IOPS
10,000 IOPS (10x increase)

Max throughput per share
Up to 60 MiB/s
Up to 300 MiB/s (5x increase)

Performance limits for a single file remain the same at 1 TiB, 1000 IOPS, and 60 MiB/s. Standard file shares are backed by hard disk drives. If your workload is latency sensitive, you should consider Azure Files premium tier, that is backed by solid-state drives. Visit Azure Files scale limits documentation to get more details.

Pricing and availability

There is no additional charge for enabling larger and increased scale for standard file shares on your storage accounts. Refer to the pricing page for further details.

We are introducing standard larger file shares in preview with support for locally redundant and zone redundant storages in select regions. We are quickly expanding the region coverage. We advise you to visit region availability documentation to get the latest information.

Getting started

Sign up for the preview of larger file shares (standard tier): Follow the onboarding steps in the Azure Files documentation.
Create a new storage account: In your registered subscription, create a new general purpose storage account in one of the supported regions with a supported redundancy option. See detailed steps on how to create a new storage account.
Create a new standard file share: Create a new standard file share under the newly created storage account. See detailed steps on how to create a standard file share.

Frequently asked questions

Do larger file shares work with premium file shares?

We recently announced general availability (GA) of premium files with 100 TiB support. All premium regions support large file shares as a GA offering.

Does Azure File Sync support larger file shares?

Absolutely. Azure File Sync performance is a function of the number of objects, files and folders, in the sync scope, rather than of the capacity of the file share in the cloud. Our team is constantly making investments in improving the performance and scale of sync and cloud tiering. So, keep your Azure File Sync agent updated with the latest version and visit Azure File Sync scalability targets to keep informed of the latest scale.

What about larger file shares support for my existing standard shares?

We are working on upgrading existing storage accounts. After the existing accounts are upgraded, you will be able to opt-in your accounts into the larger file shares feature without any disruption to your existing workloads, including Azure File Sync. Stay up to date on the latest status through the region availability section of Azure Files documentation.

Share your experiences

As always, you can share your feedback and experiences on the Azure Storage forum or just send us email at LFSFeedback@microsoft.com. Post your ideas and suggestions about Azure Storage on our feedback forum.

Happy sharing!
Quelle: Azure