Kategorie: Microsoft Azure

Azure Stack IaaS – part 3

This blog post was co-authored by David Armour Principal PM Manager, Azure Stack and Tiberiu Radu, Senior Program Manager, Azure Stack.​

Foundation of Azure Stack IaaS

Remember back in the virtualization days when you had to pick a host for your virtual machine? Some of my business units could tell by the naming convention the make and manufacturer of the hardware. Using this knowledge, they’d fill up the better gear first, leaving the teams that didn’t know better with the oldest hosts.

Clouds take a different approach. Instead of hosts, VMs are placed into a pool of capacity. The physical infrastructure is abstract. The compute, storage, and networking resources consumed by the VM are defined through software.

Azure Stack is an instance of the Azure cloud that you can run in your own datacenter. Microsoft has taken the experience and technology from running one of the largest clouds in the world to design a solution you can host in your facility. This forms the foundation of Azure Stack’s infrastructure-as-service (IaaS).

Let’s explore some of the characteristics of the Azure Stack infrastructure that allows you to run cloud-native VMs directly in your facility.

Cloud inspired hardware

Microsoft employees can’t just purchase their favorite server and rack it into an Azure datacenter. The only servers that enter an Azure datacenter have been specifically built for Azure. Not only are the servers built for Azure, so are the networking devices, the racks, and the cabling. This extreme standardization allows the Azure team to operate an Azure datacenter with just a handful of employees. Because all the servers are standardized and can be uniformly operated and automated, adding additional capacity to a datacenter doesn’t require hiring more employees to operate them.

Other advantages of standardizing hardware configurations is the standardization leads to expected, repeatable results – not only for Microsoft and Azure, but for its customers. The hardware integration has been validated and is a known recipe. Servers, storage, networking, cabling layout, and more are all well-known and based on these recipes, the ordering, delivery, and integration of new hardware components. Servicing and eventual retirement are repeatable and scalable. The full end-to-end validation of these configurations is done once with quick checks in place when the capacity is delivered and installed.

These principles are applied to Azure Stack solutions as well. The configurations, their capabilities, and validation are all well-known and the result is a repeatable and supportable product. Microsoft, its partners, and most importantly the end customer benefit. While an Azure Stack customer is limited to the defined, partner solutions, they have been built with reasonable flexibility so the customer can choose the specific capabilities or capacities required. Please note, there is one exception – the Azure Stack Development Kit (ASDK) allows you to install Azure Stack on any hardware that meets the hardware requirements. The ASDK is for evaluation purposes and not supported as a production environment.

Learn more:

Azure Stack Capacity Planner
Best practices for planning Azure Stack deployment and post-deployment integrations with Azure

Azure Stack hardware partnerships

Microsoft has partnered and co-engineered solutions with a variety of hardware partners or OEMs. The benefit is that Azure Stack can meet you where your existing relationships exist. These relationships may be based on existing hardware purchasing agreements, geographic location, or support capabilities. Keeping in mind the principles of operating a solution in a well-defined manner, Microsoft has set minimum requirements for Azure Stack hardware solutions. Each of our partners can then choose from their portfolio the components, servers, and network switches that best meet your needs. This creates a well-defined variety that continues to be supportable and delivers the overall solution value.

Our current solutions partners are as follows:

Resiliency to failure

One of the principles we have taken from Microsoft’s experience in the enterprise and from Azure is overall solution resilience. The world of software and hardware is not perfect; things fail – cables go bad, software has bugs, power outages occur, and on and on. While we work to build better software and with our solution partners to continually improve, we must expect that things fail. Azure Stack solutions are not perfect, but have been constructed with the intent to overcome the common points of failure. For example, each copy of tenant/user data is stored on three separate storage devices in three separate servers. The physical network paths are redundant and provide better performance and resiliency to potential failure. The internal software of Azure Stack are services that coordinate across multiple instances. This type of end-to-end architectural design and implementation leads to a better end experience. Combining this approach to infrastructure resilience with the well-known and validated solutions approach described above provides for a better experience for the customer.

Learn more:

Understanding architectural patterns and practices for business continuity and disaster recovery on Microsoft Azure Stack

Hardened by default

When you run your IaaS VMs in Azure Stack you should know they are running on a secure foundation. It turns out that one of the reasons people select Azure Stack is because they have data and/or processes that are either regulated or defined in a contractual agreement. Azure Stack not only gives its owners control of their data and processes, it comes with an infrastructure which is secured by default. In fact, the underlying infrastructure is locked down in a way that neither the owner nor Microsoft can access it. If it ever needs to be accessed because of a support issue, both the owner and Microsoft combine their keys to obtain access to the system and for a limited time.

Azure leads the industry in security compliance, and security compliance is important for Azure Stack as well. In Azure, Microsoft fully manages the technology, people, and processes as well as its compliance responsibilities. Things are different with Azure Stack. While the technology is provided by Microsoft, the people and processes are managed by the operator. To help operators jump-start the certification process, Azure Stack has gone through a set of formal assessments by a third party-independent auditing firm to document how the Azure Stack infrastructure meets the applicable controls from several major compliance standards. The documentation is an assessment of the technology not a certification of Azure Stack due to the standards including several personnel-related and process-related controls, but they help you get started. The technology assessments include the following standards:

PCI-DSS – Addresses the payment card industry
CSA Cloud Control Matrix –  A comprehensive mapping across multiple standards, including FedRAMP Moderate, ISO27001, HIPAA, HITRUST, ITAR, NIST SP800-53, and others
FedRAMP High – For government customers

To download the Azure Stack compliance documentation please see, "Azure Security and Compliance Additional Frameworks."

Learn more:

Azure Trust Center
Azure Stack infrastructure security posture
Security and compliance in Azure Stack
Using the privileged endpoint in Azure Stack

Get started by reviewing your options

As noted earlier, Azure Stack is sold as an integrated hardware system, with software pre-installed on the validated hardware. It typically comes in a standard server rack. You choose where your system will be located. You can host it in your data center or perhaps you want to run it in a service provider’s facility.

With the Azure Stack running in your location of choice, you also have a choice of who operates the Azure Stack infrastructure. An Azure Stack operator is responsible for giving access to the Azure Stack, keeping the software and firmware up-to-date, providing the content in the marketplace, monitoring the system health, and diagnosing issues. Azure Stack provides automation, documentation, and training for all of these processes so that someone from your organization can operate Azure Stack. e also provide trained partner experts who can operate your Azure Stack either in their facility or yours.

Here is an overview of your options when you acquire your Azure Stack:

A system you manage

Typically on-premises
You control management and ops
Buy Azure Stack from Microsoft
Buy hardware from the vendor
Call Microsoft for support

A managed service

Typically at service provider premises
Service is managed for you
Buy service from service provider
Service includes hardware and software
Call the service provider for support

Learn more:

Azure Stack operator documentation
Azure Stack technology and service partners

Tuning your IaaS VMs for a cloud infrastructure

Once you have your Azure Stack up and running and you begin to plan your first IaaS VM deployments, you need to think about these VMs as cloud deployments, not virtualization deployments. IaaS VMs run best when they take advantage of the cloud infrastructure that they are running on. Many times, the way you tune a VM in your cloud infrastructure will be very different than the way you tuned VMs in your traditional virtualization environment. That said, you can always start with what you already have and improve those solutions through modern operations.

A great example of this is the use of multiple disks to get the needed IOps and throughput required of the application. As is the case in Azure, virtual machines placed in Azure Stack have limits applied for their disk activity. This limits the impact of one VM’s activity on another VM – aka noisy neighbor. While these limits are great for IaaS environments, it may take extra work to deploy workloads that get the appropriate resources needed, and in this example, it is IOps.

For optimization of SQL Server deployments, our documentation provides guidance on how to configure storage to obtain the needed performance. In this case, the approach is to attach multiple disks and stripe them to obtain the capacity and performance. When you use managed disks for your VMs, it allows the system to optimize where the physical data gets stored within your Azure Stack. Moving from virtualization environments to IaaS is reasonably straightforward and has its benefits, but requires a little bit of work on your first deployment. You can always use tools like Azure Monitor and the Virtual Machine solutions to better understand your workloads and gain insights on the performance of your VMs. When your VMs are not answering the performance requirements, you can also use the Azure Performance Diagnostics VM Extension for Windows to troubleshoot and identify potential bottlenecks.

The great thing about IaaS, and specifically Azure Stack, is the ability to easily reuse the deployment templates or artifacts to reduce the work for migration of similar workloads. We will cover this more in a future blog post.

Learn more:

Create virtual machine disk storage in Azure Stack
Optimize SQL Performance on Azure Stack
Azure Managed Disks Overview
Frequently asked questions about Azure IaaS VM disks
Considerations for Managed Disks on Azure Stack

Infrastructure purpose built for running cloud-native VMs

Few organizations can claim that they have experience building one the largest cloud infrastructures in the world. When you buy an Azure Stack, you get the benefit of Microsoft’s Azure experience. Microsoft has partnered with the best OEMs to deliver a standardized configuration so that you don’t have to worry about these details. The infrastructure of Azure Stack is purpose-built to get the best for your IaaS VMs – keeping them safe, secure, and performant.

Learn more:

How to buy Azure Stack
Azure Stack datacenter integration

In this blog series

We hope you come back to read future posts in this blog series. Here are some of our passed and upcoming topics:

Azure Stack at its core is an Infrastructure-as-a-Service (IaaS) platform
Start with what you already have
Fundamentals of IaaS
Do it yourself
Pay for what you use
It takes a team
If you do it often, automate it
Protect your stuff
Build on the success of others
Journey to PaaS

Quelle: Azure

Intel and Microsoft bring optimizations to deep learning on Azure

This post is co-authored with Ravi Panchumarthy and Mattson Thieme from Intel.

We are happy to announce that Microsoft and Intel are partnering to bring optimized deep learning frameworks to Azure. These optimizations are available in a new offering on the Azure marketplace called the Intel Optimized Data Science VM for Linux (Ubuntu).

Over the last few years, deep learning has become the state of the art for several machine learning and cognitive applications. Deep learning is a machine learning technique that leverages neural networks with multiple layers of non-linear transformations, so that the system can learn from data and build accurate models for a wide range of machine learning problems. Computer vision, language understanding, and speech recognition are all examples of deep learning at play today. Innovations in deep neural networks in these domains have enabled these algorithms to reach human level performance in vision, speech recognition and machine translation. Advances in this field continually excite data scientists, organizations and media outlets alike. To many organizations and data scientists, doing deep learning well at scale poses challenges due to technical limitations.

Often, default builds of popular deep learning frameworks like TensorFlow are not fully optimized for training and inference on CPU. In response, Intel has open-sourced framework optimizations for Intel® Xeon processors. Now, through partnering with Microsoft, Intel is helping you accelerate your own deep learning workloads on Microsoft Azure with this new marketplace offering.

"Microsoft is always looking at ways in which our customers can get the best performance for a wide range of machine learning scenarios on Azure. We are happy to partner with Intel to combine the toolsets from both the companies and offer them in a convenient pre-integrated package on the Azure marketplace for our users” 

– Venky Veeraraghavan, Partner Group Program manager, ML platform team, Microsoft.

Accelerating Deep Learning Workloads on Azure

Built on the top of the popular Data Science Virtual Machine (DSVM), this offer adds on new Python environments that contain Intel’s optimized versions of TensorFlow and MXNet. These optimizations leverage the Intel® Advanced Vector Extensions 512 (Intel® AVX-512) and Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) to accelerate training and inference on Intel® Xeon® Processors. When running on an Azure F72s_v2 VM instance, these optimizations yielded an average of 7.7X speedup in training throughput across all standard CNN topologies. You can find more details on the optimization practice here.

As a data scientist or AI developer, this change is quite transparent. You still code with the standard TensorFlow or MXNet frameworks. You can also use the new set of Python (conda) environments (intel_tensorflow_p36, intel_mxnet_p36) on the DSVM to run your code to take full advantage of all the optimizations on an Intel® Xeon Processor based F-Series or H-Series VM instance on Azure. Since this product is built using the DSVM as the base image, all the rich tools for data science and machine learning are still available to you. Once you develop your code and train your models, you can deploy them for inferencing on either the cloud or edge.

“Intel and Microsoft are committed to democratizing artificial intelligence by making it easy for developers and data scientists to take advantage of Intel hardware and software optimizations on Azure for machine learning applications. The Intel Optimized Data Science Virtual Machine (DSVM) provides up to a 7.7X speedup on existing frameworks without code modifications, benefiting Microsoft and Intel customers worldwide”

– Binay Ackalloor, Director Business Development, AI Products Group, Intel.

Performance

In Intel’s benchmark tests run on Azure F72s_v2 instance, here are the results comparing the optimized version of TensorFlow with the standard TensorFlow builds.

Figure 1: Intel® Optimization for TensorFlow provides an average of 7.7X increase (average indicated by the red line) in training throughput on major CNN topologies. Run your own benchmarks using tf_cnn_benchmarks. Performance results are based on Intel testing as of 01/15/2019. Find the complete testing configuration here.

Getting Started

To get started with the Intel Optimized DSVM, click on the offer in the Azure Marketplace, then click “GET IT NOW”. Once you answer a few simple questions on the Azure Portal, your VM is created with all the DSVM tool sets and the Intel optimized deep learning frameworks pre-configured and ready to use.

The Intel Optimized Data Science VM is an ideal environment to develop and train deep learning models on Intel Xeon processor-based VM instances on Azure. Microsoft and Intel will continue their long partnership to explore additional AI solutions and framework optimizations to other services on Azure like the Azure Machine Learning service and Azure IoT Edge.

Next steps

Create your Intel Optimized Data Science VM instance from the Azure Marketplace.
Learn more about the Intel Optimized Data Science VM.
Build AI solutions and deploy machine learning models in production at scale using Azure Machine Learning service.
New to Azure? Get your free trial.

Quelle: Azure

Microsoft continues to build the case for data estate modernization on Azure

Special thanks to Rik Tamm-Daniels and the Informatica team for their contribution to this blog post. ​

With the latest release of Azure SQL Data Warehouse, Microsoft doubles-down on Azure SQL DW as one of the core data services for digital transformation on Azure. In addition to the fundamental benefits of agility, on-demand scaling and unlimited compute availability, the most recent price-to-performance metrics from the GigaOM report are one of several the compelling arguments they have made for customers to adopt Azure SQL DW. Interestingly, Microsoft is also announcing the general availability of Azure Data Lake Gen 2 and Azure Data Explorer. Along with Power BI for rich visualization, these enhanced set of capabilities cement Microsoft’s leadership position around Cloud Scale Analytics.

Every day, I speak with joint Informatica and Microsoft customers who are looking to transform their approach to their data estate with a cohesive data lake and cloud data warehousing solution architecture. These customers range from global logistics companies, to auto manufacturers to the world’s largest insurers, and all of them see the tremendous potential of the Microsoft modern data estate approach; in fact, just via Informatica's iPaaS (integration platform-as-a-service) offering, Informatica Intelligent Cloud Services, we’ve seen a significant quarter-to-quarter growth in customer data volumes being moved to Azure SQL DW.

Of course, as compelling as the Azure SQL DW technology is, for many customers, modernizing a legacy enterprise data warehouse is a daunting proposition to even consider. The thought of touching the intricate web of dependencies around the warehouse can keep even the most battle-tested CIO up at night. A key consideration when attempting your own cloud data warehousing/cloud data modernization initiative is to ensure you have intelligence about the existing schemas, lineage and dependencies to enable companies to incrementally unravel the data web surrounding the warehouse, and with laser-like precision, begin to move workloads and use case to Azure SQL DW.

Enter Informatica’s Enterprise Data Catalog with full end-to-end source-to-destination lineage and searchable machine-learning and AI-driven intelligent metadata about what data lives where in the warehouse to clear the fog of complexity and illuminate a clear path to cloud data warehousing. In fact, the concept of discovery and catalog driven-modernization is such a compelling leap forward that Microsoft and Informatica developed a single-sign-on Data Accelerator on Informatica’s Intelligent Cloud Services on Azure that can be accessed directly from the Azure SQL DW management console with your Azure credentials.

Data Accelerator for Azure

Want to see how Informatica and Microsoft can jumpstart your cloud data warehousing modernization initiative? Join us on Informatica's world tour of hands-on workshop at a Microsoft Technology Center near you. Workshops are taking place in North America right now and will be coming to EMEA and APJ very soon!

Register here: Cloud Data Warehouse Modernization for Azure Workshop. 
Quelle: Azure

Build a CI/CD pipeline for API Management

APIs have become mundane. They have become the de facto standard for connecting apps, data, and services. In the larger picture, APIs are driving digital transformation in organizations.

With the strategic value of APIs, a continuous integration (CI) and continuous deployment (CD) pipeline has become an important aspect of API development. It allows organizations to automate deployment of API changes without error-prone manual steps, detect issues earlier, and ultimately deliver value to end users faster.

This blog walks you through a conceptual framework for implementing a CI/CD pipeline for deploying changes to APIs published with Azure API Management.

The problem

Organizations today normally have multiple deployment environments (e.g., Development, Testing, Production) and use separate API Management instances for each environment. Some of these instances are shared by multiple development teams, who are responsible for different APIs with different release cadences.

As a result, customers often come to us with the following challenges:

How to automate deployment of APIs into API Management?
How to migrate configurations from one environment to another?
How to avoid interference between different development teams who share the same API Management instance?

We believe the approach described below will address all these challenges.

CI/CD with API Management

The proposed approach is illustrated in the above picture. In this example, there are two deployment environments: Development and Production. Each has its own API Management instance. The Production instance is managed by a designated team, called API publishers. API developers only have access to the Development instance. 

The key in this proposed approach is to keep all configurations in Azure Resource Manager templates. These templates should be kept in a source control system. We will use Git as an example. As illustrated in the picture, there is a Publisher repository that contains all configurations of the Production API Management instance in a collection of templates:

Service template: Contains all service-level configurations (e.g., pricing tier and custom domains).
Shared templates: Contains shared resources throughout an API Management instance (e.g., groups, products, and identity providers).
API templates: Includes configurations of APIs and their sub-resources (e.g., operations and policies).
Master template: Ties everything together by linking to all templates.

API developers will fork and clone the Publisher repository. In most cases, they will focus on API templates for their APIs and should not change the shared or service templates.

When working with Resource Manager templates, we realize there are two challenges for API developers:

First, API developers often work with Open API specifications and may not be familiar with Resource Manager schemas. To simplify creation of templates, we created a utility tool to automate the creation of API templates based on Open API specifications.
Second, for customers who have already been using API Management, another challenge is how to extract existing configurations into Resource Manager templates. We created another tool to generate templates based on existing configurations.

Once developers have finished developing and testing an API, and have generated the API template, they will submit a pull request to the Publisher repository. API publishers can validate the pull request and make sure the changes are safe and compliant. Most of the validations can be automated as part of the CI/CD pipeline. When the changes are approved and merged successfully, API publishers will deploy them to the Production instance. The deployment can also be easily automated with Azure Pipelines.

With this approach, the deployment of API changes into API Management instances can be automated and it is easy to promote changes from one environment to another. Since different API development teams will be working on different sets of API templates, it also reduces the chances of interference between different teams.

Next steps

You can find the guidance, examples, and tools in this GitHub repository. Please give it a try and let us know your feedback and questions.

We realize our customers bring a wide range of engineering cultures and existing automation solutions. The approach and tools provided here are not meant to be a one-size-fits-all solution. That's why we published and open-sourced everything on GitHub, so that you can extend and customize the solution.
Quelle: Azure

Real-time serverless applications with the SignalR Service bindings in Azure Functions

Since our public preview announcement at Microsoft Ignite 2018, every month thousands of developers worldwide have leveraged the Azure SignalR Service bindings for Azure Functions to add real-time capabilities to their serverless applications. Today, we are excited to announce the general availability of these bindings in all global regions where Azure SignalR Service is available!

SignalR Service is a fully managed Azure service that simplifies the process of adding real-time web functionality to applications over HTTP. This real-time functionality allows the service to push messages and content updates to connected clients using technologies such as WebSocket. As a result, clients are updated without the need to poll the server or submit new HTTP requests for updates.

Azure Functions provides a productive programming model based on triggers and bindings for accelerated development and serverless hosting of event-driven applications. It enables developers to build apps using the programming languages and tools of their choice, with an end-to-end developer experience that spans from building and debugging locally, to deploying and monitoring in the cloud. Combining Azure SignalR Service with Azure Functions using these bindings, you can easily push updates to the UI of your applications with just a few lines of code. The source of those updates can be data coming from different Azure services, or any service able to communicate over HTTP, thanks to the triggers supported in Azure Functions that will start the execution of a script responding to an event.

A common scenario worth mentioning is updating the UI of an application based on modifications made on the database. Using a combination of Cosmos DB change feed, Azure Functions, and SignalR Service, you can automate these UI updates in real-time with just a few lines of code for registering the client that will receive those updates and pushing the updates themselves. This fully managed experience is a great fit for event-driven scenarios and enables the creation of serverless backends and applications with real-time capabilities, reducing development time and operations overhead.

Using the Azure SignalR Service bindings for Azure Functions you will be able to:

Use SignalR Service without dependency on any application server for a fully managed, serverless experience.
Build serverless real-time applications using all Azure Functions generally available languages: JavaScript, C#, and Java.
Leverage the SignalR Service bindings with all event triggers supported by Azure Functions to push messages to connected clients in real-time.
Use App Service Authentication with SignalR Service and Azure Functions for improved security and out-of-the-box, fully managed authentication.

Next steps

Check out the documentation, “Build real-time apps with Azure Functions and Azure SignalR Service.”
Follow the quickstart, “Create a chat room with Azure Functions and SignalR Service using JavaScript” to get started.
Check out more code samples on the GitHub repo.
Sign up for your Azure account for free.

We'd like to hear about your feedback and comments. You can reach the product team on the GitHub repo, or by email. 
Quelle: Azure

Azure Databricks – New capabilities at lower cost

Azure Databricks provides a fast, easy, and collaborative Apache Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions backed by industry leading SLAs.

With Azure Databricks, customers can set up an optimized Apache Spark environment in minutes. Data scientists and data engineers can collaborate using an interactive workspace with languages and tools of their choice. Native integration with Azure Active Directory (Azure AD) and other Azure services enables customers to build end-to-end modern data warehouse, machine learning and real-time analytics solutions.

We have seen tremendous adoption of Azure Databricks and today we are excited to announce new capabilities that we are bringing to market.

General availability of Data Engineering Light

Customers can now get started with Azure Databricks with a new low-priced workload called Data Engineering Light that enables customers to run batch applications on managed Apache Spark. It is meant for simple, non-critical workloads that don’t need the performance, autoscaling, and other benefits provided by Data Engineering and Data Analytics workloads. Get started with this new workload.

Additionally, we have reduced the price for the Data Engineering workload across both the Standard and Premium SKUs. Both the SKUs are now available at up to 25 percent lower cost. To check out the new pricing for Azure Databricks SKUs, visit the pricing page.

Preview of managed MLflow

MLflow is an open source framework for managing the machine learning lifecycle. With managed MLflow, customers can access it natively from their Azure Databricks environment and leverage Azure Active Directory for authentication. With managed MLflow on Azure Databricks customers can:

Track experiments by automatically recording parameters, results, code, and data to an out-of-the-box hosted MLflow tracking server. Runs can now be organized in experiments from within the Azure Databricks, and results can be queried from within the Azure Databricks notebooks to identify the best performing models.
Package machine learning code and dependencies locally in a reproducible project format and execute remotely on a Databricks cluster.
Quickly deploy models to production.

Learn more about managed MLFlow.

Machine learning on Azure with Azure Machine Learning and Azure Databricks

Since the general availability of Azure Machine Learning service (AML) in December 2018, and its integration with Azure Databricks, we have received overwhelmingly positive feedback from customers who are using this combination to accelerate machine learning on big data. Azure Machine Learning complements the Azure Databricks experience by:

Unlocking advanced automated machine learning capability which enables data scientists of all skill levels to identify suitable algorithms and hyperparameters faster.
Enabling DevOps for machine learning for simplified management, monitoring, and updating of machine learning models.
Deploying models from the cloud and the edge.
Providing a central registry for experiments, machine learning pipelines, and models that are being created across the organization.

The combination of Azure Databricks and Azure Machine Learning makes Azure the best cloud for machine learning. Customers benefit from an optimized, autoscaling Apache Spark based environment, an interactive collaborate workspace, automated machine learning, and end-to-end Machine Learning Lifecycle management.

Get started today!

Try Azure Databricks and let us know your feedback!

Try Azure Databricks through a 14-day premium trial.
Try Azure Machine Learning.
Watch the webinar on Machine Learning on Azure.

Quelle: Azure

Microsoft opens first datacenters in Africa with general availability of Microsoft Azure

Today, I am pleased to announce the general availability of Microsoft Azure from our new cloud regions in Cape Town and Johannesburg, South Africa. Nedbank, Peace Parks Foundation, and eThekwini water are just a few of the organizations in Africa leveraging Microsoft cloud services today and will benefit from the increased computing resources and connectivity from our new cloud regions.

The launch of these regions marks a major milestone for Microsoft as we open our first enterprise-grade datacenters in Africa, becoming the first global provider to deliver cloud services from datacenters on the continent. The new regions provide the latest example of our ongoing investment to help enable digital transformation and advance technologies such as AI, cloud, and edge computing across Africa.

By delivering the comprehensive Microsoft Cloud — comprising Azure, Office 365, and Dynamics 365 — from datacenters in a given geography, we offer scalable, available, and resilient cloud services to companies and organizations while meeting data residency, security, and compliance needs. We have deep expertise in protecting data and empowering customers around the globe to meet extensive security and privacy requirements, including offering the broadest set of compliance certifications and attestations in the industry.

With 54 regions announced worldwide, more than any other cloud provider, Microsoft’s global cloud infrastructure will connect the new regions in South Africa with greater business opportunity, help accelerate new global investment, and improve access to cloud and Internet services across Africa.

Accelerating digital transformation in Africa

As we execute our expansion strategy, we consider the demand for locally delivered cloud services and the opportunity for digital transformation in the market. According to a study from IDC, spending on public cloud services in South Africa will nearly triple over the next five years, and the adoption of cloud services will generate nearly 112,000 net-new jobs in South Africa by the end of 2022. The increased utilization of public cloud services and the additional investments into private and hybrid cloud solutions will enable organizations in South Africa to focus on innovation and building digital businesses at scale.

Nedbank, a leading African bank that services a diverse client base in South Africa and the rest of Africa, is pursuing a transformation strategy with the Azure cloud platform to enable its digital aspirations. Microsoft has had a long relationship with Nedbank which has culminated in enabling its migration to the cloud to help increase its competitiveness, agility, and customer focus. Azure also provides compliance technologies that assist Nedbank to increase data privacy and security which are primary concerns of its customers, regulators, and investors. Nedbank has adopted a hybrid and multi-vendor cloud strategy in which Microsoft is an integral partner.

The Peace Parks Foundation, in collaboration with Cloudlogic, uses Azure to rapidly deploy infrastructure and solutions in far-flung protected spaces as well as to compute a considerable volume of data around at-risk species and wildlife in multiple conservation areas spanning thousands of kilometers. In efforts to sustain the delicate ecosystem and keystone species, such as the black and white rhinoceros, Peace Parks Foundation processes up to tens of thousands of images captured monthly on wildlife cameras in remote areas to monitor possible poaching activity. In the future, Peace Parks will leverage the new cloud infrastructure for radio over Internet protocol, a high-tech solution to a low-tech problem, to improve radio communication over remote and isolated areas.

eThekwini water is a unit of the eThekwini Municipality in Durban, South Africa responsible for the provision of water and sanitation services critical for sustaining life for 3.5 million residents in a 2,000+ square kilometer service area. In partnership with Cloudlogic, eThekwini water is using Azure for critical application monitoring as well as site failover and disaster recovery initiatives. It’ll benefit from locally delivered cloud services to improve performance of real-time reporting and monitoring of water infrastructure 24 hours a day, seven days a week.

Empowering people and organizations across Africa

Microsoft has long been working to support organizations, local start ups, and NGOs in Africa that have the potential to solve some of the biggest problems facing humanity, such as the scarcity of water and food as well as economic and environmental sustainability.

In 2013, we launched Microsoft 4Afrika investing in start-ups, partners, small-to-medium enterprises, governments, and youth on the African continent. The program is focused on delivering affordable access to the Internet, developing skilled workforces, and investing in local technology solutions. Africa has the potential to help lead the technology revolution; therefore, Microsoft is empowering organizations and people to drive economic development, inclusive growth, and digital transformation. 4Afrika is Microsoft’s business and market development engine on the continent, which is preparing the market to embrace cloud technology.

We have also extended FarmBeats, an end-to-end approach to help farmers benefit from technology innovation at the edge, to Nairobi, Kenya. FarmBeats strives to enable data-driven farming as we believe that data, coupled with the farmer’s knowledge and intuition about his or her farm, can help increase farm productivity and reduce costs. The new effort in Nairobi will be focused on addressing the specific challenges of farming in Africa with the intent of expanding to other African countries.

Bringing the complete cloud to Africa

The new cloud regions in Africa are connected with Microsoft’s other regions via our global network, one of the largest and most innovative on the planet, which spans more than 100,000 miles (161,000 kilometers) of terrestrial fiber and subsea cable systems to deliver services to customers. We’ve expanded our network footprint to reach Egypt, Kenya, Nigeria, and South Africa and will be expanding to Angola. Microsoft is bringing the global cloud closer to home for African organizations and citizens through our trans-Arabian paths between India and Europe, as well as our trans-Atlantic systems including Marea, the highest-capacity cable to ever cross the Atlantic.

Azure is the first of Microsoft’s intelligent cloud services to be delivered from the new datacenters in South Africa. Office 365, Microsoft’s cloud-based productivity solution, is anticipated to be available by the third quarter of calendar year 2019, and Dynamics 365, the next generation of intelligent business applications, is anticipated for the fourth quarter.

Follow these links to learn more about the new cloud services in South Africa and the availability of Azure regions and services across the globe.
Quelle: Azure

Rerun activities inside your Azure Data Factory pipelines

Data Integration is complex with many moving parts. It helps organizations to combine data and complex business processes in hybrid data environments. Failures are very common in data integration workflows. This can happen due to data not arriving on time, functional code issues in your pipelines, infrastructure issues etc. A common requirement is ability to rerun failed activities inside your data integration workflows. In addition to this, sometimes, you want to rerun activities to re-process the data due to some error upstream in data processing. Azure Data Factory now allows you to rerun activities inside your pipelines. You can rerun the entire pipeline or choose to rerun downstream from a particular activity inside your data factory pipelines.

Simply navigate to the ‘Monitor’ section in data factory user experience, select your pipeline run, click ‘View activity runs’ under the ‘Action’ column, select the activity and click ‘Rerun from activity <activityname>’

You can also view the rerun history for all your pipeline runs inside the data factory. Simply click on the toggle to ‘View All Rerun History’.

You can also view rerun history for a particular pipeline run by clicking ‘View Rerun History’ under the ‘Actions’ column. This allows you to see the different run attempts that you have made for your pipeline execution.

Learn more about rerunning activities inside your data factory pipelines.

Our goal is to continue adding features to improve the usability of Data Factory tools. Get started building pipelines easily and quickly using Azure Data Factory. If you have any feature requests or want to provide feedback, please visit the Azure Data Factory forum.
Quelle: Azure

Conversational AI updates for March 2019

We are thrilled to share the release of Bot Framework SDK version 4.3 and use this opportunity to provide additional updates for the Conversational AI releases from Microsoft.

New LINE Channel

Microsoft Bot Framework lets you connect with your users wherever your users are. We offer thirteen supported channels, including popular messaging apps like Skype, Microsoft Teams, Slack, Facebook Messenger, Telegram, Kik, and others. We have listened to our developer community and addressed one of the most frequently requested features – added LINE as a new channel. LINE is a popular messaging app with hundreds of millions of users in Japan, Taiwan, Thailand, Indonesia, and other countries.

To enable your bot in the new channel, follow the “Connect a bot to LINE” instructions. You can also navigate to your bot in the Azure portal. Go to the Channels blade, click on the LINE icon, and follow the instructions there.

SDK 4.3

In the 4.3 release, the team focused on improving and simplifying message and activities handling. The Bot Framework Activity schema is the underlying schema used to define the interaction model for bots. With the 4.3 release, we have streamlined the handling of some activity types in the Bot Framework Activity Schema, exposing a simple On* methods, thus simplifying the usage of such activities. On top of the activity handling improvements, for C# we have added MVC support, allowing developers to use the standard ASP.NET core application and ApiController. As with any release, we fixed a number of bugs, continue to improve LUIS and QnA integration, and further clean our engineering practices. There were additional updates across other areas like Language, Prompt and Dialogs, and Connectors and Adapters.

Review all changes that went into 4.3 in the detailed Change Log.
Stay up to date with the current list of all issues.

Simplify activity message handling

This release introduces a new way to handle incoming messages through a new class called ActivityHandler. An ActivityHandler receives incoming activities, as defined in the Bot Framework Activity Schema, then delegates the handling of each activity to one or more handler functions based on the activity’s type and other properties. For example, ActivityHandler exposes methods such as:

OnMessage – For dealing with all incoming messages
OnMembersAdded – For dealing with messages representing members being added
OnEvent – For generic event activities

You can find all the methods in the ActivityHandler.ts (for JavaScript) and ActivityHandler.cs (for .NET).

Using ActivityHandler, developers can handle events for incoming messages, application events, and a variety of conversation update events. This should make it easier to create common bot behaviors such as sending greetings and welcoming users.

This class provides an extensible base for handling incoming activities in an event-driven way. In JavaScript and TypeScript, the base ActivityHandler class can be used directly as main activity handler, as seen in the example code below. Developers can also derive subclasses from it to extend the core features.

Here is a small JavaScript code snippet example:

// Import the class from botbuilder sdk
const { ActivityHandler } = require('botbuilder');
// Create the bot “controller” object
const bot = new ActivityHandler();
server.post('/api/messages', (req, res) => {
adapter.processActivity(req, res, async (context) => {
// Route incoming activities to the ActivityHandler via the run() method
await bot.run(context);
});
});
// bind a handler for all incoming activities of type message
bot.onMessage(async (context, next) => {
// do stuff
await context.sendActivity(`Echo: ${ context.activity.text }`);
// proceed with further processing
await next();
});
// say hello when new members join
bot.onMembersAdded(async(context, next) => {
await context.sendActivity('Hello! I am a bot!');
await next();
});

Web API integration for .NET developers

A core tenant for the Bot Framework team is to drive parity across .NET and JS implementations. In that spirit, the .NET implementation of the ActivityHandler.cs exposes the same functionality with the given special programing language capabilities. However, ASP.NET Core provides a rich set of infrastructures supporting Web API, which can be easily integrated and used by bot developers. Therefore, in addition to the activity handling improvements, for C# we have added Web API support, allowing developers to use standard ASP.NET core application and ApiController.

Here is a simple code snippet for ASP.NET Web API Controller: 

[Route("api/messages")]
[ApiController]
public class BotController : ControllerBase
{
private IBotFrameworkHttpAdapter _adapter;
private IBot _bot;

public BotController(IBotFrameworkHttpAdapter adapter, IBot bot)
{
_adapter = adapter;
_bot = bot;
}

[HttpPost]
public async Task PostAsync()
{
// Delegate the processing of the HTTP POST to the adapter.
// The adapter will invoke the bot.
await _adapter.ProcessAsync(Request, Response, _bot);
}
}

Note, the _bot passed to _adapter.ProcesAsync method is the actual bot implementation and will handle any activity sent from the adapter, which has a very similar code to the above JS sample.

QnA Maker and Language Understanding

QnA Maker released Active Learning, which helps developers improve their knowledge base, based on real usage. Active learning helps identify and recommend question variations for any question and allows users to easily add them to their knowledge base.

For a user query, if QnA Maker returns top N answers where the difference in confidence score is low, Active Learning is triggered. Based on collective feedback across users, QnA Maker shows suggestions for alternate questions in your knowledge base.

To learn more about how QnA Maker Active Learning works and how to use it, read the documentation, “Use active learning to improve knowledge base.”

Templates and the Virtual Assistant Solution Accelerator

Templates and Solution Accelerators provide a mechanism to identify high growth opportunities for our Conversational AI, Speech, and broader Azure platform. These enable our customers and partners to accelerate delivery of advanced, transformational conversational experiences typically not viewed as possible or require too much effort to deliver a high-quality experience.

In this latest release we have provided significant updates to our Templates and Virtual Assistant solution. A high level summary of changes are covered in our Release Notes.

We are happy to share the availability of a JavaScript (Typescript) version of the Enterprise Template along with a Yeoman Generator. Work has started on the equivalent for the Virtual Assistant. We’ve also added coded unit tests to all Bots created by the templates providing a way to automate unit testing of dialogs along with further enhancements to the telemetry capabilities and the associated PowerBI dashboard.

We’ve also delivered a wide range of changes to the Virtual Assistant and Skills including a new template enabling Skills to be quickly created and added to a Virtual Assistant. There is also new support for proactive experiences, enabling the assistant and Skills to proactively reach out to a user or perform long running asynchronous operations.

Also, in this release are wide ranging improvements to the Productivity Skills including email, calendar, and to-do,  as well as the addition of FourSquare support to the Point of Interest Skill, and an enhanced WebChat test experience.

Web Chat 4.3

Web Chat is a popular component that lets developers add a messaging interface for their bot on the websites or mobile apps. Web Chat 4.3 release addresses the remaining accessibility issues and popular feature requests, like better indication of connectivity state for users with poor network connection.

To try Web Chat 4.3, follow the instructions on GitHub or explore code samples.

Get started

As we continue to improve our conversational AI tools and framework, we look forward to seeing what conversational experiences you will build for your customers. Get started today!
Quelle: Azure

Now available: Azure DevOps Server 2019

Following the launch of Azure DevOps in September, we’re pleased to announce the official release of Azure DevOps Server 2019! Previously known as Team Foundation Server (TFS), Azure DevOps Server 2019 brings the power of Azure DevOps into your dedicated environment. You can install Azure DevOps Server 2019 into any datacenter or sovereign, and determine when to apply updates.

About Azure DevOps Server

Azure DevOps includes developer collaboration tools which can be used together or independently, including Azure Boards (Work), Azure Repos (Code), Azure Pipelines (Build and Release), Azure Test Plans (Test), and Azure Artifacts (Packages). These tools support all popular programming languages, any platform (including macOS, Linux, and Windows) or cloud, as well as on-premises environments. Like with TFS, you control where you install Azure DevOps Server and when you apply updates. If you prefer to let us manage, use Azure DevOps Services which is available in more geographic regions than any other cloud hosted developer collaboration service.

Download Azure DevOps Server 2019

What’s new?

The release notes describe the major updates from TFS 2018 to Azure DevOps Server 2019, but my key highlights include:

The new navigation, which enables users to easily navigate between services, is more responsive and provides more space to focus on your work. But note this is a major UI overhaul – the largest we have done for several years, so please make sure your users are aware of the changes and update appropriate internal documentation as part of upgrading.
Azure Pipelines has been enhanced in many ways including new Build and Release pages, and support for YAML builds.
In addition to our existing integration between GitHub Enterprise and Azure Pipelines, which has been available in previous versions of TFS, Azure DevOps Server 2019 also enables integration of GitHub Enterprise commits and pull requests with work items in Azure Boards.
Organizations wishing to host Azure DevOps Server on their own virtual machines (VMs) on Azure can use Azure SQL Database instead of managing their own SQL Server VMs.
Azure Artifacts and Release Management licensing has evolved, making it simpler and more cost-effective for most customers.

Getting started

Whether your evaluating a new installation or planning an upgrade from a previous version of TFS, the following resources can help.

Azure DevOps Server 2019 Release Notes
Download Azure DevOps Server 2019
Product documentation (including the Installation Guide and Upgrade Guide)
System Requirements and Compatibility

Quelle: Azure