da Agency

ASOS: How they migrated from local monolith to microservices in Azure

Today we are kicking of a new series on Microsoft Mechanics called "How we built it" to share the real-world architectural back stories and best practices as told by the technology architects in our customer organizations.

We kick off the series with lead architect, Dave Green, from British online fashion retailer ASOS, to take a closer look at their design goals and approach for moving from a locally-operated monolith to a fully architected built-for-Cloud online retail system.

Breaking down the monolith

Founded almost two decades ago, ASOS was ahead of its time establishing a fully-online retail presence. The company has a growing customer base of 14 million customers, spanning 230 countries worldwide. Their exponential growth resulted in the need to move beyond a single currency and support multiple languages.

This meant building in more agility, scale and efficiency into their tech architecture. Rather than perform a lift and shift migration, the team decided to adopt a cloud-native, microservices architecture from scratch for faster iteration and release of new features.

Then they set to work breaking down their monolithic retail app which comprised stateful, intertwined services into core microservices, swapping out functions from the monolith piece-by-piece.

As Dave Green explains, core to their approach was the decoupling of their presentation layer from their compute layer. This gave them more freedom to easily add or change features and scale their developer teams to work on multiple features at once. Further, as each service maintains its own state and data, this made it easier to scale the data layer and compute layer independently.

Multi-geo footprint and performance

Breaking down the monolith and moving core services to the Cloud also enabled them to architect for greater performance and resiliency for their global customer base. This included geo-redundancy across North and West EU regions, where most of their customers reside with a handful of core services sensitive to latency, running in Asian and North American regions.

The combined approach with Cloud microservices, paid off on some of their busiest shopping days. This includes Black Friday, where they saw an increase in peak order handling from 9 orders per second using their original monolith retail system to 33 orders per seconds with their Cloud microservices architected system.

To learn more, watch Dave Green's account on "How we built it." You can also join Dave and the ASOS team, on June 29th at 9am PDT for their AMA session on the Microsoft Tech Community at http://aka.ms/how-we-built-it-ASOS.
Quelle: Azure

da Agency

Announcing Microsoft Azure Government services in the Cloud Solution Provider program

We are excited to announce that Azure Government services are now available through the Cloud Solution Provider program (CSP). The CSP program enables partners to make Microsoft Cloud services part of their customer offerings, expanding U.S. government customer options in meeting their mission goals. With the CSP program, partners can now create high-value service offerings that combine use of Azure Government with solution management, customer support, and billing to U.S. government customers. The security, privacy, compliance and transparency of the Azure Government platform give U.S. government partners the right foundation for meeting regulatory requirements while delivering innovation to the U.S. public sector.

The CSP program is a great fit for the U.S. public sector where partners already build, deploy and manage solutions on behalf of federal, state and local government entities. Over the last 60 days, we piloted Azure Government in CSP with a variety of partners and their customers running production workloads. The feedback reinforces the expected demand from U.S. Government customers for secure, compliant cloud-based solutions through a broad ecosystem of partners. Reduced cost, cloud speed, and increased efficiency are all potential benefits of the managed services that U.S. government partners can enable through CSP. 

“Smartronix and Microsoft share a common commitment to deliver innovative and valuable services to our government customers. AzureGov on CSP makes it easier for us to deliver these solutions.”   – Robert Groat, Executive Vice President Smartronix 

"As compliance with regulation and public demand rapidly increase reliance on technology in the public sector, a consistent, reliable and accessible cloud platform is the backbone supporting that change, with Microsoft Azure, through the SYNNEX CLOUDSolv marketplace, we enable our partners to drive deployment and provide comprehensive solutions that help them compete and grow their business in the government vertical.” – Darren Harbaugh, Vice President, Cloud & Emerging Services, SYNNEX Corporation

“Our partnership with Microsoft enabled us to seize a tremendous opportunity as we were closing the year. Not only did the fast turnaround have us transacting on the Azure Cloud in less than a week, but now we can provide our services on Microsoft’s infrastructure which opens a wave of new possibilities for new government customers.” – Eric Van Heel, VP of Cloud Solutions Support Avtex Solutions

“In partnering with Microsoft through CSP and leveraging technologies such as the Azure Pricing Calculator, which is bleeding edge to provide a comprehensive and easy to use web-based user interface to select and dynamically price out all of Azure Cloud Services, Microsoft enabled us to turn around quotes and spin up resources based on our clients’ requirements within minutes not days.” – Sonoka Ho, VP of Business Operations @ TechTrend.us

The array of services available in Azure Government is rapidly increasing, and nearly all are available today through CSP.  From infrastructure services like virtual machines, storage, and networking to platform services like data, analytics, web and mobile services. Transacting Azure in the Government Cloud through CSP can be done in three steps:

Learn about CSP: the requirements for participation and the option to own the customer relationship end-to-end.
Decide on the model: direct or indirect.
Get started: sign up, get ready to sell, transact, and support.

Once you decide which model is right for you, follow the enrollment path in CSP for Azure Government.

Become an indirect reseller

As an indirect reseller in the CSP program, you’ll work with an indirect provider (also known as a distributor). Indirect providers can provide your customers with product support, provide you with technical assistance and marketing, and help you establish financing and credit terms.

Minimum requirements include having a Microsoft Partner Network (MPN) ID and the ability to sign legal agreements on behalf of your organization.

If you don’t have the infrastructure to provide customer support and billing, you can connect with an indirect provider. This gives you more time to spend with your customers building specialized service offers. Review the authorized indirect providers in your area, to get help with value added services, support and billing.  Here’s where you can learn more about the indirect reseller model and find a provider.

Become a direct partner

As a direct partner, you provide your customers with cloud services, cloud products, customer support, and you bill your customers directly. If you don’t have the infrastructure for doing this type of business, join as an indirect reseller.   Minimum requirements for direct partners include: A service business model, a customer support infrastructure, customer billing and invoicing capabilities, and the ability to scale.  There is a deeper commitment required to be a direct partner, and details on support requirement, billing and invoicing, managing customers, incentives and licensing can be found here.

Enroll in the CSP Program

After you understand the requirements and commitment, apply now and we’ll review your application.  Please note it can take several days to review and verify your information.

Have questions, please email azgovcsp@microsoft.com and join us on Yammer.
Quelle: Azure

da Agency

Deep learning for predictive maintenance with Long Short Term Memory Networks

Deep learning has proven to show superior performance in certain areas such as object recognition and image classification. It has also gained popularity in other domains such as finance where time-series data plays an important role. Similarly, in predictive maintenance, the data is collected over time to monitor the state of an asset with the goal of finding patterns to predict failures which can benefit from certain deep learning algorithms. Among the deep learning networks, Long Short Term Memory (LSTM) networks are especially appealing to the predictive maintenance domain since they are very good at learning from sequences. This fact lends itself to their applications using time series data by making it possible to look back for longer periods of time to detect failure patterns.

The traditional predictive maintenance machine learning models are based on feature engineering which is manual construction of right features using domain expertise and similar methods. This usually makes these models hard to reuse since feature engineering is specific to the problem scenario and the available data which varies from one business to the other. Perhaps the most attractive part of applying deep learning in the predictive maintenance domain is the fact that these networks can automatically extract the right features from the data, eliminating the need for manual feature engineering. However, determining the topology of deep learning networks such as deciding on the optimal number of layers, number of nodes and hyperparameters is also considered very labor intensive with no clear guidance.

In the notebook Deep Learning Basics for Predictive Maintenance, we build an LSTM network for the data set and scenario described at Predictive Maintenance Template to predict remaining useful life of aircraft engines using the Turbofan Engine Degradation Simulation Data Set. This notebook serves as a tutorial for beginners looking to apply deep learning in predictive maintenance domain and uses simulated aircraft sensor values to predict when an aircraft engine will fail in the future so that maintenance can be planned in advance.
Quelle: Azure

da Agency

Predictive maintenance using PySpark

Predictive maintenance is one of the most common machine learning use cases and with the latest advancements in information technology, the volume of stored data is growing faster in this domain than ever before which makes it necessary to leverage big data analytic capabilities to efficiently transform large amounts of data into business intelligence. Microsoft has published a series of learning materials including blogs, solution templates, modeling guides and sample tutorials in the domain of predictive maintenance. Recently, we extended those materials by providing a detailed step-by-step tutorial of using Spark Python API PySpark to demonstrate how to approach predictive maintenance for big data scenarios. The tutorial covers typical data science steps such as data ingestion, cleansing, feature engineering and model development.

Business Scenario and Data

The input data is simulated to reflect features that are generic for most of the predictive maintenance scenarios. To enable the tutorial to be completed very quickly, the data was simulated to be around 1.3 GB but the same PySpark framework can be easily applied to a much larger data set. The data is hosted on a publicly accessible Azure Blob Storage container and can be downloaded by clicking this link. In this tutorial, we import the data directly from the blob storage.

The data set has around 2 million records with 172 columns simulated for 1900 machines collected over 4 years. Each machine includes a device which stores data such as warnings, problems and errors generated by the machine over time. Each record has a Device ID and time stamp for each day and aggregated features for that day such as total number of a certain type of warning received in a day. Four categorical columns were also included to demonstrate generic handling of categorical variables. The goal is to predict if a machine will fail in the next 7 days. The last column of the data set indicates if a failure occurred on that day.

Jupyter Notebooks

There are three Jupyter Notebooks on the GitHub repository. To visit the repository, click the green "View Tutorial" button at right of the gallery page.

Notebook_1_DataCleansing_FeatureEngineering
Notebook_2_FeatureEngineering_RollingCompute
Notebook_3_Labeling_FeatureSelection_Modeling

We formatted this tutorial as Jupyter notebooks because it is easy to show the step-by-step process this way. You can also easily compile the executable PySpark script(s) using your favorite IDE.

Specifications & Configurations

The hardware used in this tutorial is a Linux Data Science Virtual Machine with 32 cores and 448 GB memory. For more detailed information of the Data Science Virtue Machine, please visit the link. For the size of the data used in this tutorial (1.3 GB), a machine with less cores and memory would also be adequate. However, in real life scenarios, one should choose the hardware configuration that is appropriate for the specific big data use case. Jupyter Notebooks included in this tutorial can also be downloaded and run on any machine that has PySpark enabled.

The Spark version installed on the Linux Data Science Virtual Machine for this tutorial is 2.0.2 with Python version 2.7.5. Please see the tutorial page for some configurations that needs to be performed before running this tutorial on a Linux machine.

Prerequisites

The user should already know some basics of PySpark. This is not meant to be a PySpark 101 tutorial.
Have PySpark (Spark 2.0., Python 2.7) already configured. Please note if you are using Python 3 on your machine, a few functions in this tutorial require some very minor tweaks because some Python 2 functions deprecated in Python 3.

References

Blog post: Predictive Maintenance Modelling Guide in the Cortana Intelligence Gallery
Predictive Maintenance Modelling Guide
Predictive Maintenance Modelling Guide R Notebook
Predictive Maintenance Modelling Guide Python Notebook
Predictive Maintenance solution
Predictive Maintenance Template

Acknowledgement

Special thanks to Said Bleik, Yiyu Chen and Ke Huang for learning PySpark together. Thank Fidan Boylu Uz and Danielle Dean for proof reading and modifying the tutorial materials.
Quelle: Azure

da Agency

Azure Database for PostgreSQL team hosts Ask Me Anything session

The Azure Database for PostgreSQL team will host a special Ask Me Anything session on /r/Azure, Thursday, June 22, 2017 from 9:00 am to 1:00 pm Pacific Time.

What's an AMA session?

We'll have folks from our Azure Database for PostgreSQL engineering team available to answer any questions you have. You can ask us anything about our product or even our team!

Why are you doing an AMA?

We have done a few AMAs for Microsoft Databases, Azure cloud as well as on-premises, over the past several months and have received some great questions and valuable feedback from you! We are now extending our scope to the recently announced Azure Database for PostgreSQL service and want to hear from you. Your questions will provide us insight into how we can continue to make your experience better.

Who will be there?

You, of course! We'll also have PMs and Developers from the Engineering team participating throughout the day.

Have any questions about Azure Database for PostgreSQL? Bring them to the AMA!

Why should I ask questions here instead of StackOverflow, MSDN, or Twitter? Can I really ask anything?

An AMA is a great place to ask us anything. StackOverflow and MSDN have restrictions on which questions can be asked while Twitter only allows 140 characters. With an AMA, you’ll get answers directly from the team and have a conversation with the people who build this product.

Here are some question ideas:

What are the advantages of using Azure Database for PostgreSQL instead of a local PostgreSQL server?
What aspects of the database service are managed by Azure?
Do I need to worry about minor version upgrades of the database engine?

Go ahead, ask us anything about our public products or the team. Please note, we cannot comment on unreleased features and future plans.

Join us! We're looking forward to having a conversation with you!
Quelle: Azure

da Agency

Managing updates for your Azure VM

In this blog post, I will talk about how to use Update Management solution to manage updates for your Azure VMs. Right from within your Azure VM you can quickly assess the status of available updates, initiate the process of installing required updates, and review deployment results to verify that updates were applied successfully to the VM.

This feature is currently in private preview. If you’re interested in giving it a try, please sign up!

Enabling Update Management

From your VM, you can select “Manage Updates” on the virtual machines blade, under Automation + Control. After selecting it, validation is performed to determine if the Update Management solution is enabled for this VM. If it is not enabled, you will have the option to enable the solution.

The solution enablement process can take up to 15 minutes, and during this time you should not close the browser window. Once the solution is enabled and log data starts to flow to the workspace, it can take more than 30 minutes for data to be available for analysis in the dashboard described in the next section. We expect this timing to significantly improve in the future.

Review update assessment

From the Manage Updates dashboard, you can review the update compliance state of the VM from the Missing updates by severity tile, which displays a count and graphical representation of the number of updates missing on the VM. The table below shows how the tile categorizes the updates missing by update classification.

To create an update deployment and bring the VM into compliance, you configure a deployment that follows your release schedule and service window. This entails what update types to include in the deployment, such as only critical or security updates, or if you want to exclude certain updates.

Create a new Update Deployment for the VM by clicking the “Schedule deployment for this VM” button at the top of the blade and specify the required values. 

After you have completed configuring the schedule, click the “OK” button and you return to the status dashboard. You will notice that the Scheduled table shows the deployment schedule you just created.

View update deployment state

When the scheduled deployment executes, you see the status appear for that deployment under the Completed and in-progress table. Double-clicking the completed update deployment takes you to the detailed deployment status page.

To review all detailed activities performed as part of the update deployment, select “All Logs and Output tiles”. This will show the job stream of the runbook responsible for managing the update deployment on the target VM.

OS support

Windows: Windows 2012 and above
Linux: RedHat Linux 6 & 7, Ubuntu Server 12.04 LTS, 14.04 LTS, 15.10, and 16.04

New to OMS Update Management

If you are new to OMS Update Management, you can view the current capabilities which include Update Insights across Windows and Linux, and the ability to deploy updates, as well as documentation.

In future posts, I’ll talk about how to manage updates for multiple VMs in your subscription and how to orchestrate the update deployments including running pre/post steps, sequencing, and much more!
Quelle: Azure

da Agency

Enable client side monitoring in Azure with Application Insights

With the Application Insights JavaScript SDK you can collect and investigate the performance and usage of your web page or app. Historically we have offered onboarding through manually adding a script to your application and redeploying. Manually adding the script is still supported, but recently we have added the ability to add client-side monitoring from the Azure portal in a few clicks as well.

Enablement

If you have enabled Application Insights in Azure, you can add page view and user telemetry. You can learn how to switch on server-side monitoring in our documentation.

     1. Select Settings -> Application Settings

     2. Under App Settings, add a new key value pair:

Key: APPINSIGHTS_JAVASCRIPT_ENABLED

Value: true

 

     3. Save the settings and Restart your app in the Overview tab.

 

The Application Insights JavaScript SDK is now injected into each web page.

Feedback

If you have any questions or experiencing any problems with the JavaScript SDK, feel free to open an issue on GitHub.
Quelle: Azure

da Agency

GDPR Questions? Azure has answers.

Microsoft is here to help

Please have a look at our white paper How Microsoft Azure Can Help Organizations Become Compliant with the EU General Data Protection Regulation to gain an understanding of how your organization can use currently available features in Azure to optimize your preparation for GDPR compliance. We are here to help you with your compliance efforts in the face of the coming EU law.

May 25, 2018: a new era begins for data privacy

On this date in a little less than a year, the new European Union (EU) data protection law will be implemented, replacing the old Data Protection Directive, which has been in effect since 1995. The new law, known as the General Data Protection Regulation (GDPR), gives individuals greater control over their personal data and imposes many new obligations on organizations that collect, handle, or analyze personal data.

This is what we do

Azure has developed a tradition of compliance which gives our customers the tools they need to comply with complex regulations. Our attention to, and preparation for the impact of GDPR continues to show how we equally prioritize the best cloud technology with the best compliance offerings.

Additional information about how Microsoft helps you to fulfill specific GDPR requirements are available at the GDPR section of our Microsoft Trust Center.
Quelle: Azure

da Agency

Azure Site Recovery now supports managed disks

Azure Site Recovery (ASR) now supports managed disks. This follows the announcement of Azure’s support for managed disks in February. With the integration of managed disks into ASR, you can attach managed disks to your machines during a failover or migration to Azure. 

Managed disks provide the following advantages:

Simplified disk management for Azure IaaS VMs by removing the hassle of managing storage accounts for your machines after failover to Azure.
Improved reliability for Availability Sets by ensuring that the disks of the failed over VMs are automatically placed in different storage scale units (stamps) to avoid single points of failure.

To attach managed disks to your machine on a failover, set “Use managed disks” to “Yes” in the Compute and Network settings for the virtual machine as shown below.

 

Below are a few considerations to keep in mind when using this feature: 

Managed disks can be created only for virtual machines deployed using the Resource manager deployment model.  
Virtual machines with managed disks can only be part of availability sets with "Use managed disks" property set to "Yes". Learn more about managed disks and availability sets.
If the storage account used for replication was encrypted with Storage Service Encryption (SSE) at any point in time, creation of managed disks during failover will fail. In such a scenario, you can either set "Use managed disks" to "No" in the Compute and Network settings for the virtual machine and retry failover or disable protection for the virtual machine and protect it to a storage account which did not have Storage service encryption enabled at any point in time. Learn more about managed disks and Storage service encryption.
For Hyper-V VM’s managed by/not under the management of System Center VMM, set the option to use managed disks only if you intend to migrate your machine to Azure. This is because failback from Azure to on-premises Hyper-V environment is not currently supported for machines with managed disks.
Data from on-premises VMs replicates to a target storage account in Azure, as is with the experience today. Managed disks are created and attached to your machine only on a failover to Azure.
Disaster Recovery of Azure IaaS machines with managed disks is not supported currently and will be made available in the future.

The latest Deployment Planner tool, version 1.3, supports managed disks. You can download the tool from the ASR Deployment Planner doc. For a complete understanding of how managed disks works, please refer to the detailed Managed disks documentation.

Ready to start using ASR?

Check out additional product information to start replicating your workloads to Microsoft Azure using Azure Site Recovery today. You can use the powerful replication capabilities of Site Recovery for 31 days at no charge for every new physical server or virtual machine that you replicate. Visit the Azure Site Recovery forum on MSDN for additional information and to engage with other customers. You can also use the ASR UserVoice to let us know what features you want us to enable next.

Azure Site Recovery, as part of Microsoft Operations Management Suite, enables you to gain control and manage your workloads no matter where they run, including Azure, AWS, Windows Server, Linux, VMware, or OpenStack, with a cost-effective, all-in-one cloud IT management solution. Existing System Center customers can take advantage of the Microsoft Operations Management Suite add-on, empowering them to do more by leveraging their current investments. Get access to all the new services that OMS offers, with a convenient step-up price for all existing System Center customers. You can also access only the IT management services that you need, enabling you to on-board quickly and have immediate value, paying only for the features that you use.
Quelle: Azure

da Agency

Mesosphere DCOS, Azure, Docker, VMware and everything between – Architecture and CI/CD Flow

These days, I try to be involved in any Containers, DevOps, Automation, etc. related discussion. Part of my role is to consult my customers around how to architect their containers platform and orchestration tools in Azure. But what happens when you have a chance to do something cool like architecting a solution which involves Mesosphere DC/OS, Azure Container Service, Azure Container Registry, Docker, and VMware vSphere?! Let’s find out…

In this first multiple-part blog post series I will describe the motivation behind it, the requirements and constraints, architecture, and of course the “how to” – let’s begin.

Motivation, Requirements & Constraints

The motivation for this one was pretty straightforward – start looking into Docker containers and integrate several applications with it.

Without going into too many details, I had one major constraint – The DevOps team had the production environment deployed on top of vSphere and it’s dev/QA/integration environments in Azure. Why this is constraint you might ask?! Well, you will soon find out.

As for requirements, these are the main ones:

Use Azure Container Service with DC/OS
Store Docker images in the same private container registry which will be used by all parties – dev/QA/integration/Prod
Unified containers orchestration platform across all stacks

Going back to the constraint part for a second, the reason I consider this a constraint is because if the production was also part of Azure, I wouldn’t have to do anything with regard to vSphere and everything was pure “cloudish”.

Dev to Production CI/CD Flow

The continuous integration and deployment flow presented below goes as follows:

A developer does some coding on a container deployed locally on his workstation.
He then pushes an “Integration Ready” docker image to a private container registry.
The integration team pulls the image into a DC/OS cluster deploy on Azure to do some extra integration and testing work. Once done, a new “Production Ready” image is being pushed to the container registry.
The “Production Ready” container is being pulled to the DC/OS production cluster deployed on vSphere.

Architecture

Below is the infrastructure logical design for our deployment which will serve the process previously described. Please note that I will not touch the Visual Studio Team Services (VSTS) or the Team Foundation Server (TFS) in this series as I wanted to focus more on the infra side of things.

In the next blog post in this series, I will be talking about the DC/OS 1.9 deployment on top of vSphere. I went with the advanced way of doing things so I will share my knowledge around the configurations, caveats, and steps needed for such a deployment.
Quelle: Azure