Kategorie: Microsoft Azure

Azure Container Registry now generally available

Companies of all sizes are embracing containers as a fast and portable way to lift, shift and modernize into cloud-native apps. As part of this process, customers need a way to store and manage images for all types of container deployments. In November, we announced the preview of Azure Container Registry, which enables developers to create and maintain Azure container registries to store and manage private Docker container images.

Today, we're announcing the general availability of Azure Container Registry supporting a network-close, private registry for Linux and Windows container images. Azure Container Registry integrates well with orchestrators hosted in Azure Container Service, including Docker Swarm, Kubernetes and DC/OS as well as other Azure Services including Service Fabric and Azure App Services. Customers can benefit from using familiar tooling capable of working with the open source Docker Registry v2. Learn more by watching this Azure Container Registry GA video.
Building on the November Preview, we’ve added the following features and capabilities:

Availability in 23 regions, with a global footprint (with more coming)
Repositories, tag, and manifest listing in the Azure Portal
Dual-key password providing key rotation
Nested level repositories
Azure CLI 2.0 support

Global Availability

Container registry is now available globally. As part of our general availability release, all features are now available in all regions.

A full list of the supported regions are:

Australia East
Australia Southeast
Brazil South
Canada Central
Canada East
Central India
Central US

East US 2
East US
Japan East
Japan West
North Central US
North Europe
South Central US

South India
Southeast Asia
UK South
UK West
West Central US
West Europe
West US 2
West US

Multi-Arch Support

With the release of Windows Containers, we’re increasingly seeing customers who want both Windows and Linux images. While the Azure Container Registry supports both Windows and Linux, docker has added the ability to pull a single named image and have it resolve the os version based on the host the image is pulled from. Using multi-arch support, a customer can push both Windows and Linux based tags and their development teams can create their dockerfiles using FROM contoso.comaspnetcorecorpstandard. The Azure Container Registry multi-arch features will pull the appropriate image based on the host it’s called from.

Nested Repositories

Development teams often work in hierarchies and deploy solutions based on collections. The bikesharing team may have a collection of images they wish to group together (bikesharingweb, bikesharingapi), while the headtrax team has their collection (headtraxweb, headtraxapi, headtraxadmin), with a set of corporate images available to all members (aspnet:corpstandard).
The Azure Container Registry supports nested repos to enable teams to group their repos and images to match their development.

Repositories, tags, manifests

Customers have requested visibility into the contents of their registry. With the GA release, you will now have an integrated experience in the Azure portal to view the repositories, images, tags and the contents of manifests associated with an image.

To view repositories and tags you’ve already created in your repository:

Log in to the Azure Portal.
Select "More Services" on the left-side panel.
Search for "Container registries".
Select the registry you want to inspect.
On the left-hand side panel, select "Repositories".

The repositories blade will display a list of all the repositories (including nested registries) that you have created, as well as the images that are stored in these repositories.

If you select a specific image, it will open up a "Tags" blade containing the tags associated with that image. Additionally, if you select a tag, you will have the ability to see the manifest for that image tag.

 

Improved passwords

We have also made improvements for registry admin accounts. While we recommend using a service principal as a best practice, we wanted to improve on the safety of this alternative providing the ability to do key rotation. As such, new container registries will have access to two admin passwords, both of which can be regenerated. Having two passwords allows you to maintain connections by allowing you to swap to another password if you need to regenerate one.

To regenerate passwords, go to the "Access Keys" section of a registry on which you have enabled an Admin user.

Summary

We hope you enjoy the new features and capabilities of container registries. If you have any comments, requests, or issues, you can reach out to us on StackOverflow or log issues at https://github.com/azure/acr/issues
Quelle: Azure

What’s brewing in Visual Studio Team Services: April 2017 Digest

This post series provides the latest updates and news for Visual Studio Team Services and is a great way for Azure users to keep up-to-date with new features being released every three weeks. Visual Studio Team Services offers the best DevOps tooling to create an efficient continuous integration and release pipeline to Azure. With the rapidly expanding list of features in Team Services, teams can start to leverage it more efficiently for all areas of their Azure workflow, for apps written in any language and deployed to any OS.

Git tags

We’ve now added tag support into the web experience. Instead of creating tags from the command line and pushing the tags to the repository, you can now simply go to a commit and add a tag. The tag creation dialog will also let you tag any other ref in the repo.

Your commits will now show the tags that you have created.

The commit list view also supports a context menu. No need to go to the commit details page to create tags and create new branches.

Soon we will add a page for tag management.

Git branch policy improvements

Branch policies provide a great way to help maintain quality in your repos by allowing you to require a passing build, require code reviewers, and more. As part of review pull requests, users often leave comments. You can now ensure that all comments in pull requests are being addressed with the new Comments policy. Once enabled, active comments will block completion of the PR. Reviewers that leave comments for the PR author but optimistically approve the pull request can be sure that comments won’t be missed.

Sometimes you need to override policies, such as in the middle of the night when addressing an issue in production. Users bypassing pull request policies must now specify a reason. In the Complete pull request dialog, users will see a new Reason field, if they choose to bypass.

After entering the reason and completing the pull request, the message will be displayed in the pull request’s Overview.

Import Team Foundation Version Control into a Git repo

If you’re using Team Foundation Version Control (TFVC) and are looking for an easy way to migrate to Git, try out the new TFVC import feature. Select Import Repository from the repository selector drop-down.

Select TFVC for the source type. Individual folders or branches can be imported to a new Git repository, or the entire TFVC repository can be imported (minus the branches). You can import up to 180 days of history.

Team Foundation Version Control support for Android Studio, IntelliJ, and Rider

We’ve now officially released support for TFVC in Android Studio and the variety of JetBrains IDE’s such as IntelliJ IDEA and Rider EAP. Users can seamlessly develop without needing to switch back and forth from the IDE to the command line to perform their Team Services actions. It also includes additional features that you otherwise wouldn’t get from the command line client, such as seeing an updated status of your repository’s related builds along with the capability to browse work items assigned to you or from your personal queries.

Currently we support:

Checkout a TFVC repository from Team Services or Team Foundation Server 2015+
Execute all basic version control actions such as add, delete, rename, move, etc.
View local changes and history for your files
Create, view, and edit your workspace
Checkin and update local files
Merge conflicts from updates
Lock and unlock files and directories
Add labels to files and directories
Configure a TFS proxy

Check out our brief demo of getting up and running inside of Android Studio. For a more comprehensive look at the plugin, checkout our presentation and tutorial inside of IntelliJ.

To start using the TFVC features, download the latest version of the plugin and follow the setup steps.

Continuous delivery in the Azure portal using any Git repo

You can now configure a continuous delivery (CD) workflow for an Azure App Service for any public or private Git repository that is accessible from the Internet. With a few clicks in the Azure portal, you can set up a build and release definition in Team Services that will periodically check your Git repository for any changes, sync those changes, run an automated build and test, followed by a deployment to Azure App Service.

Start using this feature today by navigating to your app’s menu blade in the Azure portal and clicking Continuous Delivery (Preview) under the App Deployment section.

Conditional build tasks

If you’re looking for more control over your build tasks, such as a task to clean things up or send a message when something goes wrong, we now support four built-in choices for you to control when a task is run:

If you are looking for more flexibility, such as a task to run only for certain branches, with certain triggers, under certain conditions, you can express your own custom conditions:

and(failed(), eq(variables['Build.Reason&039;], &039;PullRequest&039;))

Take a look at the conditions for running a task.

Customizable backlog levels

You can now add backlog levels to manage the hierarchy of their work items and name them in a way that makes sense for your work item types. You can also rename and recolor existing backlog levels, such as Stories or Features. See Customize your backlogs or boards for a process for details on how to get started.

Mobile work item discussion

Our mobile discussion experience has been optimized to provide a mobile-friendly, streamlined experience for submitting a comment. Discussion is the most common action that takes place in a mobile device. We look forward to hearing what you think about our new experience!

Extension of the month

If you are like us, you use open source software in your development projects. Reusing components enables great productivity gains. However, you can also reuse security vulnerabilities or violate licenses without realizing it.

The WhiteSource Bolt extension for build makes it easy to find out whether you are using vulnerable components. After installing it in your account, add it to your build definition and queue a new build. You’ll get a report like the following. In the table under the summary, you will see a list of components with issues and the recommended way to address those issues.

If you have Visual Studio Enterprise, you get 6 months of WhiteSource Bolt for one team project included with your subscription (redeem the code from your benefits page or see this page for VS subscribers for more detailed instructions).

Have a look at the full list of new features by checking out the release notes for March 8th and March 29th.

Happy coding!
Quelle: Azure

Real-time machine learning on globally-distributed data with Apache Spark and DocumentDB

At the Strata + Hadoop World 2017 Conference in San Jose, we have announced the Spark to DocumentDB Connector. It enables real-time data science, machine learning, and exploration over globally distributed data in Azure DocumentDB. Connecting Apache Spark to Azure DocumentDB accelerates our customer’s ability to solve fast-moving data science problems, where data can be quickly persisted and queried using DocumentDB. The Spark to DocumentDB connector efficiently exploits the native DocumentDB managed indexes and enables updateable columns when performing analytics, push-down predicate filtering against fast-changing globally-distributed data, ranging from IoT, data science, and analytics scenarios. The Spark to DocumentDB connector uses the Azure DocumentDB Java SDK. You can get started today and download the Spark connector from GitHub!

What is DocumentDB?

Azure DocumentDB is our globally distributed database service designed to enable developers to build planet scale applications. DocumentDB allows you to elastically scale both, throughput and storage across any number of geographical regions. The service offers guaranteed low latency at P99, 99.99% high availability, predictable throughput, and multiple well-defined consistency models, all backed by comprehensive SLAs. By virtue of its schema-agnostic and write optimized database engine, by default DocumentDB is capable of automatically indexing all the data it ingests and serve SQL, MongoDB, and JavaScript language-integrated queries in a scale-independent manner. As a cloud service, DocumentDB is carefully engineered with multi-tenancy and global distribution from the ground up.
These unique benefits make DocumentDB a great fit for both operational as well as analytical workloads for applications including web, mobile, personalization, gaming, IoT, and many other that need seamless scale and global replication.

What are the benefits of using DocumentDB for machine learning and data science?

DocumentDB is truly schema-free. By virtue of its commitment to the JSON data model directly within the database engine, it provides automatic indexing of JSON documents without requiring explicit schema or creation of secondary indexes. DocumentDB supports querying JSON documents using well-familiar SQL language. DocumentDB query is rooted in JavaScript's type system, expression evaluation, and function invocation. This, in turn, provides a natural programming model for relational projections, hierarchical navigation across JSON documents, self joins, spatial queries, and invocation of user defined functions (UDFs) written entirely in JavaScript, among other features. We have now expanded the SQL grammar to include aggregations, thus enabling globally-distributed aggs in addition to these capabilities.

Figure 1: With Spark Connector for DocumentDB, data is parallelized between the Spark worker nodes and DocumentDB data partitions

Distributed aggregations and advanced analytics

While Azure DocumentDB has aggregations (SUM, MIN, MAX, COUNT, SUM and working on GROUP BY, DISTINCT, etc.) as noted in Planet scale aggregates with Azure DocumentDB, connecting Apache Spark to DocumentDB allows you to easily and quickly perform an even larger variety of distributed aggregations by leveraging Apache Spark. For example, below is a screenshot of calculating a distributed MEDIAN calculation using Apache Spark&039;s PERCENTILE_APPROX function via Spark SQL.

select destination, percentile_approx(delay, 0.5) as median_delay
from df
where delay < 0
group by destination
order by percentile_approx(delay, 0.5)

Figure 2: Area visualization for the above distributed median calculation via Jupyter notebook service on Spark on Azure HDInsight.

Push-down predicate filtering

As noted in the following animated gif, the queries from Apache Spark will push down predicated to Azure DocumentDB and take advantage that DocumentDB indexes every attribute by default. Furthermore, by pushing computation close to the where the data lives, we can do processing in-situ, and reduce the amount of data that needs to be moved. At global scale, this results in tremendous performance speedups for analytical queries.

For example, if you only want to ask for the flights departing from Seattle (SEA), the Spark to DocumentDB connector will:

Send the query to Azure DocumentDB.
As all attributes within Azure DocumentDB are automatically indexed, only the flights pertaining to Seattle will be returned to the Spark worker nodes quickly.

This way as you perform your analytics, data science, or ML work, you will only transfer the data you need.

Blazing fast IoT scenarios

Azure DocumentDB is designed for high-throughput, low-latency IoT environments. The animated GIF below refers to a flights scenario.

Together, you can:

Handle high throughput of concurrent alerts (e.g., weather, flight information, global safety alerts, etc.)
Send this information downstream for device notifications, RESTful services, etc. (e.g., alert on your phone of an impending flight delay) including the use of change feed
At the same time, as you are building up ML models against your data, you can also make sense of the latest information

Updateable columns

Related to the previously noted blazing fast IoT scenarios, let&039;s dive into updateable columns:

As the new piece of information comes in (e.g. the flight delay has changed from 5 min to 30 min), you want to be able to quickly re-run your machine learning (ML) models to reflect this newest information. For example, you can predict the impact of the 30min for all the downstream flights. This event can be quickly initiated via the Azure DocumentDB Change Feed to refresh your ML models.

Next steps

In this blog post, we’ve looked at the new Spark to DocumentDB Connector. The Spark with DocumentDB enables both ad-hoc, interactive queries on big data, as well as advanced analytics, data science, machine learning, and artificial intelligence. DocumentDB can be used for capturing data that is collected incrementally from various sources across the globe. This includes social analytics, time series, game or application telemetry, retail catalogs, up-to-date trends and counters, and audit log systems. Spark can then be used for running advanced analytics and AI algorithms at scale on top of the data coming from DocumentDB.

Companies and developers can employ this scenario in online shopping recommendations, spam classifiers for real time communication applications, predictive analytics for personalization, and fraud detection models for mobile applications that need to make instant decisions to accept or reject a payment. Finally, internet of things scenarios fit in here as well, with the obvious difference that the data represents the actions of machines instead of people.

To get started running queries, create a new DocumentDB account from the Azure Portal and work with the project in our Azure-DocumentDB-Spark GitHub repo. Complete instructions are available in the Connecting Apache Spark to Azure DocumentDB article.

Stay up-to-date on the latest DocumentDB news and features by following us on Twitter @DocumentDB or reach out to us on the developer forums on Stack Overflow.
Quelle: Azure