da Agency

Announcing Public Preview of HDInsight HBase on Azure Data Lake Store

On November 21, Microsoft announced the general availability of Azure Data Lake Store. Azure Data Lake Store is a hyperscale cloud storage for big data analytics built to the open Hadoop File System (HDFS) standard. Azure Data Lake Store provides enterprise grade security, including SSL and encryption at rest by default along with role based access control.

Today we are excited to announce the public preview of HDInsight HBase on Azure Data Lake Store. Customers can harness the power of a columnar NoSQL distributed database with the proven performance and infinite scalability of Azure Data Lake Store. Azure Data Lake Store has no limits to capacity so customers will never need to worry about the limitations of their storage system.  Furthermore, customers can store all their data and do all their analytics in one single storage account.

Here are some examples of how to leverage HDInsight HBase on Azure Data Lake Store:

Internet of Things (IoT) – HBase can store billions of real time events coming from sensors, devices, machinery, equipment, and social media. Hadoop with HDInsight can then perform batch analysis on the data that was stored in Azure Data Lake Store.
Web Logs – store and index web logs and clickstream data using HBase. Hadoop with HDInsight can then do batch analysis on this data.
Social Sentiment – use HBase to write and store data from the social sentiment fire hose (e.g. Twitter)

We invite you to learn more through our documentation and getting started guides:

Azure Data Lake Store

Overview of Azure Data Lake Store
Get started with Azure Data Lake Store
More Azure Data Lake Store documentation

HDInsight HBase

Overview of HBase
Get started with HBase
Real-time social sentiment analysis using HBase
More HDInsight documentation

Quelle: Azure

da Agency

Announcing support for additional Blockchain Protocols on Azure

As we approach Consensus 2017, it is with great pleasure that we announce support for complex blockchain network deployments for many more blockchain and distributed ledger protocols on Azure, including HyperLedger Fabric, R3 Corda, Quorum, Chain Core, and BlockApps, to further our goal and meet customers where they are.
Quelle: Azure

da Agency

Azure introduces a new blockchain proof of concept framework for developers

Microsoft is laser-focused on enabling and accelerating enterprise adoption of blockchain technologies. Our blockchain offerings are well known for providing the ability to rapidly and consistently deploy blockchain infrastructure.

However, as our customers and partners began to build their distributed applications, they identified the application layer as an area where Microsoft could take even greater steps to reduce the time and cost associated with blockchain Proof of Concept (PoC) projects. 

More time on smart contracts, less time on “scaffolding”

When our customers and partners estimate the time and costs for developing a blockchain PoC, they often find that it can take 8-12 weeks and cost as much as $300,000. Besides being time consuming and expensive, this is a huge missed opportunity. Quickly understanding the viability of a PoC can accelerate a business’s understanding of blockchain and save the time and cost associated with a less impactful project.

Microsoft identified that most of the time in these PoC projects was spent developing code and building capabilities that surrounded the blockchain, often referred to as “scaffolding.”  That scaffolding typically required building a responsive web client, writing and deploying a gateway API, implementing support for off-chain storage in technologies such as SQL DB, building out reporting and analytics, and integrating identity and key vault services into the solution.

Lower costs and faster time to value with a PoC Framework

We realized there were a common set of challenges related to PoC development that we could address by creating a type of “Proof of Concept Framework” that would dramatically reduce the amount of time needed to build a blockchain PoC.

The framework provides code assets and ARM template driven deployment for all the scaffolding needed for blockchain PoCs, including the blockchain network, a gateway API, a responsive web application, Azure Active Directory integration, Azure Key Vault integration, SQL DB that is configured and collecting on-chain data, and a set of supporting code and services such as a Hashing Service and a Signing Service. The framework uses Azure’s Event Hubs at its core, which provides the ability to readily add new capabilities such as sending raw data to Azure Data Lake or providing transaction data to Azure Search.

The framework also makes it possible to create the web application without writing any code. It uses meta-data provided for smart contracts to dynamically deliver a contextual user experience for participants. Since the framework populates SQL DB as an off-chain store, it enables an organization to leverage existing skills and tools to light up additional capabilities such as APIs, reporting with PowerBI, chat bots, Azure Data Factory, R, and machine learning.

With the framework, customers and partners can focus on creating truly innovative applications that demonstrate the potential of blockchain, and spend less time and resources on integration tasks that required to get even a basic PoC up and running.

At the Consensus Conference in New York, we’re looking forward to the opportunity to demonstrate the framework for the first time, and to connect with customers and partners to discuss how it can help significantly accelerate blockchain PoC development.
Quelle: Azure

da Agency

April 7, 2017 – Azure Thursday’s meeting summary

This blog post was authored by the Azure TSP team.

Meeting Topics: Azure News, Azure Managed Disks, Azure EA Portal, Azure Governance, and Configuration best practices resources.

Speakers: Chris Hanson, Azure TSP and Kyle Wilson, Azure TSP

Discussion notes & key links

1. Azure News: Azure Monitor & Azure Advisor both were released for general availability. You can track Azure service news by following the Service Updates Feed, for high level announcements, and by reading the core Azure Blog, typically longer detailed explanations.

2. Azure Managed Disks: A new way of creating VM disks that is simpler to implement and manage and provides better scale and resilience. Here is a short overview video and a longer overview video (27 minutes).

In addition we have posted a guide for converting to managed disks from traditional storage accounts and page blobs.

3. Azure EA Portal: We talked briefly about the benefits of the EA Portal managing multiple subscriptions and reporting on usage with Resource Tags, project cost tracking. There is a good video series that explains the various roles in the EA Portal and how to use its features.

4. Azure Configuration best practices: We talked about how the Azure Advisor combined with the Azure Enterprise Scaffold Documentation are both good resources for best practices and governance.
Quelle: Azure

da Agency

WannaCrypt attacks: guidance for Azure customers

The recent global ransomware attack, known as WannaCrypt, has brought forward the importance of running a well secured infrastructure. Whether or not you were impacted by the recent WannaCrypt malware, we recommend all Azure customers take the following 8 steps to further protect your organization from attacks like these.

This recent WannaCrypt malware exploits a Service Message Block (SMB) vulnerability (CVE-2017-0145). Customers should immediately install MS17-010 to resolve this vulnerability.
Review all Azure subscriptions that have SMB endpoints exposed to the internet, commonly associated with ports TCP 139, TCP 445, UDP 137, UDP 138. Microsoft recommends against opening any ports to the internet that are not essential to your operations.
Disable SMBv1 – instructions located here: https://aka.ms/disablesmb1
Utilize Windows Update to keep your machines up-to-date with the latest security updates. If you are running Azure Cloud Services (Platform as a Service Web Roles and Worker Roles or Infrastructure as a Service (IaaS)) automatic updates are enabled by default, so there is no further action required.  All Guest OS versions released after March 14th, 2017 contain the MS17-010 update. You can view the update status of your resources on an on-going basis in Azure Security Center.
Use the Azure Security Center to continuously monitor your environment for threats. Collect and monitor event logs and network traffic to look for potential attacks using the Azure Security Center, and check for new security alerts and quickly investigate any threats detected.
Use Network Security Groups (NSGs) to restrict network access. To reduce exposure to attacks, configure NSGs with in-bound rules that restrict access to only required ports. You can use network firewalls from a range of companies for additional security. Azure Security Center provides a view of the security for all your networks in Azure, and helps you identify those with internet accessible endpoints, insufficient NSG protections, and in some cases recommends a firewall solution.
Confirm that anti-malware is deployed and updated. If you are using Microsoft anti-malware for Azure or Windows Defender, Microsoft released an update last week which detects this threat as Ransom:Win32/WannaCrypt. If you are running anti-malware software from any number of security companies, you should confirm with your provider that your are protected. You can also use Azure Security Center to verify that anti-malware, and other critical security controls, are configured for all of your Azure virtual machines.
Configure backups with multifactor authentication. An important part of recovery from any compromise is having a strong backup solution in place. If you are already using Azure Backup, you can recover data if your servers are attacked by ransomware. Only users with valid Azure credentials can access the backups stored in Azure. We also recommend enabling Azure Multi-Factor Authentication to provide an additional layer of security to your backups in Azure.

For a comprehensive look at the Affected Software, Vulnerability Information and Security Update Deployment, see Microsoft Security Bulletin MS17-010.

For more information about this update, see Microsoft Knowledge Base Article 4013389.

Support

For understanding your cloud security state: Azure Security Center

Help for installing updates: Windows Update FAQ

Security solutions for IT professionals: TechNet Security Support and Troubleshooting

Help for protecting your Windows-based computer from viruses and malware: Microsoft Secure
Quelle: Azure

da Agency

In-Memory OLTP Update

In November 2016, we announced the general availability (GA) of In-Memory OLTP technology in Azure SQL Database worldwide. From then on we have been working towards unlocking some more useful features which we believe will significantly help you leverage this technology more efficiently into your applications.

Today we are pleased to announce a string of improvements to In-Memory OLTP technology in Azure SQL Database, which spans multiple avenues and continues to unlock and expand the T-SQL surface area and improve the overall manageability.

The new feature improvements are listed below and are available today in Azure SQL Database and will also be available in SQL Server vNext.  

Increased number of indexes for memory-optimized tables

The limitation of 8 indexes for memory-optimized tables has been eliminated. You can now create as many indexes on memory-optimized tables as you can create on disk-based tables.

Any disk-based table in your database that you could not migrate previously because of this limitation can now be memory-optimized, and thus leverage the performance benefits of In-Memory OLTP technology.  

In-Memory OLTP schema management

The GA of In-Memory OLTP in Azure SQL Database included support for ALTER operations on memory-optimized objects. This enabled you to perform most schema management operations such as adding or dropping columns, managing indexes, and updating natively compiled stored procedures. At that time, the one operation not yet supported was renaming objects.

We have addressed this now by adding support for the system stored procedure sp_rename with memory-optimized tables and natively compiled T-SQL modules. With this, Azure SQL Database now supports all the main schema management operations for In-Memory OLTP.

sp_rename – sp_rename is used to change the name of the user object in the current database. In-Memory OLTP objects such as tables, table columns, natively compiled procedures, etc. can now all be renamed with this stored procedure.

Note that natively compiled T-SQL modules must be created as schema-bound modules. Tables, columns, and modules referenced from a schema-bound module cannot be renamed. Therefore, to rename a table or column referenced from a natively compiled T-SQL module, first drop the module, then perform the rename, and finally recreate the module referencing the new name.

Max commit dependencies limit eliminated

The limit on the maximum number of transactions a given transaction depends on or the number of transactions that depend on a given transaction is eliminated. 

So you will no longer receive the corresponding error message: "41839: Transaction exceeded the maximum number of commit dependencies".

Extended T-SQL support for In-Memory OLTP

In-Memory OLTP in Azure SQL Database at GA supported T-SQL surface area that was on full parity with SQL Server 2016, including support for constructs like DISTINCT, SUBQUERIES, IN, EXISTS, OR, NOT, OUTER JOIN, etc.

We have further extended the T-SQL support and added the following:

CASE – CASE expressions are now fully supported in SELECT statements in natively compiled T-SQL modules. Prior to this you would have to create a table variable to construct the single result set and from this table variable you would then SELECT based on the predicate that defines the CASE.
COMPUTED COLUMNS – Computed columns, as well as indexes on computed columns, are now supported in memory-optimized tables. Additionally, you can now also create indexes on these computed columns in memory-optimized tables.
CROSS APPLY and JSON – The CROSS APPLY operator is now supported in natively compiled T-SQL modules. In addition, all JSON functions and clauses are now supported in natively compiled T-SQL modules and on constraints in memory-optimized tables. Indexes on computed columns allow indexing JSON data. Learn more about optimizing JSON processing using In-Memory OLTP.

Another feature worth mentioning is the support for sp_spaceused. Even though on Azure SQL Database the storage is managed for you by the platform, sp_spaceused will give you a good insight into the storage characteristics of the data stored in memory-optimized tables.

sp_spaceused – sp_spaceused can now be used to understand the storage footprint associated with In-Memory OLTP. A new parameter '@include_total_xtp_storage' has been added to the stored procedure for the same. Get more details about sp_spaceused​.

If you are currently leveraging In-Memory OLTP technology in your application or if you do plan on using it in the future, we sincerely hope that these improvements will enable new scenarios for you whilemaking it easier for you to leverage this technology in your application.
Quelle: Azure

da Agency

Application Insights support for Microservices and Containers

We’ve recently made it much easier to monitor microservices and containerized applications using Azure Application Insights. A single Application Insights resource can be used for all the components of your application. The health of the services and the relationships between them are displayed on a single Application Map. You can trace individual operations through multiple services with automatic HTTP correlation. Metrics from Docker, and other containers, can be integrated and correlated with application telemetry. Segmenting Application Insights data by role Until now, Application Insights has assumed that you create one Application Insights resource, and instrumentation key, for each server component or microservice in your application. With a microservices application, a single application can be composed of many different services, and it can be very time consuming to create and maintain separate resources. It’s also difficult to correlate results between them. To solve this problem, we have added the capability for you to segment data in Application Insights by the cloud_RoleName property that is attached to all telemetry. This allows you to send all the data from all your servers to a single Application Insights resource, and filter on cloud_RoleName to see performance and health information for individual microservices. The cloud_RoleName property is set by SDKs to represent the appropriate name for your microservice, container, or app name in Azure App Service. In the Failures, Performance, and metrics explorer blades, you will see new Cloud role name properties in the filter menu: In the above example, we filtered the failures blade to show just the information for our front-end web service, filtering out failures from the CRM API backend. We’ve also enabled a preview feature that allows the Application Map to segment server nodes by cloud_RoleName. To enable this capability, set Multi-role Application Map to On from the Previews blade. After enabling this preview, your map will show one server node for each unique value set in the cloud_RoleName field:   The calls between servers are tracked by using correlation Ids passed in the headers of HTTP requests, which we’ll talk about next. Automatic HTTP correlation that works in containers With the latest SDKs, 2.4.0-beta3 for ASP.NET and 2.1.0-beta3 for ASP.NET Core, we automatically correlate calls between services by injecting headers into the HTTP requests and responses. Previously this functionality required installing an instrumenting profiler, using the Application Insights Site Extension for App Services, or installing Status Monitor for other Azure Compute services. These extensions are difficult to provision in Service Fabric or Docker container environments. This allows you to see all your microservices and containerized applications on the application map. You can also see all the telemetry related to cross-server calls in a single view. For example, here is an exception from our visitors app sample: Click through to a correlated list of telemetry for this operation across the front-end web server and the back-end API: To take advantage of this capability, install the current pre-release versions of Application Insights SDKs available on NuGet. SDK Support for .NET Core 2.0, Service Fabric, and Kubernetes We have made improvements to our .NET SDKs so that the above features will work for .NET applications running in Service Fabric and Kubernetes. Our ASP.NET Core 2.1 SDK now supports both .NET Core 1.1 and .NET Core 2.0 Preview 1, including automatic request and dependency tracking capabilities. This allows you to get the full Application Insights experience with the latest versions of .NET Core. If your application runs on Service Fabric, you can use our Service Fabric SDKs for Application Insights. Add the Microsoft.ApplicationInsights.ServiceFabric.Native NuGet package for Service Fabric reliable services and the Microsoft.ApplicationInsights.ServiceFabric package for guest executables and Docker containers. Many Service Fabric applications use EventSource for high-scale logging. You can now log EventSource events to Application Insights by adding the Microsoft.ApplicationInsights.EventSourceListener package. If your application runs in Docker on Kubernetes, you can use our Kubernetes SDK by adding the Microsoft.ApplicationInsights.Kubernetes package to your app. Use NuGet package manager in Visual Studio to add the packages mentioned in this section. Check the “include previews” option to find the packages mentioned in this section. Container metrics for Windows Docker Containers It’s useful to see CPU, Memory, and other metrics about your individual Docker containers so that you can understand the health of your containers and achieve the right density of containers on physical machines. We have enriched our Windows Azure Diagnostics Extension (WAD) with support for collecting metrics from Docker containers running on Windows. Once you have WAD installed, you can simply modify your diagnostics configuration file to collect Docker stats and send them to Application Insights. Try it out today We hope these new capabilities will allow you to have a great experience using Application Insights with microservices and containerized applications. Be sure to check out the docs, try out the SDKs and let us know how we can make Application Insights work better for you in these new environments. In addition to the capabilities listed in this post, we’ve recently announced many other improvements to Application Insights that will help you find and fix issues in your applications. As always, please share your ideas for new or improved features on the Application Insights UserVoice page. For any questions visit the Application Insights Forum.
Quelle: Azure

da Agency

Dear DocumentDB customers, welcome to Azure Cosmos DB!

Dear DocumentDB customers,

We are very excited that you are now a part of the Azure Cosmos DB family!

Azure Cosmos DB, announced at the Microsoft Build 2017 conference, is the first globally distributed, multi-model database service for building planet scale apps. You can easily build globally-distributed applications without the hassle of complex, multiple-datacenter configurations. Designed as a globally distributed database system, Cosmos DB automatically replicates all of your data to any number of regions of your choice, for fast, responsive access. Cosmos DB supports transparent multi-homing and guarantees 99.99% high availability.

Only Cosmos DB allows you to use key-value, graph, and document data in one service, at global scale and without worrying about schema or index management. Cosmos DB allows you to use your favorite API including SQL (Document DB), JavaScript, Gremlin, MongoDB, and Azure Table storage to query your data. As the first and only schema-agnostic database, regardless of the data model, Azure Cosmos DB automatically indexes all your data to eliminate any friction, so you can perform blazing fast queries and focus on your app.

One of the APIs Azure Cosmos DB supports is the SQL (DocumentDB) API and the document data-model. You're already very well familiar with it and already using it to run your current DocumentDB applications. You are already using to run your current DocumentDB applications. These APIs are not changing – the NuGet package, the namespaces, and all dependencies remain the same. You don't need to change anything to continue running your apps built with SQL (DocumentDB) API. You are simply now a part of the service that gives you more capabilities at your disposal.

Why the move to Azure Cosmos DB?

The Cosmos DB project started in 2010 as “Project Florence” to address developer pain-points that are faced by large Internet-scale applications inside Microsoft. Observing that these problems are not unique to Microsoft’s applications, we decided to make Cosmos DB generally available to external developers in 2015 in the form of Azure DocumentDB – the service you’ve been using. The exponential growth of the service has validated our design choices and the unique tradeoffs we have made.

Azure Cosmos DB is the next big leap in globally distributed, at scale, cloud databases. As a DocumentDB customer, you now have access to the new breakthrough system and capabilities offered by Azure Cosmos DB. As a part of this release of Azure Cosmos DB, DocumentDB customers, with their data, are automatically Azure Cosmos DB customers. The transition is seamless and you now have access to all capabilities offered by Azure Cosmos DB. These capabilities are in the areas of the core database engine as well as global distribution, elastic scalability, and industry-leading, comprehensive SLAs.

Specifically, Cosmos DB is all about providing intelligent choices to developers and enabling you to build planet scale apps.

Cosmos DB exposes multiple well-defined consistency models: Databases today only offer two extreme choices for consistency – “strong” consistency and “eventual” consistency. In contrast, Cosmos DB is the first production globally distributed database service to have harvested a set of useful consistency models from decades of research and have operationalized them. Cosmos DB offers five well-defined consistency models which provide clear tradeoffs with respect to latency/availability, backed by SLAs.

Cosmos DB allows developers to model real world in its true form: No data is born relational. Cosmos DB allows developers to store and query their data in its original form. It exposes graph, documents, key-values, column-family data models and will enable others. The multi-model and multi-API capabilities remove the friction, allowing you to build with any data model and API.

Cosmos DB meets developers where they are: Cosmos DB offers a multitude of APIs to access and query data including, SQL and various popular OSS APIs.

What are the extra capabilities you get?

The current developer facing manifestation of this work is the new support for Gremlin and Table Storage APIs. And this is just the beginning… We will be adding other popular APIs and newer data models over time with more advances towards performance and storage at global scale.

It is important to point out that DocumentDB’s SQL dialect has always been just one of the many APIs that the underlying Cosmos DB was capable of supporting. As a developer using a fully managed service like Azure Cosmos DB, the only interface to the service is the APIs exposed by the service. To that end, nothing really changes for you as an existing DocumentDB customer. Azure Cosmos DB offers exactly the same SQL API that DocumentDB did. However, now (and in the future) you can get access to other capabilities, which were previously not accessible.

Another manifestation of our continued work is the extended foundation for global and elastic scalability of throughput and storage. One of the very first manifestations of it is the RU/m but we have more capabilities that we will be announcing in these areas. These new capabilities help reduce costs for our customers for various workloads. Please read our recent blog on RU/m here. We have made several foundational enhancements to the global distribution subsystem. One of the many developer facing manifestations of this work is the consistent prefix consistency model (making in total five well-defined consistency models). However, there are many more interesting capabilities we will release as they mature.

If you still have more questions

Here you can read the answers to the most frequently asked questions by other DocumentDB customers about Cosmos DB experience.

Next Steps

Thank you for being our customers! Cosmos DB wouldn’t be the same without you. We brought together your feedback, decades of distributed systems research combined with superb engineering and craftsmanship to create this service. Azure Cosmos DB is the database of the future – it is what we believe is the next big thing in the world of massively scalable databases! It makes your data available close to where your users are, worldwide. Our mission is to be the most trusted database service in the world and to enable you to build amazingly powerful, cosmos-scale apps, more easily.

Next, we recommend you:

Read these:  Azure Cosmos DB announcement blog and the technical overview blog
Understand the core concepts of Azure Cosmos DB
Learn more about the service and its capabilities by reading the documentation
Visit the pricing page to understand the billing

Try out the new capabilities in Azure Cosmos DB and let us know what you think! If you need any help or have questions or feedback, please reach out to us through askcosmosdb@microsoft.com. Stay up-to-date on the latest Azure Cosmos DB news (#CosmosDB) and features by following us on Twitter @AzureCosmosDB and join our LinkedIn Group. We are really excited to see what you will build with Cosmos DB.

— Your friends at Azure Cosmos DB @AzureCosmosDB
Quelle: Azure