Kategorie: Microsoft Azure

DocumentDB: API for MongoDB now generally available

Today, we are excited to announce that DocumentDB: API for MongoDB is generally available. The API for MongoDB allows developers to experience the power of the DocumentDB database engine with the comfort of a managed service and the familiarity of the MongoDB SDKs and tools. With the announcement of its general availability, we are introducing a suite of new features for improvements in availability, scalability, and usability of the service.

What is API for MongoDB?

DocumentDB: API for MongoDB is a flavor of DocumentDB that enables MongoDB developers to use familiar SDKs, tool chains, and libraries to develop against DocumentDB. MongoDB developers can now enjoy the advantages of DocumentDB, which include auto-indexing, no server management, limitless scale, enterprise-grade availability backed by service level agreements (SLAs), and enterprise-grade customer support.

What’s new?

From preview to general availability, we have reached a few important milestones. We are proud to introduce a number of major feature releases:

Sharded Collections
Global Databases
Read-only Keys
Additional portal metrics

Sharded Collections – By specifying a shard key, API for MongoDB will automatically distribute your data amongst multiple partitions to scale out both storage and throughput. Sharded collections are an excellent option for applications to ingest large volumes of data or for applications that require high throughput, low latency access to date. Sharded collections can be scaled in a matter of seconds in the Azure portal. They can scale to a nearly limitless amount of both storage and throughput.

Global Databases – API for MongoDB now allows you to replicate your data across multiple regions to deliver high availability. You can replicate your data across any of Azure’s 30+ datacenters with just a few clicks from the Azure portal. Global databases are a great option for delivering low latency requests across the world or in preparation for disaster recovery (DR) scenarios. Global databases have support for both manual and policy driven failovers for full user control.

Read-only Keys – API for MongoDB now supports read-only keys, which will only allow read operations on the API for MongoDB database.

Portal Metrics – To improve visibility into the database, we are proud to announce that we have added additional metrics to the Azure portal. For all API for MongoDB databases, we provide metrics on the numbers of requests, request charges, and errored requests. Supplementing the portal metrics, we have also added a custom command, GetLastRequestStatistics, which allows you to programmatically determine a command’s request charge.

What’s next?

General availability is just the beginning for all the features and improvements we have in stored for DocumentDB: API for MongoDB. In the near future, we will be releasing support for Unique indexes and  a couple major performance improvements. Stay tuned!

In addition to API for MongoDB’s general availability, we are announcing a preview Spark connector. Visit our Github repo for more information.

We hope you take advantage of these new features and capabilities. Please continue to provide feedback on what you want to see next. Try out DocumentDB: API for MongoDB today by signing up for a free trial and create a API for MongoDB account.

Stay up-to-date on the latest Azure DocumentDB news and features by following us on Twitter @DocumentDB.
Quelle: Azure

Announcing new capabilities of HDInsight and DocumentDB at Strata

This week in San Jose, Microsoft will be at Strata Hadoop + World where will be announcing new capabilities of Azure HDInsight, our fully managed OSS analytics platform for running all open-source analytics workloads at scale, with enterprise grade security and SLA and Azure DocumentDB, our planet-scale fully-managed NoSQL database service. Our vision is to deeply integrate both services and make it seamless for developers to process massive amounts of data with low-latency and global scale.

DocumentDB announcements

DocumentDB is Microsoft’s globally distributed database service designed to enable developers to build planet-scale applications. DocumentDB allows you to elastically scale both throughput and storage across any number of geographical regions. The service offers guaranteed single-digit millisecond low latency at the 99th percentile, 99.99% high availability, predictable throughput, and multiple well-defined consistency models—all backed by comprehensive SLAs for latency, availability, throughput, and consistency. By virtue of its schema-agnostic and write-optimized database engine, DocumentDB, by default, is capable of automatically indexing all the data it ingests and serves across SQL, MongoDB, and JavaScript language-integrated queries in a scale-independent manner. As one of the foundational services of Azure, DocumentDB has been used virtually ubiquitously as a backend for first-party Microsoft services for many years. Since its general availability in 2015, DocumentDB is one of the fastest growing services on Azure.

Real-time data science with Apache Spark and DocumentDB

At Strata, we are pleased to announce Spark connector for DocumentDB. It enables real-time data science and exploration over globally distributed data in DocumentDB. Connecting Apache Spark to Azure DocumentDB accelerates our customer’s ability to solve fast-moving data sciences problems where data can be quickly persisted and retrieved using DocumentDB. The Spark to DocumentDB connector efficiently exploits the native DocumentDB managed indexes and enables updateable columns when performing analytics, push-down predicate filtering, and advanced analytics to data sciences against fast-changing globally-distributed data, ranging from IoT, data science, and analytics scenarios. The Spark to DocumentDB connector uses the Azure DocumentDB Java SDK. Get started today and download the Spark connector from GitHub!

General availability of high-fidelity, SLA backed MongoDB APIs for DocumentDB

DocumentDB is architected to natively support multiple data models, wire protocols, and APIs. Today we are announcing the general availability of our DocumentDB’s API for MongoDB. With this, existing applications built on top of MongoDB can seamlessly target DocumentDB and continue to use their MongoDB client drivers and toolchain. This allows customers to easily move to DocumentDB while continuing to use the MongoDB APIs, but get comprehensive enterprise grade SLAs, turn-key global distribution, security, compliance, and a fully managed service.

HDInsight announcements

Cloud-first with Hortonworks Data Platform 2.6

Microsoft’s cloud-first strategy has already shown success with customers and analysts, having recently been placed as a leader in the Forrester Big Data Hadoop Cloud Solutions Wave and a Leader in the Gartner Magic Quadrant for Data Management Solutions for Analytics. Operating a fully managed cloud service like HDInsight, which is backed by enterprise grade SLA, enable customers to deploy the latest bits of Hadoop & Spark, on demand. To that end, we are excited that the latest Hortonworks Data Platform 2.6 will be continuously available to HDInsight even before its on-premises release. Hortonworks’ commitment to being cloud-first is especially significant given the growing importance of cloud with Hadoop and Spark workloads.

"At Hortonworks we have seen more and more Hadoop related work loads and applications move to the cloud. Starting in HDP 2.6, we are adopting a “Cloud First” strategy in which our platform will be available on our cloud platforms – Azure HDInsight at the same time or even before it is available on traditional on-premises settings. With this in mind, we are very excited that Microsoft and Hortonworks will empower Azure HDInsight customers to be the first to benefit from our HDP 2.6 innovation in the near future."
– Arun Murthy, co-founder, Hortonworks

Most secured Hadoop in a managed cloud offering

Last year at Strata + Hadoop World Conference in New York, we announced the highest levels of security for authentication, authorization, auditing, and encryption natively available in HDInsight for Hadoop workloads. Now, we are expanding our security capabilities across other workloads including Interactive Hive (powered by LLAP) and Apache Spark. This allows customers to use Apache Ranger over these popular workloads to provide a central policy and management portal to author and maintain fine-grained access control. In addition, customers can now analyze detailed audit records in the familiar Apache Ranger user interface.

New fully managed, SLA-backed Apache Spark 2.1 offering

With the latest release of Apache Spark for Azure HDInsight, we are providing the only fully managed, 99.9% SLA-backed Spark 2.1 cluster in the market. Additionally, we are introducing capabilities to support real-time streaming solutions with Spark integration to Azure Event Hubs and leveraging the structured streaming connector in Kafka for HDInsight. This will allow customers to use Spark to analyze millions of real-time events ingested into these Azure services, thus enabling IoT and other real-time scenarios. We made this possible through DirectStreaming support, which improves the performance and reliability of Spark streaming jobs as it processes data from Event Hubs. The source code and binary distribution of this work is now available publicly on GitHub.

New data science experiences with Zeppelin and ISV partnerships

Our goal is to make big data accessible for everybody. We have designed productivity experiences for different audiences including the data engineer working on ETL jobs with Visual Studio, Eclipse, and IntelliJ support, the data scientists performing experimentation with Microsoft R Server and Jupyter notebook support, and the business analysts creating dashboards with Power BI, Tableau, SAP Lumira, and Qlik support. As part of HDInsight’s support for the latest Hortonworks Data Platform 2.6, Zeppelin notebooks, a popular workspace for data scientists, will support both Spark 2.1 and interactive Hive (LLAP). Additionally, we have added popular independent software vendors (ISVs) Dataiku and H20.ai to our existing set of ISV applications that are available on the HDInsight platform. Through the unique design of HDInsight edge nodes, customers can spin up these data science solutions directly on HDInsight clusters, which are integrated and tuned out-of-the-box making it easier for customers to build intelligent applications.

Enabling Data Warehouse scenarios through Interactive Hive

Microsoft has been involved from the beginning in making Apache Hive run faster with our contributions to Project Stinger and Tez that sped up Hive query performance up to 100x. We announced support for Hive using LLAP (Long Lived and Process) to speed up query performance up to an additional 25x. With support for the newest version of Apache Hive 2.1.1, customers can expect sub-second query performance, thus enabling data warehouse scenarios over all enterprise data, without the need for data movement. Interactive Hive clusters also support popular BI tools, which is useful for business analysts who want to run their favorite tools directly on top of Hadoop. 

Announcing SQL Server CTP 1.4

Microsoft is excited to announce a new preview for the next version of SQL Server Community Technology Preview (CTP) 1.4 is available on both Windows and Linux. This preview offers an enhancement to SQL Server v.Next on Linux. Another enhancement to SQL Server v.Next on Windows and Linux is resumable online index builds b-tree rebuild support which extends flexibility in index maintenance scheduling and recovery. You can try the preview in your choice of development and test environments now and for additional detail on CTP 1.4, please visit What’s New in SQL Server v.Next, Release Notes and Linux documentation.

Earlier today, we also announced a new online event that will take place next month – Microsoft Data Amp. During the event, Scott Guthrie and Joseph Sirosh will share some exciting new announcements around investments we are making that put data front and center of application innovation and artificial intelligence. I encourage you to check out Mitra Azizirad’s blog post to learn more about Microsoft Data Amp and save the date for what’s going to be an amazing event.

This week the big data world is focused on Strata + Hadoop World in San Jose, a great event for the industry and community. We are committed to making the innovations in big data and NoSQL natively available, easily accessible, and highly productive as part of our Azure services.
Quelle: Azure

Using templates to customize restored VMs from Azure Backup

Last week, we covered how you can configure backup on Azure VMs using Azure Quickstart templates. In this blog post, we will cover how you can customize the VM that will be created as part of restore operation from Azure backup to match your restore requirements.

Azure Backup provides three ways to restore from VM backup – Create a new VM from VM backup, Restore disks from VM backup and use them to create a VM or instant file recovery from VM backup. While a creating a VM from VM backup creates a restored VM, it will not let you customize the configuration from what is present during backup. If you want a test restore or spin a new VM with a different configuration, you can use restore disks and attach those disks to a different VM configuration using PowerShell. Today, we are happy to announce a feature which provides a customizable template to be deployed along with restore disks option which lets you customize the configuration for restore VM.

Customizing restored VM:

You can use restore disks option to customize parameters which are not possible with create a new VM option as part of restore process. Create VM option will generate unique identifiers and use them for some resource names to guarantee a restored success. If you want to customize or add new parameters as part of restore process, you can restore disks and use the template generated as part of restoring disks to customize the restored VM as per your requirements. This will also enable you to create VM with your choice of configuration from restored disks more seamlessly or help you to restore a VM to different network settings to test restore periodically at your environment.

Once you trigger restore job using Restore disks option, Azure Backup will copy data from its vault to storage account selected. Once this job is completed, you can go to corresponding restore job to find the template generated. This will be stored under parameter Template Blob Uri. Using the path mentioned, go to specified storage account and container, to download the template.  Once downloaded, you can use it in Azure template deployment to trigger a new VM creation. By default, template will have few parameters like Vnet Name, Public IP name, OS Disk name, Data Disk name prefix, NIC name prefix and Availability set option(only available if your original VM is part of availability set). If you want to specify a different configurations parameters, edit the template and submit the template for deployment.

Template will be provided for all non-encrypted standard and premium non-managed disk VMs and we will add support for encrypted and Managed Disks VMs in coming release. Please let us know your feedback at azurevmrestore@service.microsoft.com.

Related links and additional content

Want more details? Check out Azure Backup documentation and Azure Template walkthrough
Browse through Azure Quickstart templates for sample templates
Learn more about Azure Backup
Need help? Reach out to Azure Backup forum for support
Tell us how we can improve Azure Backup by contributing new ideas and voting up existing ones.
Follow us on Twitter @AzureBackup for the latest news and updates

Quelle: Azure

Disaster recovery for applications, not just virtual machines using Azure Site Recovery

Let’s say your CIO stops by one day and asks you, “What if we are hit by an unforeseen disaster tomorrow? Do you have the confidence to be able to run our critical applications on the recovery site, and guarantee that our users will be able to connect to their apps and conduct business as usual?” Note that your CIO is not going to ask you about just recovering your servers or virtual machines, the question is always going to be about recovering your applications successfully. So why is it that many disaster recovery offerings stop at just booting up your servers, and offer no promise of actual end to end application recovery? What makes Azure Site Recovery different that allows you as the business continuity owner to sleep better? To answer this, let’s first understand what an application constitutes: A typical enterprise application comprises of multiple virtual machines spanning different application tiers. These different application tiers mandate write-order fidelity for data correctness. The application may also require its virtual machines to boot up in a particular sequence for proper functioning. A single tier will likely have two or more virtual machines for redundancy and load balancing. The application may have different IP address requirements, either use DHCP or require static IP addresses. Few virtual machines may require a public IP address or DNS routing for end user internet access. Few virtual machines may need specific ports to be open or have security certificate bindings. The application may rely on user authentication via an identity service like Active Directory. To recover your applications in the event of a disaster, you need a solution that facilitates all of the above, gives you the flexibility to potentially do more application specific customizations post recovery, and do everything at an RPO and RTO that meets your business needs. Using traditional backup solutions to achieve true application disaster recovery is extremely cumbersome, error prone and not scalable. Even many replication based software only recover individual virtual machines and cannot handle the complexity of bringing up a functioning enterprise application. Azure Site Recovery combines a unique cloud-first design with a simple user experience to offer a powerful solution that lets you recover entire applications in the event of a disaster. How do we achieve this? With support for single and multi-tier application consistency and near continuous replication, Azure Site Recovery ensures that no matter what application you are running, shrink-wrapped or homegrown, you are assured of a working application when a failover is issued. Many vendors will tell you that having a crash-consistent disaster recovery solution is good enough, but is it really? With crash consistency, in most cases, the operating system will boot. However, there are no guarantees that the application running in the virtual machines will work because a crash-consistent recovery point does not ensure correctness of application data. As an example, if a transaction log has entries that are not present in the database, then the database software needs to rollback until the data is consistent, in the process significantly increasing your RPO. This will cause a multi-tier application like SharePoint to have very high RTO, and even after the long wait it is still uncertain that all features of the application will work properly. To avoid these problems, Azure Site Recovery not only supports application consistency for a single virtual machine (application boundary is the single virtual machine), we also support application consistency across multiple virtual machines that compose the application. Most multi-tier real-world applications have dependencies, e.g. the database tier should come up before the app and web tiers. The heart and soul of the Azure Site Recovery application recovery promise is extensible recovery plans, that allow you to model entire applications and organize application aware recovery workflows. Recovery plans are comprised of the following powerful constructs: Parallelism and sequencing of virtual machine boot up to ensure the right recovery order of your n-tier application. Integration with Azure Automation runbooks that automate necessary tasks both outside of and inside the recovered virtual machines. The ability to perform manual actions to validate recovered application aspects that cannot be automated. Your recovery plan is what you will use when you push the big red button and trigger a single-click stress free end to end application recovery when needed, with a low RTO. Another key challenge for many of these multi-tier applications to function properly is network configuration post recovery. With advanced network management options to provide static IP addresses, configure load balancers, or use traffic manager to achieve low RTOs, Azure Site Recovery ensures that user access to the application in the event of a failover is seamless. A common myth around protecting your applications is the fact that many applications come with in-built replication technologies – hence the question, why do you need Azure Site Recovery? The simple answer: Replication != Disaster Recovery Azure Site Recovery is Microsoft’s single disaster recovery product that offers you a choice to work with different first and third-party replication technologies, while providing an in-built replication solution for those applications where there is no native replication construct, or native replication does not meet your needs. As mentioned earlier, getting application data and virtual machines to the recovery site is only a piece of what is takes to bring up a working application. Whether Azure Site Recovery replicates the data or you use the application’s built-in capability for this, Azure Site Recovery does the complex job of stitching together the application, including boot sequence, network configurations, etc., so that you can failover with the single click. In addition, Azure Site Recovery allows you to perform test failovers (disaster recovery drills) without production downtime or replication impact, as well as failback to the original location. All these features work with both Azure Site Recovery replication and with application level replication technologies. Here are a few examples of application level replication technologies Azure Site Recovery integrates with: Active Directory replication SQL Server Always On Availability Groups Exchange Database Availability Groups Oracle Data Guard So, you ask, what does this really mean? Azure Site Recovery provides you with powerful disaster recovery application orchestration no matter whether you choose to use its built-in replication for all application tiers or mix and match native application level replication technologies for specific tiers, e.g. Active Directory or SQL Server. Enterprises have various reasons why they may go with one or the other replication choice, e.g. tradeoffs between no data loss and cost and overhead of having an active-active standby deployment. The next time you get asked, why do you need Azure Site Recovery when say you already have SQL Server Always On Availability Groups, do make sure you clarify that having application data replicated is necessary but not sufficient for disaster recovery, and Azure Site Recovery complements native application level replication technologies to provide you a full end to end disaster recovery solution. We have learnt from our enterprise customers who are protecting hundreds of applications using Azure Site Recovery, what the most common deployment patterns and popular application topologies are. So not only does Azure Site Recovery work with any application, Microsoft tests and certifies popular first and third-party application suites, a list that is constantly growing. As part of this effort to test and provide Azure Site Recovery solution guides for various applications, Microsoft provides a rich Azure Automation library with production-ready, application specific and generic runbooks for most common automation tasks that enterprises need in their application recovery plans. Let’s close with a few examples: An application like SharePoint typically has three tiers with multiple virtual machines that need to come up in the right sequence, and requires application consistency across the virtual machines for all features to work properly. Azure Site Recovery solves this by giving you recovery plans and multi-tier application consistency. Opening a port / adding a public IP / updating DNS on an application’s virtual machine, having an availability set and load balancer for redundancy and load management, are examples of common asks of all enterprise applications. Microsoft solves this by giving you a rich automation script library for use with recovery plans, and the ability to set up complex network configurations post recovery to reduce RTO, e.g. setting up Azure Traffic Manager. Most applications will need an Active Directory / DNS deployed and use some kind of database, e.g. SQL Server. Microsoft tests and certifies Azure Site Recovery solutions with Active Directory replication and SQL Server Always On Availability Groups. Enterprises always have a number of proprietary business critical applications. Azure Site Recovery protects these with in-built replication and lets you test your application’s performance and network configuration on the recovery site using the test failover capability, without production downtime or replication impact. With relentless focus on ensuring that you succeed with full application recovery, Azure Site Recovery is the one-stop shop for all your disaster recovery needs. Our mission is to democratize disaster recovery with the power of Microsoft Azure, to enable not just the elite tier-1 applications to have a business continuity plan, but offer a compelling solution that empowers you to set up a working end to end disaster recovery plan for 100% of your organization’s IT applications. You can check out additional product information and start replicating your workloads to Microsoft Azure using Azure Site Recovery today. You can use the powerful replication capabilities of Azure Site Recovery for 31 days at no charge for every new physical server or virtual machine that you replicate, whether it is running on VMware or Hyper-V. To learn more about Azure Site Recovery, check out our How-To Videos. Visit the Azure Site Recovery forum on MSDN for additional information and to engage with other customers, or use the Azure Site Recovery User Voice to let us know what features you want us to enable next.
Quelle: Azure