Kategorie: Microsoft Azure

Announcing a new OpenAI feature for developers on Azure 

We are thrilled to announce the launch of OpenAI’s latest model on Azure. This new model, officially named GPT-4o-2024-08-06, brings innovative features designed to elevate developer experiences on Azure. Specifically, the new model focuses on enhancing productivity through Structured Outputs, like JSON Schemas, for the new GPT-4o and GPT-4o mini models. 

Azure OpenAI Service
Build your own copilot and generative AI applications.

Learn more

A focus on Structured Outputs 

GPT-4o was first announced in May 2024, as OpenAI’s new multimodal model, followed by GPT-4o mini in July 2024. Today’s version is designed with a specific use case in mind: simplifying the process of generating well-defined, structured outputs from AI models. This feature is particularly valuable for developers who need to validate and format AI outputs into structures like JSON Schemas. Developers often face challenges validating and formatting AI outputs into well-defined structures like JSON Schemas.  

Structured Outputs addresses this by allowing developers to specify the desired output format directly from the AI model. This feature enables developers to define a JSON Schema for text outputs, simplifying the process of generating data payloads that can seamlessly integrate with other systems or enhance user experiences. 

Use cases for JSON 

JSON Schema is essential for defining the structure and constraints of JSON documents, ensuring they follow specific formats with mandatory properties and value types. It enhances data understandability through semantic annotation and serves as a domain-specific language for optimized application requirements. Development teams use JSON Schema to maintain consistency across platforms, drive model-driven UI constraints, and automatically generate user interfaces. It aids in data serialization, security testing, and partial validation in technical scenarios. JSON Schema also supports automated testing, Schema inference, and machine-readable web profiles, improving data interoperability. It standardizes validation interfaces and reporting, handles external validation, and ensures data consistency within and across documents. It can also help with customer support and how to communicate in a timely manner. 

Two flavors of Structured Outputs 

Structured Outputs is available in two forms: 

User-defined JSON Schema: This option allows developers to specify the exact JSON Schema they want the AI to follow, supported by both GPT-4o-2024-08-06 and GPT-4o-mini-2024-07-18.

More Accurate Tool Output (“Strict Mode”): This limited version lets developers define specific function signatures for tool use, supported by all models that support function calling, including GPT-3.5 Turbo, GPT-4, GPT-4 Turbo, and GPT-4o models from June 2023 onwards. 

Technical guidance on using Structured Outputs 

To help you get started with Structured Outputs, we recommend the following approach. 

Getting started with Structured Outputs 

Define Your JSON Schema: Determine the structure you want your AI outputs to follow. This can include required fields, data types, and other constraints. 

Configure the AI model: Use the Structured Outputs feature to specify your JSON Schema within the API call. This ensures that the AI output adheres to your defined structure. 

Integration and testing: Integrate the output into your application or system, and test thoroughly to ensure compliance with your JSON Schema. 

Example use case: Customer support automation 

Imagine you’re developing a customer support chatbot that needs to generate responses in a specific format for logging and analytics. By using Structured Outputs, you can define a JSON Schema that includes fields like responseText, intent, confidenceScore, and timestamp. This ensures that every response generated by the chatbot is formatted correctly, making it easier to log, analyze, and act upon. 

Example API call 

Here’s an example API call to illustrate how to use Structured Outputs:

{
"model": "gpt-4o-2024-08-06",
"prompt": "Generate a customer support response",
"structured_output": {
"schema": {
"type": "object",
"properties": {
"responseText": { "type": "string" },
"intent": { "type": "string" },
"confidenceScore": { "type": "number" },
"timestamp": { "type": "string", "format": "date-time" }
},
"required": ["responseText", "intent", "confidenceScore", "timestamp"]
}
}
}

Pricing 

We will make pricing for this feature available soon. Please bookmark the Azure OpenAI Service pricing page. 

Learn more about the future of AI

We’ve been rolling out several new models recently, and we understand it can be a lot to keep up with. This flurry of activity is all about empowering developer innovation. Each new model brings unique capabilities and enhancements, helping you build even more powerful and versatile applications. 

The launch of this new model feature for GPT-4o and GPT-4o mini marks a significant milestone in our ongoing efforts to push the boundaries of AI capabilities. We’re excited to see how developers will leverage these new features to create innovative and impactful applications. 

Azure ai studio

Craft AI solutions your way

Stay tuned for more updates and get ready to experience the future of AI with these new developer features for GPT-4o and mini. Start experimenting in the Azure OpenAI Playground. 

Explore Azure OpenAI Service

The post Announcing a new OpenAI feature for developers on Azure  appeared first on Azure Blog.
Quelle: Azure

Build AI-enabled applications with Azure AI and NVIDIA

Learn how Azure AI, combined with NVIDIA AI, can help you create innovative and intelligent solutions using your preferred tools and workflows.

An explosion of interest in generative AI across many industries has sparked, a direct result of the collaboration of Microsoft and NVIDIA and the breakthrough technology behind OpenAI’s ChatGPT. As a result, artificial intelligence (AI) is transforming the way we interact with digital products and services, from chatbots and voice assistants to smart cameras and recommendation systems. It’s now almost a daily demand to leverage the power of AI to create applications that can understand, interact, and learn from their users and environments.

However, building these AI applications presents significant challenges in terms of time, resources, access to AI infrastructure, and costs—which can be prohibitive for many developers and organizations.

To alleviate these challenges, developers can benefit from the combined benefits of Azure AI—a set of cloud-based AI and machine learning services that can help you build, train, and deploy AI-enabled applications with ease—and the NVIDIA AI platform to maximize application performance throughout every stage of development and deployment. To make access and entry even easier to build the best AI-enabled applications, Microsoft and NVIDIA have launched a collaborative resource for developers and organizations to experience the better together benefits.

Azure AI
Lead your market with multimodal and generative AI

Innovate with AI

In this blog, we’ll discuss how combining the power of both Azure AI and the NVIDIA AI Platform can help you create your most impactful AI-enabled applications, providing you with flexibility, productivity, efficiency, and innovation.

Better together: Microsoft and NVIDIA

Recognizing the barriers developers face, NVIDIA and Microsoft have worked closely to democratize access to the same core technology that powers ChatGPT to accelerate adoption. The partnership focuses on optimizing every layer of the generative AI stack—from highly performant and scalable AI infrastructure to developer-friendly tools and services—to reduce complexity and cost, making advanced AI capabilities more accessible and feasible for a broader range of applications and industries.

Used by more than 60,000 organizations, Azure AI integrates with popular developer environments Visual Studio Code and GitHub, allowing you to use your preferred tools and workflows to develop, test, and deploy your AI solutions. Whether you want to use pre-built models and APIs, or build and train your own custom models, Azure AI can support various AI scenarios including building your own copilot with enterprise chat, speech analytics, document processing automation, and more.

Azure’s leading cloud AI supercomputing infrastructure, leveraging both state of the art NVIDIA GPUs and NVIDIA InfiniBand networking, provides the best performance, scalability, and built-in security needed to build, train, and deploy the most demanding AI workloads with confidence, at any scale. This combination accelerates time to solution, lowers deployment costs by supporting more users with fewer compute resources, and enhances user experience through optimized performance and faster data throughput.

Benefits for developers in GitHub and Visual Studio Code

Whether a developer that uses Visual Studio Code or GitHub, Azure AI integrates with your existing development environment, allowing you to use the same tools and workflows that you’re already familiar with.

Some of the benefits of using these AI tools and services for developers in GitHub and Visual Studio Code include:

Flexibility and choice: Choose the AI solution that best suits you, whether using pre-built models and APIs or building and training your own custom models. Choose the framework and language that you prefer, such as LangChain, Semantic Kernel, TensorFlow, PyTorch, Scikit-learn, Python, or R. You can even use the Azure OpenAI Service to access the latest GPT models from OpenAI. Additionally, folks can use the new Prompty format to work with prompts in their preferred environments (like Visual Studio Code and GitHub) all while using the trusted platform.

Productivity and efficiency: Simplify and accelerate the AI development process by using Visual Studio Code extensions and GitHub Actions. For example, use prompt flow to manage various versions of your flow assets like prompts, code, configurations, and environments via code repo, with tracked changes and rollback to previous versions, promoting a collaborative LLMOps ethos. For machine learning workloads, use GitHub Actions for Azure Machine Learning to automate model training, testing, and deployment.

Performance and scalability: Harness NVIDIA-optimized software from NVIDIA AI Enterprise, available in the Azure marketplace, to streamline workflows and embrace powerful AI capabilities. With support for remote development using Visual Studio Code extensions, you can write, debug, and optimize GPU-accelerated applications—including AI models—while using NVIDIA GPU-powered Azure Virtual Machines.

Innovation and creativity: Build applications that understand, interact, and learn from their users and environments, and that deliver personalized and engaging experiences. Use Azure AI to build a comprehensive generative AI stack and enrich your applications with retrieval-augmented generation, natural language processing, machine learning, and more.

Start building your most innovative applications

The strategic partnership between Microsoft and NVIDIA has significantly enhanced the Azure AI ecosystem. The integration of the NVIDIA AI Enterprise software platform combined with Azure’s AI toolsets and libraries ensures a robust and efficient environment for advancing your AI projects. Accelerate your time to deployment with the optimized NVIDIA Nemotron models, NVIDIA NIM inference microservices, Langchain and Hugging Face integrations, and APIs, inside of your Azure AI environment.

By building AI-enabled applications within the Microsoft ecosystem, developers can benefit from the productivity and efficiency gains that come from using a single, integrated set of tools and services. This can help reduce development time, support costs, and enhance collaboration and communication among team members. You can also benefit from the innovation and creativity that Azure AI enables, allowing you to create applications that understand, interact, and learn from users and environments, and deliver personalized and engaging experiences.

Learn more about how you can streamline development and build AI-enabled applications faster and easier with the combined power of Microsoft and NVIDIA.
The post Build AI-enabled applications with Azure AI and NVIDIA appeared first on Azure Blog.
Quelle: Azure

Embrace the future of container native storage with Azure Container Storage

We are thrilled to announce the general availability of Microsoft Azure Container Storage, the industry’s first platform-managed container native storage service in the public cloud. With Kubernetes driving cloud evolution, we are witnessing a transformative shift as enterprises move from virtual machines (VMs) to containers, optimizing for scalability, flexibility, and cost efficiency. We introduce Azure Container Storage to meet these demands, providing best in class price performance for hosting stateful containers on cloud-based storage and delivering lowest latency on locally attached storage.

Azure Container Storage joins our suite of container services, tightly integrating with Kubernetes and simplifying stateful workload management across Azure’s set of comprehensive storage offerings. Previously, customers needed to retrofit stateful workloads to VM-centric storage options with scalability constraints or deploy self-managed open-source container storage solutions. Since Azure Container Storage is built purposefully for Azure Kubernetes Service (AKS), it simplifies the process, allowing developers to focus on innovating and running applications without worrying about managing storage. With the ability to perform all storage operations directly through Kubernetes APIs—such as creating persistent volumes and scaling up capacity on demand, it eliminates the need to interact with control plane APIs of the underlying infrastructure.

Azure Container Storage

Try it today

Azure Container Storage also streamlines storage management across multiple backing storage options. With its general availability, Azure Container Storage supports Ephemeral Disks (local NVMe and temp SSD) and Azure Disks, just the start in our journey to transform the container storage landscape. Ephemeral Disks support marks a pivotal moment for container users, providing the most comprehensive volume management support for containers on local storage in the cloud. Beyond basic persistent volume (PV) provisioning, Azure Container Storage offers built-in capabilities such as snapshots and autoscaling, capabilities that cannot be found outside of Azure.  

During preview, customers have already begun taking advantage of Azure Container Storage to evolve their business-critical, next-generation solutions. Whether it be optimizing Redpanda cluster performance on Ephemeral Disks or scaling past existing persistent volume limits for Postgres workloads on Azure Disks, Azure Container Storage supports a wide range of workloads. For building stateful applications operating containers, this is just the beginning. Shortly after general availability, we will expand our offerings to include Elastic SAN and later, options like Azure Blobs and Azure Files for shared storage use cases.  

A cloud native solution for all use cases

Azure Container Storage ensures essential resiliency and security for every workload through built-in resiliency design and security enforcements. 

Built-in resiliency: Easily run highly available stateful applications on Azure Container Storage and protect against zonal failures on all levels of the resource hierarchy. You can choose between zone-redundant storage (ZRS) options or multi-zone storage pools on local-redundant storage (LRS) to deliver a highly available solution across zones. For local storage, we optimally place a pod’s persistent volumes on ephemeral disks that exist on the same node as the AKS pod, reducing the number of failure points that could impact your application’s runtime. Moreover, we offer the best balance for availability, cost, and performance—providing the most cost-efficient block storage offering on the cloud with multi-zonal high availability support and sub millisecond read latency. 

Security by default: Security is our top priority. We offer server-side encryption (SSE) with platform-managed keys by default and enforce network security per respective backing storage options. Customers can further enhance security through extensive options, such as SSE with customer-managed keys, per their security standards.

Modernizing existing applications

For any enterprise looking to modernize its applications, Azure Container Storage consolidates management across familiar block storage offerings, simplifies the movement of workloads, and provides continuity in backup and disaster recovery.  

We streamline and consolidate the management experience across our comprehensive portfolio of familiar Azure block storage offerings. Rather than needing to certify and manage multiple container orchestration solutions for each storage resource you deploy, Azure Container Storage efficiently coordinates volume provisioning within a storage pool, a concept we introduce to group storage resources into a unified resource for your AKS cluster. This storage pool can be backed by your preferred storage option, empowering you to choose the most cost-efficient resource tailored to your specific workload performance requirements. For example, Ephemeral Disk, newly introduced as a supported block storage offering for containers, is well-suited for latency-sensitive workloads that benefit from local NVMe or temp SSD storage. KPN, a Dutch telecommunications company, shared their positive experience using Azure Container Storage with local NVMe to host a mail solution on AKS: 

“With Azure Container Storage, we have been able to achieve improved performance in our KPN consumer mail workload by leveraging ephemeral disks and taking advantage of the pooling of resources that Azure Container Storage enables. Instead of the manual configuration of storage, we can focus on running our workloads, and Azure Container Storage will take care of auto-discovering and formatting the NVMe disks, making it simple to use and in line with the Kubernetes way.” 
—Peter Teeninga, Cloud Architect, Personal Cloud by KPN

To make your journey to the cloud as painless as possible we partnered with CloudCasa, a key player in Kubernetes data mobility, to simplify mission-critical migration to the cloud. To continue supporting your cloud estate, we partnered with Kasten, the leading service for data protection for Kubernetes, offering robust backup and disaster recovery capabilities. For more details on our data migration and backup experience provided through our partners, please refer to the later section. 

Building cloud native applications

For application developers building solutions in the cloud, Azure Container Storage offers seamless integration with Kubernetes, providing a container-native experience designed for scalability from the ground up. This ensures that your applications can grow easily and cost-efficiently over time. By supporting industry-standard protocols, such as NVMe-of and iSCSI, we simplify interoperability, providing additional performance options. For instance, you can take advantage of the lower persistent volume attach and detach latencies these protocols offer and achieve rapid scale-out and fast failover. Azure Container Storage allows customers to attach more storage resources to a single VM, increasing the limit to 75 volumes for any VM sizes. The added flexibility increases customers’ ability to optimize Azure resources to meet their cost and performance goals. Sesam, a Norwegian data synchronization and management company, has effectively leveraged this capability to reduce costs by scaling up their persistent volumes more efficiently: 

“Azure Container Storage (ACS) has enabled us to achieve lower total cost of ownership in our workload. We have a large number of pods that need their own persistent volumes, and through the use of Azure Disks and storage pool resource pooling, we are able to fulfill this in a more cost-effective manner, without hitting the limits on the number of disks that can be attached to a node. In practice this makes us able to allocate capacity more easily and more efficiently.”
—Geir Ove Grønmo, Product Manager, Sesam.io 

Data migration and backup support through our trusted partners

Highly efficient and operational storage management is the baseline experience Azure Container Storage strives for. Azure Container Storage tightly integrates with two key third-party solutions—CloudCasa and Kasten—to offer you an integrated migrate, backup, and disaster recovery experience for workloads hosted on stateful containers.  

With the ability to automatically recreate an entire cluster, CloudCasa centralizes the management of cluster recovery and migration, making it easy to move your existing Kubernetes workloads to and within AKS. To modernize your existing workloads on Azure, simply do a full backup of the existing storage resources then set up a restore, indicating Azure Container Storage as the new storage resource for your cluster. 

“With Azure Container Storage, Microsoft has removed much of the management burden from Kubernetes storage, allowing development and DevOps teams to focus on their data and applications. This approach enables organizations to more easily operate stateful production applications at scale. We are pleased to have worked with the Azure Container Storage team to certify CloudCasa for backup and recovery of stateful applications running on it, and to provide a jointly tested solution for easy migration to it.”
—Bob Adair, Head of Product Management, CloudCasa By Catalogic

Kasten automates the end-to-end workflow of backup and disaster recovery, protecting your Kubernetes clusters and application operations. When you deploy your storage pool in Azure Container Storage, you can enable Kasten during snapshot setup. Using dynamic policies, Kasten helps you manage backups at scale in a crash-consistent manner. 

“With Azure Container Storage and Kasten by Veeam, organizations can maximize performance, flexibility and resiliency, while protecting their cloud native workloads from ransomware attacks. Kasten by Veeam collaborated with Microsoft Product and Engineering teams to validate provisioning, volume snapshot and restore capabilities on Azure Container Storage to ensure joint Microsoft and Kasten by Veeam customers can backup, protect, and migrate their stateful workloads to Azure Kubernetes Service (AKS). Through our strategic partnership, we simplify organizations’ cloud journeys without sacrificing performance, scalability or resiliency.” 
—Matt Slotten, Principal Solution Architect, Cloud Native Partnerships Kasten by Veeam  

What is new with general availability?

Our announcement builds on the updates we’ve shared throughout our preview, highlighting several differentiated capabilities. We’ve enhanced the resiliency of stateful containers with multi-zone storage pools and volume replication for local NVMe storage pools to protect against availability loss during single node failures. We’ve also added snapshot support across all storage options for backup and disaster recovery. Additionally, we’ve expanded the Ephemeral Disk portfolio from local NVMe to include temp SSD support, enhancing cost efficiency for use cases that can leverage directly attached local storage. With this announcement, we are excited to introduce three new capabilities that will further increase the resiliency and performance of running stateful workloads: 

Enhance the resiliency of your persistent volumes hosted on local NVMe storage (L-series ephemeral disks) with replication support.

Improved persistent volume recovery after a restart of an Azure Kubernetes Service (AKS) cluster.

Customize the performance of your local NVMe storage with new performance tier options. 

Learn more about Azure Container Storage

Get started with installing Azure Container Storage to your AKS cluster! For a comprehensive guide, watch our step-by-step walkthrough video. You can also explore workload samples from our newly launched community repository to create your first stateful application. To learn more, refer to our AKS engineering blog. We encourage everyone to contribute and share your insights as you explore our newest storage offering.  

If you have any questions, please reach out to AskContainerStorage@microsoft.com. Embrace the future of stateful containers with Azure and unlock new possibilities! 

Explore Azure Container Storage capabilities

The post Embrace the future of container native storage with Azure Container Storage appeared first on Azure Blog.
Quelle: Azure

OpenAI’s fastest model, GPT-4o mini is now available on Azure AI

We are also announcing safety features by default for GPT-4o mini, expanded data residency and service availability, plus performance upgrades to Microsoft Azure OpenAI Service.

GPT-4o mini allows customers to deliver stunning applications at a lower cost with blazing speed. GPT-4o mini is significantly smarter than GPT-3.5 Turbo—scoring 82% on Measuring Massive Multitask Language Understanding (MMLU) compared to 70%—and is more than 60% cheaper.1 The model delivers an expanded 128K context window and integrates the improved multilingual capabilities of GPT-4o, bringing greater quality to languages from around the world.

GPT-4o mini, announced by OpenAI today, is available simultaneously on Azure AI, supporting text processing capabilities with excellent speed and with image, audio, and video coming later. Try it at no cost in the Azure OpenAI Studio Playground.

Azure AI
Where innovators are creating the future

Try for free

We’re most excited about the new customer experiences that can be enhanced with GPT-4o mini, particularly streaming scenarios such as assistants, code interpreter, and retrieval which will benefit from this model’s capabilities. For instance, we observed the incredible speed while testing GPT-4o mini on GitHub Copilot, an AI pair programmer that assists you by delivering code completion suggestions in the tiny pauses between keystrokes, rapidly updating recommendations with each new character typed.

We are also announcing updates to Azure OpenAI Service, including extending safety by default for GPT-4o mini, expanded data residency, and worldwide pay-as-you-go availability, plus performance upgrades. 

Azure AI brings safety by default to GPT-4o mini

Safety continues to be paramount to the productive use and trust that we and our customers expect.

We’re pleased to confirm that our Azure AI Content Safety features—including prompt shields and protected material detection— are now ‘on by default’ for you to use with GPT-4o mini on Azure OpenAI Service.

We have invested in improving the throughput and speed of the Azure AI Content Safety capabilities—including the introduction of an asynchronous filter—so you can maximize the advancements in model speed while not compromising safety. Azure AI Content Safety is already supporting developers across industries to safeguard their generative AI applications, including game development (Unity), tax filing (H&R Block), and education (South Australia Department for Education).

In addition, our Customer Copyright Commitment will apply to GPT-4o mini, giving peace of mind that Microsoft will defend customers against third-party intellectual property claims for output content.

Azure AI now offers data residency for all 27 regions

From day one, Azure OpenAI Service has been covered by Azure’s data residency commitments.

Azure AI gives customers both flexibility and control over where their data is stored and where their data is processed, offering a complete data residency solution that helps customers meet their unique compliance requirements. We also provide choice over the hosting structure that meets business, application, and compliance requirements. Regional pay-as-you-go and Provisioned Throughput Units (PTUs) offer control over both data processing and data storage.

We’re excited to share that Azure OpenAI Service is now available in 27 regions including Spain, which launched earlier this month as our ninth region in Europe.

Azure AI announces global pay-as-you-go with the highest throughput limits for GPT-4o mini

GPT-4o mini is now available using our global pay-as-you-go deployment at 15 cents per million input tokens and 60 cents per million output tokens, which is significantly cheaper than previous frontier models.

We are pleased to announce that the global pay-as-you-go deployment option is generally available this month, allowing customers to pay for the resources they consume, making it flexible for variable workloads, while traffic is routed globally to provide higher throughput, and still offering control over where data resides at rest.

Additionally, we recognize that one of the challenges customers face with new models is not being able to upgrade between model versions in the same region as their existing deployments. Now, with global pay-as-you-go deployments, customers will be able to upgrade from existing models to the latest models.

Global pay-as-you-go offers customers the highest possible scale, offering 15M tokens per minute (TPM) throughput for GPT-4o mini and 30M TPM throughput for GPT-4o. Azure OpenAI Service offers GPT-4o mini with 99.99% availability and the same industry leading speed as our partner OpenAI.

Azure AI offers leading performance and flexibility for GPT-4o mini

Azure AI is continuing to invest in driving efficiencies for AI workloads across Azure OpenAI Service.

GPT-4o mini comes to Azure AI with availability on our Batch service this month. Batch delivers high throughput jobs with a 24-hour turnaround at a 50% discount rate by using off-peak capacity. This is only possible because Microsoft runs on Azure AI, which allows us to make off-peak capacity available to customers.

We are also releasing fine-tuning for GPT-4o mini this month which allows customers to further customize the model for your specific use case and scenario to deliver exceptional value and quality at unprecedented speeds. Following our update last month to switch to token based billing for training, we’ve reduced the hosting charges by up to 43%. Paired with our low price for inferencing, this makes Azure OpenAI Service fine-tuned deployments the most cost-effective offering for customers with production workloads.

With more than 53,000 customers turning to Azure AI to deliver breakthrough experiences at impressive scale, we’re excited to see the innovation from companies like Vodafone (customer agent solution), the University of Sydney (AI assistants), and GigXR (AI virtual patients). More than 50% of the Fortune 500 are building their applications with Azure OpenAI Service.

We can’t wait to see what our customers do with GPT-4o mini on Azure AI!

1GPT-4o mini: advancing cost-efficient intelligence | OpenAI
The post OpenAI’s fastest model, GPT-4o mini is now available on Azure AI appeared first on Azure Blog.
Quelle: Azure

Harnessing the full power of AI in the cloud: The economic impact of migrating to Azure for AI readiness

As the digital landscape rapidly evolves, AI stands at the forefront, driving significant innovation across industries. However, to fully harness the power of AI, businesses must be AI-ready; this means having defined use-cases for their AI apps, being equipped with modernized databases that seamlessly integrate with AI models, and most importantly, having the right infrastructure in place to power and realize their AI ambitions. When we talk to our customers, many have expressed that traditional on-premises systems often fall short in providing the necessary scalability, stability, and flexibility required for modern AI applications.

A recent Forrester study1, commissioned by Microsoft, surveyed over 300 IT leaders and interviewed representatives from organizations globally to learn about their experience migrating to Azure and if that enhanced their AI impact. The results showed that migrating from on-premises infrastructure to Azure can support AI-readiness in organizations, with lower costs to stand up and consume AI services plus improved flexibility and ability to innovate with AI. Here’s what you should know before you start leveraging AI in the cloud.

Challenges faced by customers with on-premises infrastructure

Many organizations who attempted to implement AI on-premises encountered significant challenges with their existing infrastructure. The top challenges with on-premises infrastructure cited were:

Aging and costly infrastructure: Maintaining or replacing aging on-premises systems is both expensive and complex, diverting resources from strategic initiatives.

Infrastructure instability: Unreliable infrastructure impacts business operations and profitability, creating an urgent need for a more stable solution.

Lack of scalability: Traditional systems often lack the scalability required for AI and machine learning (ML) workloads, necessitating substantial investments for infrequent peak capacity needs.

High capital costs: The substantial upfront costs of on-premises infrastructure limit flexibility and can be a barrier to adopting new technologies.

Forrester’s study highlights that migrating to Azure effectively addresses these issues, enabling organizations to focus on innovation and business growth rather than infrastructure maintenance.

Azure AI
Where innovators are creating for the future

Try for free today

Key Benefits

Improved AI-readiness: When asked whether being on Azure helped with AI-readiness, 75% of survey respondents with Azure infrastructure reported that migrating to the cloud was essential or significantly reduced barriers to AI and ML adoption. Interviewees noted that the AI services are readily available in Azure, and colocation of data and infrastructure that is billed only on consumption helps teams test and deploy faster with less upfront costs. This was summarized well by an interviewee who was the head of cloud and DevOps for a banking company:

We didn’t have to go and build an AI capability. It’s up there, and most of our data is in the cloud as well. And from a hardware-specific standpoint, we don’t have to go procure special hardware to run AI models. Azure provides that hardware today.”
—Head of cloud and DevOps for global banking company

Cost Efficiency: Migrating to Azure significantly reduces the initial costs of deploying AI and the cost to maintain AI, compared to on-premises infrastructure. The study estimates that organizations experience financial benefits of USD $500 thousand plus over three years and 15% lower costs to maintain AI/ML in Azure compared to on-premises infrastructure.

Flexibility and scalability to build and maintain AI: As mentioned above, lack of scalability was a common challenge for survey respondents with on-premises infrastructure as well. Respondents with on-premises infrastructure cited lack of scalability with existing systems as a challenge when deploying AI and ML at 1.5 times the rate of those with Azure cloud infrastructure.

Interviewees shared that migrating to Azure gave them easy access to new AI services and the scalability they needed to test and build them out without worrying about infrastructure. 90% of survey respondents with Azure cloud infrastructure agreed or strongly agreed they have the flexibility to build new AI and ML applications. This is compared to 43% of respondents with on-premises infrastructure. A CTO for a healthcare organization said:

After migrating to Azure all the infrastructure problems have disappeared, and that’s generally been the problem when you’re looking at new technologies historically.”
—CTO for a healthcare organization

They explained that now, “The scalability [of Azure] is unsurpassed, so it adds to that scale and reactiveness we can provide to the organization.” They also said: “When we were running on-prem, AI was not as easily accessible as it is from a cloud perspective. It’s a lot more available, accessible, and easy to start consuming as well. It allowed the business to start thinking outside of the box because the capabilities were there.”

Holistic organizational improvement: Beyond the cost and performance benefits, the study found that migration to Azure accelerated innovation with AI by having an impact on the people at all levels of an organization:

Bottoms-up: skilling and reinvestment in employees. Forrester has found that investing in employees to build understanding, skills, and ethics is critical to successfully using AI. Both interviewees and survey respondents expressed difficulty finding skilled resources to support AI and ML initiatives at their organizations.

Migrating to the cloud freed up resources and changed the types of work needed, allowing organizations to upskill employees and reinvest resources in new initiatives like AI. A VP of AI for a financial services organization shared: “As we have gone along this journey, we have not reduced the number of engineers as we have gotten more efficient, but we’re doing more. You could say we’ve invested in AI, but everything we have invested—my entire team—none of these people were new additions. These are people we could redeploy because we’re doing everything else more efficiently.”

Top-down: created a larger culture of innovation at organizations. As new technologies—like AI—disrupt entire industries, companies need to excel at all levels of innovation to succeed, including embracing platforms and ecosystems that help drive innovation. For interviewees, migrating to the cloud meant that new resources and capabilities were readily available, making it easier for organizations to take advantage of new technologies and opportunities with reduced risk.

Survey data indicates that 77% of respondents with Azure cloud infrastructure find it easier to innovate with AI and ML, compared to only 34% of those with on-premises infrastructure. An executive head of cloud and DevOps for a banking organization said: “Migrating to Azure changes the mindset from an organization perspective when it comes to innovation, because services are easily available in the cloud. You don’t have to go out to the market and look for them. If you look at AI, originally only our data space worked on it, whereas today, it’s being used across the organization because we were already in the cloud and it’s readily available.”

Learn more about migrating to Azure for AI-readiness

Forrester’s study underscores the significant economic and strategic advantages of migrating to Azure for be AI-ready. Lower costs, increased innovation, better resource allocation, and improved scalability make migration to Azure a clear choice for organizations looking to thrive in the AI-driven future.

Ready to get started with your migration journey? Here are some resources to learn more:

Read the full Forrester TEI study on migration to Azure for AI-readiness.

The solutions that can support your organization’s migration and modernization goals.

Our hero offerings that provide funding, unique offers, expert support, and best practices for all use-cases, from migration to innovation with AI.

Learn more in our e-book and video on how to migrate to innovate.

Refrences

Forrester Consulting The Total Economic Impact™ Of Migrating to Microsoft Azure For AI-Readiness, commissioned by Microsoft, June 2024

The post Harnessing the full power of AI in the cloud: The economic impact of migrating to Azure for AI readiness appeared first on Azure Blog.
Quelle: Azure

Announcing Phi-3 fine-tuning, new generative AI models, and other Azure AI updates to empower organizations to customize and scale AI applications

AI is transforming every industry and creating new opportunities for innovation and growth. But, developing and deploying AI applications at scale requires a robust and flexible platform that can handle the complex and diverse needs of modern enterprises and allow them to create solutions grounded in their organizational data. That’s why we are excited to announce several updates to help developers quickly create customized AI solutions with greater choice and flexibility leveraging the Azure AI toolchain:

Serverless fine-tuning for Phi-3-mini and Phi-3-medium models enables developers to quickly and easily customize the models for cloud and edge scenarios without having to arrange for compute.

Updates to Phi-3-mini including significant improvement in core quality, instruction-following, and structured output, enabling developers to build with a more performant model without additional cost.

Same day shipping earlier this month of the latest models from OpenAI (GPT-4o mini), Meta (Llama 3.1 405B), Mistral (Large 2) to Azure AI to provide customers greater choice and flexibility.

Unlocking value through model innovation and customization  

In April, we introduced the Phi-3 family of small, open models developed by Microsoft. Phi-3 models are our most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up. As developers look to tailor AI solutions to meet specific business needs and improve quality of responses, fine-tuning a small model is a great alternative without sacrificing performance. Starting today, developers can fine-tune Phi-3-mini and Phi-3-medium with their data to build AI experiences that are more relevant to their users, safely, and economically.

Given their small compute footprint, cloud and edge compatibility, Phi-3 models are well suited for fine-tuning to improve base model performance across a variety of scenarios including learning a new skill or a task (e.g. tutoring) or enhancing consistency and quality of the response (e.g. tone or style of responses in chat/Q&A). We’re already seeing adaptations of Phi-3 for new use cases.

Phi-3 models
A family of powerful, small language models (SLMs) with groundbreaking performance at low cost and low latency

Try today

Microsoft and Khan Academy are working together to help improve solutions for teachers and students across the globe. As part of the collaboration, Khan Academy uses Azure OpenAI Service to power Khanmigo for Teachers, a pilot AI-powered teaching assistant for educators across 44 countries and is experimenting with Phi-3 to improve math tutoring. Khan Academy recently published a research paper highlighting how different AI models perform when evaluating mathematical accuracy in tutoring scenarios, including benchmarks from a fine-tuned version of Phi-3. Initial data shows that when a student makes a mathematical error, Phi-3 outperformed most other leading generative AI models at correcting and identifying student mistakes.

And we’ve fine-tuned Phi-3 for the device too. In June, we introduced Phi Silica to empower developers with a powerful, trustworthy model for building apps with safe, secure AI experiences. Phi Silica builds on the Phi family of models and is designed specifically for the NPUs in Copilot+ PCs. Microsoft Windows is the first platform to have a state-of-the-art small language model (SLM) custom built for the Neural Processing Unit (NPU) and shipping inbox.

You can try fine-tuning for Phi-3 models today in Azure AI.

I am also excited to share that our Models-as-a-Service (serverless endpoint) capability in Azure AI is now generally available. Additionally, Phi-3-small is now available via a serverless endpoint so developers can quickly and easily get started with AI development without having to manage underlying infrastructure. Phi-3-vision, the multi-modal model in the Phi-3 family, was announced at Microsoft Build and is available through Azure AI model catalog. It will soon be available via a serverless endpoint as well. Phi-3-small (7B parameter) is available in two context lengths 128K and 8K whereas Phi-3-vision (4.2B parameter) has also been optimized for chart and diagram understanding and can be used to generate insights and answer questions.

We are seeing great response from the community on Phi-3. We released an update for Phi-3-mini last month that brings significant improvement in core quality and instruction following. The model was re-trained leading to substantial improvement in instruction following and support for structured output. We also improved multi-turn conversation quality, introduced support for <|system|> prompts, and significantly improved reasoning capability.

The table below highlights improvements across instruction following, structured output, and reasoning.

Benchmarks Phi-3-mini-4k Phi-3-mini-128k Apr ’24 release Jun ’24 update Apr ’24 release Jun ’24 update Instruction Extra Hard 5.7 6.0 5.7 5.9 Instruction Hard 4.9 5.1 5 5.2 JSON Structure Output 11.5 52.3 1.9 60.1 XML Structure Output 14.4 49.8 47.8 52.9 GPQA 23.7 30.6 25.9 29.7 MMLU 68.8 70.9 68.1 69.7 Average 21.7 35.8 25.7 37.6 

We continue to make improvements to Phi-3 safety too. A recent research paper highlighted Microsoft’s iterative “break-fix” approach to improving the safety of the Phi-3 models which involved multiple rounds of testing and refinement, red teaming, and vulnerability identification. This method significantly reduced harmful content by 75% and enhanced the models’ performance on responsible AI benchmarks. 

Expanding model choice, now with over 1600 models available in Azure AI

With Azure AI, we’re committed to bringing the most comprehensive selection of open and frontier models and state-of-the-art tooling to help meet customers’ unique cost, latency, and design needs. Last year we launched the Azure AI model catalog where we now have the broadest selection of models with over 1,600 models from providers including AI21, Cohere, Databricks, Hugging Face, Meta, Mistral, Microsoft Research, OpenAI, Snowflake, Stability AI and others. This month we added—OpenAI’s GPT-4o mini through Azure OpenAI Service, Meta Llama 3.1 405B, and Mistral Large 2.

Continuing the momentum today we are excited to share that Cohere Rerank is now available on Azure. Accessing Cohere’s enterprise-ready language models on Azure AI’s robust infrastructure enables businesses to seamlessly, reliably, and safely incorporate cutting-edge semantic search technology into their applications. This integration allows users to leverage the flexibility and scalability of Azure, combined with Cohere’s highly performant and efficient language models, to deliver superior search results in production.

TD Bank Group, one of the largest banks in North America, recently signed an agreement with Cohere to explore its full suite of large language models (LLMs), including Cohere Rerank.

At TD, we’ve seen the transformative potential of AI to deliver more personalized and intuitive experiences for our customers, colleagues and communities, we’re excited to be working alongside Cohere to explore how its language models perform on Microsoft Azure to help support our innovation journey at the Bank.”
Kirsti Racine, VP, AI Technology Lead, TD.

Atomicwork, a digital workplace experience platform and longtime Azure customer, has significantly enhanced its IT service management platform with Cohere Rerank. By integrating the model into their AI digital assistant, Atom AI, Atomicwork has improved search accuracy and relevance, providing faster, more precise answers to complex IT support queries. This integration has streamlined IT operations and boosted productivity across the enterprise. 

The driving force behind Atomicwork’s digital workplace experience solution is Cohere’s Rerank model and Azure AI Studio, which empowers Atom AI, our digital assistant, with the precision and performance required to deliver real-world results. This strategic collaboration underscores our commitment to providing businesses with advanced, secure, and reliable enterprise AI capabilities.”
Vijay Rayapati, CEO of Atomicwork

Command R+, Cohere’s flagship generative model which is also available on Azure AI, is purpose-built to work well with Cohere Rerank within a Retrieval Augmented Generation (RAG) system. Together they are capable of serving some of the most demanding enterprise workloads in production. 

Earlier this week, we announced that Meta Llama 3.1 405B along with the latest fine-tuned Llama 3.1 models, including 8B and 70B, are now available via a serverless endpoint in Azure AI. Llama 3.1 405B can be used for advanced synthetic data generation and distillation, with 405B-Instruct serving as a teacher model and 8B-Instruct/70B-Instruct models acting as student models. Learn more about this announcement here.

Mistral Large 2 is now available on Azure, making Azure the first leading cloud provider to offer this next-gen model. Mistral Large 2 outperforms previous versions in coding, reasoning, and agentic behavior, standing on par with other leading models. Additionally, Mistral Nemo, developed in collaboration with NVIDIA, brings a powerful 12B model that pushes the boundaries of language understanding and generation. Learn More.

And last week, we brought GPT-4o mini to Azure AI alongside other updates to Azure OpenAI Service, enabling customers to expand their range of AI applications at a lower cost and latency with improved safety and data deployment options. We will announce more capabilities for GPT-4o mini in coming weeks. We are also happy to introduce a new feature to deploy chatbots built with Azure OpenAI Service into Microsoft Teams.  

Enabling AI innovation safely and responsibly  

Building AI solutions responsibly is at the core of AI development at Microsoft. We have a robust set of capabilities to help organizations measure, mitigate, and manage AI risks across the AI development lifecycle for traditional machine learning and generative AI applications. Azure AI evaluations enable developers to iteratively assess the quality and safety of models and applications using built-in and custom metrics to inform mitigations. Additional Azure AI Content Safety features—including prompt shields and protected material detection—are now “on by default” in Azure OpenAI Service. These capabilities can be leveraged as content filters with any foundation model included in our model catalog, including Phi-3, Llama, and Mistral. Developers can also integrate these capabilities into their application easily through a single API. Once in production, developers can monitor their application for quality and safety, adversarial prompt attacks, and data integrity, making timely interventions with the help of real-time alerts.

Azure AI uses HiddenLayer Model Scanner to scan third-party and open models for emerging threats, such as cybersecurity vulnerabilities, malware, and other signs of tampering, before onboarding them to the Azure AI model catalog. The resulting verifications from Model Scanner, provided within each model card, can give developer teams greater confidence as they select, fine-tune, and deploy open models for their application. 

We continue to invest across the Azure AI stack to bring state of the art innovation to our customers so you can build, deploy, and scale your AI solutions safely and confidently. We cannot wait to see what you build next.

Stay up to date with more Azure AI news

Watch this video to learn more about Azure AI model catalog.

Listen to the podcast on Phi-3 with lead Microsoft researcher Sebastien Bubeck.

The post Announcing Phi-3 fine-tuning, new generative AI models, and other Azure AI updates to empower organizations to customize and scale AI applications appeared first on Azure Blog.
Quelle: Azure