10 ways to impact business velocity through Azure OpenAI Service

The phrase, “time is money,” is commonly attributed to Benjamin Franklin, who first used it in his essay “Advice to a Young Tradesman,” published in 1748. Franklin was addressing the economic value of time, a concept increasingly relevant when discussing AI’s impact on business today. AI is adept at processing and analyzing troves of data much faster than a human brain—enabling quicker, more informed decision-making. Leaders who embrace AI now and take action to understand it, experiment with it, and envision how it can solve hard problems are going to run companies that thrive in an AI world.1 From automating routine tasks to providing deep insights through data analysis, AI technologies are enabling businesses to make quicker, more informed decisions, driving growth and competitive advantage.

Azure OpenAI Service

Power business efficiency

Learn more

10 ways AI can turbocharge business efficiency

Automating repetitive tasks: AI can handle mundane and repetitive tasks such as data entry, scheduling, and email sorting.

Real-time data analysis: AI algorithms can analyze vast amounts of data in real-time, providing immediate insights and allowing businesses to make faster, data-driven decisions.

Predictive analytics: AI can forecast trends and behaviors based on historical data, enabling companies to anticipate market changes and customer needs more rapidly.

Customer support chatbots: AI-powered chatbots provide instant customer service, addressing inquiries and resolving issues without human intervention.

Supply chain optimization: AI can predict demand, optimize inventory levels, and streamline logistics.

Fraud detection: AI systems can quickly detect and respond to fraudulent activities by analyzing transaction patterns and identifying anomalies in real-time.

Personalized marketing: AI can tailor marketing campaigns to individual preferences and behaviors, increasing engagement and conversion rates more swiftly.

Enhanced recruitment processes: AI can screen resumes, conduct initial interviews, and identify the best candidates faster than traditional methods.

Process automation: Robotic Process Automation (RPA) driven by AI can execute business processes faster and with fewer errors, from financial transactions to regulatory compliance.

Product development: AI accelerates product development cycles by simulating different design scenarios, optimizing prototypes, and predicting performance outcomes.

Below, we look at three Microsoft customers who used Azure OpenAI Service to accelerate the speed at which they do business.  

Akbank empowers staff to search through 10,000 records in seconds with Azure OpenAI ServiceAkbank, one of Türkiye’s largest banks, has significantly improved its customer support operations by integrating Azure OpenAI Service. Whereas their customer representatives previously had to search through a hefty 10,000-article knowledge base in hopes of finding correct responses, they now interact with an AI chatbot that generates correct answers 90% of the time. This integration saves three minutes per interaction, thus enhancing both the quality and accuracy of the support provided. Akbank has also incorporated proactive suggestions into the chatbot, enabling staff to get faster responses and continually improve customer support.

Serving customers 78% faster: VOCALLS’ voicebots supercharge call handing with Azure OpenAIVOCALLS, a Prague and London-based telecommunications company, leverages Microsoft Azure AI technologies to support its customer service with AI-powered voicebots. Specializing in conversational AI solutions, VOCALLS automates over 50 million interactions annually, improving customer experiences for companies like Estafeta. Estafeta, a logistics pioneer in Latin America, saw a 78% reduction in average handling time and a 120% increase in answered calls after deploying VOCALLS’ voicebot, Beatriz. This voicebot provides immediate support, eliminating wait times and boosting customer satisfaction scores.

RepsMate empowers businesses to create super agents by leveraging Azure’s AI superpowersAs a member of the Microsoft for Startups program, RepsMate has leveraged Microsoft’s networks, support, and the Azure Marketplace to gain traction in Eastern Europe. RepsMate’s solution, driven by AI and data analysis, has led to significant efficiency gains, reducing average handling times by 12%, decreasing chat durations by 20 to 30%, and increasing first-call resolution rates by 5 to 10%. Additionally, RepsMate has automated up to 25% of interactions with predefined answers, enhancing both speed and accuracy. Additionally, their strategic use of Microsoft’s full suite of technologies, has allowed RepsMate to train large datasets faster and avoid unnecessary costs, further enhancing efficiency.

Azure marketplace

Discover, try, and deploy cloud software

A faster more efficient future

Examples like those of Akbank, VOCALLS, and RepsMate demonstrate the impact of AI on business speed and productivity. By integrating AI solutions like Microsoft Azure OpenAI Service, companies can achieve faster decision-making, optimize their processes, and better support customer experiences. As businesses continue to adopt and innovate with AI, they’re in a better position to meet the demands of a rapidly evolving market.

Our commitment to responsible AI

Organizations across industries are leveraging Microsoft Azure OpenAI Service and Copilot services and capabilities to drive growth, increase productivity, and create value-added experiences. From advancing medical breakthroughs to streamlining manufacturing operations, our customers trust that their data is protected by robust privacy protections and data governance practices. As our customers continue to expand their use of our AI solutions, they can be confident that their valuable data is safeguarded by industry-leading data governance and privacy practices in the most trusted cloud on the market today.  

At Microsoft, we have a long-standing practice of protecting our customers’ information. Our approach to responsible AI is built on a foundation of privacy, and we remain dedicated to upholding core values of privacy, security, and safety in all our generative AI products and solutions.   

Get started with Azure OpenAI Service 

Apply for access to Azure OpenAI Service by completing this form. 

Learn about Azure OpenAI Service and the latest enhancements. 

Get started with GPT-4 in Azure OpenAI Service in Microsoft Learn. 

Read our partner announcement blog, empowering partners to develop AI-powered apps and experiences with ChatGPT in Azure OpenAI Service. 

Learn how to use the new Chat Completions API (in preview) and model versions for ChatGPT and GPT-4 models in Azure OpenAI Service.

Learn more about Azure AI Content Safety.

Explore possibilities with Azure OpenAI Service

1 Harvard Business Review, Build a Winning AI Strategy For Your Business, July 14, 2023.
The post 10 ways to impact business velocity through Azure OpenAI Service appeared first on Azure Blog.
Quelle: Azure

Build your own copilot with Microsoft Azure AI Studio  

In the rapidly evolving world of AI technology, Microsoft Azure AI Studio, now generally available, is empowering organizations to create their own AI copilots. With AI Studio, organizations can customize and build their own copilot to meet their unique needs.  

We’re excited to see organizations across various sectors staying competitive, driving innovation, and transforming their operations.  No matter your use case, AI Studio accelerates the generative AI development lifecycle, empowering organizations to build and shape the future with AI.  

Build a copilot to streamline call center operations 

Azure ai services

Built cutting-edge applications

Vodafone used AI Studio to modernize their existing customer service chatbot TOBi, and to develop a new copilot with a conversational AI search interface, called SuperAgent, to support Vodafone’s human agents when responding to complex customer inquiries.  

TOBi serves customers by addressing common questions like account status and simple technical troubleshooting. SuperAgent involves summarizing call center transcripts, which condenses lengthy calls into brief summaries stored in the customer relationship management system (CRM). This allows agents to quickly understand the reason for a customer’s previous call and detect new issues, improving response times and customer satisfaction. The system is fully automated, with calls transcribed and summarized by Microsoft Azure OpenAI Service in Azure AI Studio, providing actionable insights for agents. 

Together, Vodafone’s call center is getting impressive results, TOBi manages nearly 45 million customer calls a month, fully resolving 70%. On average, customer call times have been reduced by at least one minute, saving customers and agents’ valuable time.2  

Part of using new technologies is experimentation and the ability to easily collaborate. With Azure AI Studio, you can interact with other people and with projects through a code-first approach to seamlessly explore, build, test, and deploy, using cutting-edge AI tools and machine learning models.” 
—Ahmed Elsayed, CIO UK & Europe Digital Engineering Director, Vodafone Group

Build a copilot to improve customer experiences 

H&R Block built AI Tax Assist, “a generative AI experience that streamlines online tax filing by enabling clients to ask questions during the workflow”, was developed with AI Studio.  

AI Tax Assist addresses individuals’ questions as they prepare and file their taxes and can answer tax theory questions or offer navigation instructions when needed. It can provide answers on tax forms, deductions, and credits to help customers maximize potential refunds and minimize tax liability. AI Tax Assist also answers free-form, tax-related questions, providing dynamic responses to customer questions. 

“Since launching the AI Tax Assist experience, we’ve received positive customer feedback and seen increased usage throughout the 2024 tax season,” says Jody Vanarsdale, Director of Product Management at H&R Block. “We’ve been most pleased to see customers complete the entire filing process quickly and without leaving the app.” 

With Azure AI Studio, our devs can code faster, so they had time to ‘experiment’ to fine-tune features like enabling individuals to ask as many questions as needed conversationally and the ability to revisit previous conversation threads. It’s an approach we’re continuing—to push innovation and deliver the best experiences.” 
—Aditya Thadani, Vice President of Artificial Intelligence Platforms, H&R Block 

Build a copilot to boost employee productivity  

Sweco, a leading European architecture and engineering firm, recognized the need for a custom copilot solution to help employees in the flow of their work. They chose AI Studio to build their own copilot, SwecoGPT, which automates document creation and analysis, delivers advanced search, and provides language translation.  

Azure AI Solutions

Learn more

Shah Muhammad, Head of AI Innovation at Sweco, appreciates the “one-click deployment of the models in Azure AI Studio and that it makes Microsoft Azure AI offerings transparent and available to the user.”1 Since its deployment, nearly half of Sweco’s employees use SwecoGPT and report increased productivity, giving them more time to focus on creativity and helping customers. 

With Azure AI Studio, [we were] able to rapidly develop a proof of concept (POC) to show how a SwecoGPT could look, operate, and benefit our consultants and our business as a whole. This just showcases the power and scalability of Azure AI.” 
 —David Hunter, Head of AI and Automation, Sweco

Microsoft Azure AI Studio

Develop and deploy generative AI responsibly

Learn more

Build on a foundation of trust  

In the journey of AI integration, trust is table stakes. With Azure AI, you can confidently create a copilot knowing your data is always your data. Your data is never used to train the models.  

We owe it to our clients to handle all their information responsibly. Microsoft has shown a lot of leadership in establishing those principles of responsible AI, maintaining clients’ trust, maintaining privacy, and ensuring that any capabilities we deliver are consistent with our promise of expertise.” 
—Aditya Thadani, Vice President of Artificial Intelligence Platforms, H&R Block 

Your organizational data is encrypted in your Microsoft Azure subscription and protected by Microsoft’s comprehensive enterprise compliance and security controls.  

We are a company that uses a lot of Microsoft products, and we trust Microsoft for its security, compliance, and leadership in generative AI.”
—David Hunter, Head of AI and Automation, Sweco 

AI Studio is designed with responsible AI principles and practices in mind. Build your copilot with technologies, templates, and best practices to help manage risk, improve accuracy, protect privacy, reinforce transparency, and simplify compliance. Safeguard your copilot with Azure AI Content Safety’s configurable filters and controls. 

Beyond the technology, the Microsoft commitment to responsible AI was a differentiator,” says Garcia. “As we work with new technologies to build the inclusive digital communities of tomorrow, this is a critical foundation.”
—Ignacio Garcia, CIO Italy & Global Director Data Analytics and AI, Vodafone Group 

Build for the future 

Azure AI is where innovators are creating the future, and we’re continuing to invest in our AI Studio platform to provide cutting edge services and tools to our customers. As we look ahead to what’s next for custom copilots, we’re excited about how agents, like Azure OpenAI Assistants application programming interface (API) can accelerate and improve custom copilot development. Using Azure OpenAI models, developers can provide specific instructions to tune AI capabilities and access multiple tools in parallel, including code interpreter and file search, or custom tools built and accessed through function calling.   

We’re excited to see what customers build next.  

I can’t wait to see what we’re doing in six months or a year from now. The potential of Azure AI Studio for us—and what we can do with it for our customers—is infinite.”
—Shah Muhammad, Head of AI Innovation, Sweco 

Get started with Azure AI Studio  

Explore AI Studio.

Watched the recorded session from Build.

Learn about the Azure OpenAI Assistants API.

Sources

1 Microsoft Customer Story-Sweco Group empowers its architects and engineers with a timesaving AI assistant built in Azure AI Studio 

2 Microsoft Customer Story-Vodafone amplifies call center innovation, customer service, and employee inclusion with Azure AI
The post Build your own copilot with Microsoft Azure AI Studio   appeared first on Azure Blog.
Quelle: Azure

Plans on Microsoft Learn: Your online blueprint for building AI and Azure skills

We get it—you have a mountain of online learning resources available at your fingertips, and with that information overload you can find yourself lost in a maze of unorganized materials. That’s why we’re thrilled to introduce Plans on Microsoft Learn, a new tool that provides guided learning for top Azure tools and solutions.

This innovative approach provides a structured roadmap for individuals or teams to acquire new skills, offering focused content, clear milestones, and support to speed up the learning process. In this blog we’ll introduce you to Plans on Microsoft Learn and explore a handful of curated Plans to get you started!

Reimaging online learning to keep pace with technology

Developing in-demand technical skills is crucial for career advancement, but finding the right starting point can be overwhelming. Plans on Microsoft Learn helps individuals, teams, and organizations reach their learning goals faster, offering curated content, milestones that you can accomplish in the recommended time or at your own pace, and automated reminders to keep you motivated.

Build new skills with expertly designed learning paths, or create your own Plans tailored to your specific needs. Team leaders can easily manage and track their organization’s progress with the ability to copy and administer existing Plans.

Explore an expansive collection of AI-related Plans

We have a growing library of Plans on Microsoft Learn designed to accelerate your learning journey:

Explore Microsoft Learn Career Paths: Unlock curated Plans for 15 in-demand career tracks.

Dive into Microsoft Learn for Organizations: Access eight Plans focused on the hottest tech topics and team training, including building skills for Microsoft’s powerful AI services.

We are developing more Plans all the time, but let’s go in-depth on a few to show you the full value of this new skilling experience.

Make your data AI ready with Microsoft Fabric

Microsoft Fabric is our powerful analytics platform designed to simplify the process of turning data into insights. Unifying data science, real-time analytics, and business intelligence, Fabric offers a seamless end-to-end experience, eliminating the need for businesses to piece together tools from different vendors.

The goal of this Official Plan is to help you use AI to unify your intelligent data and analytics on the Fabric platform, and has learning milestones that include:

Ingest data: Learn various methods for bringing data into Fabric, including shortcuts, pipelines, and dataflows.

Transform data: Transform data using dataflows, procedures, and notebooks.

Store data: Learn how to store processed data in both the Lakehouse and Data Warehouse for efficient retrieval.

Expose data: Discover how to create reusable semantic models in Power BI to make the transformed data accessible for analysis.

Migrate and modernize with Azure cloud-scale databases to enable AI

Migrating and modernizing with Azure helps individuals and organizations seamlessly transition their workloads and applications to the cloud. With step-by-step guidance and tools for migrating on-premises infrastructure and data to Azure, you can update legacy applications and take advantage of Azure’s advanced cloud-native features.

This Official Plan is about migrating and securing Windows Server, SQL Server, and Linux Estate to Azure, with particular focus on:

Cloud-native and Hybrid Solutions: Learn to create and manage data platforms that work effectively in cloud-based and on-premises environments.

SQL Server and Azure SQL Services: Gain hands-on experience with SQL Server, Azure Database for MySQL, and Azure Database for PostgreSQL to build and manage robust database solutions tailored to your application needs.

Azure Arc for Multi-Cloud: Understand how Azure Arc enables SQL Server across different cloud environments.

Performance Optimization: Understand strategies to maximize the speed and efficiency of data platform solutions.

Data for the era of AI: Build intelligent apps with Azure Cosmos DB

The need for intelligent apps is surging as businesses and individuals seek to harness the power of AI and machine learning. These apps offer the ability to deliver personalized experiences, analyze complex data for better decision-making, and streamline workflows for increased efficiency. As AI technology evolves, intelligent apps will become even more indispensable, revolutionizing how we interact with technology in our daily lives.

This Official Plan, focuses on building and modernizing AI-powered intelligent apps, and includes lessons on:

Database selection: Learn to choose the right Azure Cosmos DB option for your AI application.

AI integration: Understand strategies for incorporating AI into your technical or non-technical solutions.

AI Copilot development: Explore how to build an AI-powered assistant using vCore-based Azure Cosmos DB for MongoDB.

Accelerate Developer Productivity with GitHub and Azure for Developers

AI-powered tools offer immense potential to accelerate developer productivity by streamlining coding, automating tasks, and providing intelligent recommendations that save time. By embracing these tools, developers can focus on innovation and strategic problem-solving rather than mundane coding tasks.

With this Official Plan you’ll learn about coding in the cloud with GitHub Copilot, our AI-powered assistant that enables developers to focus on the more creative and strategic aspects of their work:

Coding with GitHub and Azure: Seamlessly integrate these platforms and leverage GitHub Copilot to streamline your development workflow.

DevOps planning: Identify the critical GitHub features to optimize your organization’s enterprise DevOps strategy.

Build communities: Promote collaboration, establish clear guidelines, and uphold security standards within your developer community.

Improve Reliability, Security and Performance on Azure

Building resiliency into your cloud infrastructure is crucial for maximizing efficiency and controlling costs. By carefully aligning resources to match your workload demands, you can eliminate waste and avoid unnecessary expenses for unused capacity.

This Official Plan demonstrates how to optimize your cloud architecture and workloads so your organization can invest in ongoing growth and innovation, and includes teaching on:

Tools and frameworks: Analyze and improve workloads and environments using FinOps Framework, Cloud Adoption Framework, Well Architected Framework, Cloud Center of Excellence, and more.

Landing zones: Leverage best practices from the Cloud Adoption Framework and deploy landing zones to align business and technical strategies for successful cloud adoption.

Resilient workloads: Manage and optimize your environment to maximize business value, cloud security, new workload migration, modernization, and reliability.

Migrate and run VMware in Azure with Azure VMware Solution

Migrating your VMware workloads to the cloud can unlock new levels of flexibility, scalability, and cost optimization. The Azure VMware Solution allows you to seamlessly migrate and run your VMware workloads natively on Azure, leveraging familiar VMware tools and skills.

This official Plan guides you through the process, ensuring a smooth transition and effective management of your VMware environments on Azure, including comprehensive instruction on:

Azure VMware Solution basics: Understand the core concepts, benefits, and architecture of running VMware on Azure.

Migration strategies: Explore various approaches, tools, and best practices to ensure a smooth transition of your VMware workloads to Azure VMware Solution—including both live and cold migrations—to minimize downtime and disruption.

Deployment and configuration: Get detailed guidance on setting up and managing your Azure VMware Solution environment, including networking, storage, and security to ensure optimal performance.

Begin your AI learning journey today

You and your team deserve better tools to get skilled and stay competitive in today’s fast-moving market. Visit our AI learning hub for resources to help your team understand the potential of AI, develop AI skills, and successfully adopt AI solutions. There you’ll also find eight current AI-related Plans and complete guidance on how to get started!

Microsoft learn: Spark possibility
See all you can do with documentation, hands-on training, and certifications to help you get the most from Microsoft products

Start today

The post Plans on Microsoft Learn: Your online blueprint for building AI and Azure skills appeared first on Azure Blog.
Quelle: Azure

8 key VMware questions answered at the Azure VMware Solution digital event

The 2023 changeups in the VMware landscape have created uncertainty and spurred discussions about long-term strategy—your company may be wondering what the future holds for the VMware workloads running in your datacenter. Over the past 25 years, many organizations like yours have built their on-premises IT foundation on VMware—and today the cost of remaining on-premises is higher than ever.

Your competitors are likely shifting their investments to the cloud—and positioning themselves to take advantage of the increasingly mainstream adoption of AI. As you consider pivoting your strategy, it’s crucial to find a way to maximize your existing investments while also choosing a platform that can support your future business needs and applications.

How can you find a path forward for your VMWare workloads that won’t subject you to massive price increases or business disruption? Can you rehost your VMware workloads without compromising on security, control, or budget? Can you use this moment to future-proof your IT platform so you are ready to support the technology changes that will arise over the next 25 years?

You’re not alone in this dilemma. These questions are top of mind for many leaders and practitioners facing similar challenges as they try to navigate the changing landscape of VMware and the cloud. I’m excited to announce that my team will be hosting a free digital event on Tuesday, July 16, 2024 from 9:00 AM–11:00 AM PDT around Azure VMware Solution. I hope you’ll join us to get answers to your questions and learn all the options available for your VMware workloads.

Register now for The Future of VMware Is in Azure to explore strategies and solutions for VMware workloads and learn about Azure VMware Solution—a fully managed VMware environment operated and supported by Microsoft. Azure VMware Solution offers price protection and is designed to help your organization quickly migrate using existing VMware skills while learning new Azure skills.

Azure VMware Solution
Move or extend on-premises VMware environments to Azure without refactoring.

Discover more

Join me on July 16 as we discuss the top-of-mind questions that many VMware customers are asking:

How do we avoid big price increases—is there a way to get price protection? Attend the keynote to hear about limited-time offers to secure predictable pricing and cost savings with Azure VMware Solution. Learn about price protection—with lock-in pricing for up to five years with reserved instances, and why Azure is the most cost-effective destination for the Windows Server and SQL Server workloads that you may be running on VMware.

How can we migrate faster while reducing costs? Learn about tailored programs such as VMware license portability benefits, and how the VMware Rapid Migration Plan can help reduce the cost and time it takes for organizations to migrate to Azure VMware Solution.

How do we find a long-term solution for our VMware workloads? Learn how you can set your organization up for success with Azure VMware Solution—offering the fastest path to the cloud with the cost savings, flexibility, scalability, and performance of Azure.

What are our options to get to the cloud with minimal disruption? Learn how to move VMware workloads to Azure “as is” with minimal need for refactoring. Streamline your migration and equip your practitioners to continue using familiar skills and technologies while adopting new cloud competencies.

How can we give developers access to cutting-edge tools for innovation? Learn how making a strategic shift to Azure VMware Solution eliminates routine maintenance and minimizes administrative tasks, while giving developers proximity to the latest data and AI services—allowing your IT teams to redirect focus toward initiatives that directly contribute to business value.

Where can we see real-world use cases and get practical advice from companies that have successfully migrated? Join Brett Tanzer’s fireside chat with the Deputy CIO of the State of Alaska and get insights from their large-scale cloud migration project. Also, watch a demo deployment and ask product experts your specific questions in the live chat Q&A.

What if we have workloads that need to stay on-premises? Learn about hybrid cloud options for VMware workloads. For workloads that cannot be migrated, learn about solutions that function effectively at the edge, bringing Azure cloud services to your on-premises environment.

How can we empower IT practitioners to make the move to Azure? Attend technical sessions and see demos on networking configurations, business continuity plans, and integration with Azure’s extensive portfolio of over 200 services. Also, hear about the newest learning resources, including the Azure VMware Solution Microsoft Learn Challenge.

The Future of VMware Is in Azure
Join us for talks from industry experts, technical sessions, and a live Qu0026amp;A.

Register now

Here’s a preview of the sessions on July 16:

Keynote address—The Future of VMware Is in Azure

Join Microsoft Vice President of Azure Solutions and Ecosystem, Brett Tanzer, and myself for an overview of all the options for your VMware workloads. Learn migration strategies, hear insights and advice from a Microsoft customer, watch an Azure VMware Solution deployment demo, and ask your questions at the live Q&A. Stay for deep-dive technical sessions with topics tailored for VMware administrators.

Speakers: Brent Tanzer and Liam Sosinsky

Technical sessions—After the keynote, fireside chat, and demo, stay for sessions with detailed technical aspects of migrating VMware workloads, networking configurations, business continuity plans, and integration with an extensive portfolio of over 200 cloud services from Azure.

Securing your future: Migrate applications and data to Azure VMware Solution. Learn the strategic benefits of moving apps and data to Azure VMware Solution. See a migration demo and hear customer evidence and details on implementing a Zero Trust security model and role-based access control (RBAC) within Azure VMware Solution.

Speakers: Scott Gruenemeier and Joe Sarabia

Building end-to-end networking with Azure VMware Solution. Get an understanding of the Azure VMware Solution networking architecture and see a demo of key connectivity patterns from your software-defined data center (SDDC) in Azure VMware Solution to your on-premises environment. This session also covers default NSX-T topology, native security capabilities offered by Azure VMware Solution, integrations with other services, and design best practices for workload security.

Speaker: Kenyon Hensler

Implementing a robust business continuity and disaster recovery plan. In this session, you’ll learn how to implement your business continuity and disaster recovery (BCDR) strategy with Azure VMware Solution. Get tips and best practices for business continuity and disaster recovery and take a deeper dive into common scenarios, such as BCDR strategies when moving to Azure as well as strategies for BCDR within Azure.

Speaker: Melissa Palmer

Unlocking Azure cloud services with Azure VMware solution. See demos of how to use services on the Azure platform to expand the capabilities of your applications on Azure VMware Solution—without changes to your existing app architecture. Learn about your options for Azure platform as a service (PaaS), as well as options to access to AI capabilities and a wide variety of cloud services using Azure VMware Solution.

Speaker: Joe Sarabia

Hybrid cloud options for VMware workloads. Explore hybrid cloud solutions as a complement to the public cloud. This session will show you how to combine Azure Arc, Azure Stack HCI, and Arc-enabled vSphere with Azure VMware Solution to create a seamless, adaptive cloud experience. Learn when and how to implement these innovative technologies to optimize your cloud strategy and stay ahead of the curve.

Speaker: Shriram Natarajan

The Future of VMware Is in Azure

Tuesday, July 16, 2024

9:00 AM–11:00 AM PDT

Register now

The post 8 key VMware questions answered at the Azure VMware Solution digital event appeared first on Azure Blog.
Quelle: Azure

How to Run Hugging Face Models Programmatically Using Ollama and Testcontainers

Hugging Face now hosts more than 700,000 models, with the number continuously rising. It has become the premier repository for AI/ML models, catering to both general and highly specialized needs.

As the adoption of AI/ML models accelerates, more application developers are eager to integrate them into their projects. However, the entry barrier remains high due to the complexity of setup and lack of developer-friendly tools. Imagine if deploying an AI/ML model could be as straightforward as spinning up a database. Intrigued? Keep reading to find out how.

Introduction to Ollama and Testcontainers

Recently, Ollama announced support for running models from Hugging Face. This development is exciting because it brings the rich ecosystem of AI/ML components from Hugging Face to Ollama end users, who are often developers. 

Testcontainers libraries already provide an Ollama module, making it straightforward to spin up a container with Ollama without needing to know the details of how to run Ollama using Docker:

import org.testcontainers.ollama.OllamaContainer;

var ollama = new OllamaContainer("ollama/ollama:0.1.44");
ollama.start();

These lines of code are all that is needed to have Ollama running inside a Docker container effortlessly.

Running models in Ollama

By default, Ollama does not include any models, so you need to download the one you want to use. With Testcontainers, this step is straightforward by leveraging the execInContainer API provided by Testcontainers:

ollama.execInContainer("ollama", "pull", "moondream");

At this point, you have the moondream model ready to be used via the Ollama API. 

Excited to try it out? Hold on for a bit. This model is running in a container, so what happens if the container dies? Will you need to spin up a new container and pull the model again? Ideally not, as these models can be quite large.

Thankfully, Testcontainers makes it easy to handle this scenario, by providing an easy-to-use API to commit a container image programmatically:

public void createImage(String imageName) {
var ollama = new OllamaContainer("ollama/ollama:0.1.44");
ollama.start();
ollama.execInContainer("ollama", "pull", "moondream");
ollama.commitToImage(imageName);
}

This code creates an image from the container with the model included. In subsequent runs, you can create a container from that image, and the model will already be present. Here’s the pattern:

var imageName = "tc-ollama-moondream";
var ollama = new OllamaContainer(DockerImageName.parse(imageName)
.asCompatibleSubstituteFor("ollama/ollama:0.1.44"));
try {
ollama.start();
} catch (ContainerFetchException ex) {
// If image doesn't exist, create it. Subsequent runs will reuse the image.
createImage(imageName);
ollama.start();
}

Now, you have a model ready to be used, and because it is running in Ollama, you can interact with its API:

var image = getImageInBase64("/whale.jpeg");
String response = given()
.baseUri(ollama.getEndpoint())
.header(new Header("Content-Type", "application/json"))
.body(new CompletionRequest("moondream:latest", "Describe the image.", Collections.singletonList(image), false))
.post("/api/generate")
.getBody().as(CompletionResponse.class).response();

System.out.println("Response from LLM " + response);

Using Hugging Face models

The previous example demonstrated using a model already provided by Ollama. However, with the ability to use Hugging Face models in Ollama, your available model options have now expanded by thousands. 

To use a model from Hugging Face in Ollama, you need a GGUF file for the model. Currently, there are 20,647 models available in GGUF format. How cool is that?

The steps to run a Hugging Face model in Ollama are straightforward, but we’ve simplified the process further by scripting it into a custom OllamaHuggingFaceContainer. Note that this custom container is not part of the default library, so you can copy and paste the implementation of OllamaHuggingFaceContainer and customize it to suit your needs.

To run a Hugging Face model, do the following:

public void createImage(String imageName, String repository, String model) {
var model = new OllamaHuggingFaceContainer.HuggingFaceModel(repository, model);
var huggingFaceContainer = new OllamaHuggingFaceContainer(hfModel);
huggingFaceContainer.start();
huggingFaceContainer.commitToImage(imageName);
}

By providing the repository name and the model file as shown, you can run Hugging Face models in Ollama via Testcontainers. 

You can find an example using an embedding model and an example using a chat model on GitHub.

Customize your container

One key strength of using Testcontainers is its flexibility in customizing container setups to fit specific project needs by encapsulating complex setups into manageable containers. 

For example, you can create a custom container tailored to your requirements. Here’s an example of TinyLlama, a specialized container for spinning up the DavidAU/DistiLabelOrca-TinyLLama-1.1B-Q8_0-GGUF model from Hugging Face:

public class TinyLlama extends OllamaContainer {

private final String imageName;

public TinyLlama(String imageName) {
super(DockerImageName.parse(imageName)
.asCompatibleSubstituteFor("ollama/ollama:0.1.44"));
this.imageName = imageName;
}

public void createImage(String imageName) {
var ollama = new OllamaContainer("ollama/ollama:0.1.44");
ollama.start();
try {
ollama.execInContainer("apt-get", "update");
ollama.execInContainer("apt-get", "upgrade", "-y");
ollama.execInContainer("apt-get", "install", "-y", "python3-pip");
ollama.execInContainer("pip", "install", "huggingface-hub");
ollama.execInContainer(
"huggingface-cli",
"download",
"DavidAU/DistiLabelOrca-TinyLLama-1.1B-Q8_0-GGUF",
"distilabelorca-tinyllama-1.1b.Q8_0.gguf",
"–local-dir",
"."
);
ollama.execInContainer(
"sh",
"-c",
String.format("echo '%s' > Modelfile", "FROM distilabelorca-tinyllama-1.1b.Q8_0.gguf")
);
ollama.execInContainer("ollama", "create", "distilabelorca-tinyllama-1.1b.Q8_0.gguf", "-f", "Modelfile");
ollama.execInContainer("rm", "distilabelorca-tinyllama-1.1b.Q8_0.gguf");
ollama.commitToImage(imageName);
} catch (IOException | InterruptedException e) {
throw new ContainerFetchException(e.getMessage());
}
}

public String getModelName() {
return "distilabelorca-tinyllama-1.1b.Q8_0.gguf";
}

@Override
public void start() {
try {
super.start();
} catch (ContainerFetchException ex) {
// If image doesn't exist, create it. Subsequent runs will reuse the image.
createImage(imageName);
super.start();
}
}
}

Once defined, you can easily instantiate and utilize your custom container in your application:

var tinyLlama = new TinyLlama("example");
tinyLlama.start();
String response = given()
.baseUri(tinyLlama.getEndpoint())
.header(new Header("Content-Type", "application/json"))
.body(new CompletionRequest(tinyLlama.getModelName() + ":latest", List.of(new Message("user", "What is the capital of France?")), false))
.post("/api/chat")
.getBody().as(ChatResponse.class).message.content;
System.out.println("Response from LLM " + response);

Note how all the implementation details are under the cover of the TinyLlama class, and the end user doesn’t need to know how to actually install the model into Ollama, what GGUF is, or that to get huggingface-cli you need to pip install huggingface-hub.

Advantages of this approach

Programmatic access: Developers gain seamless programmatic access to the Hugging Face ecosystem.

Reproducible configuration: All configuration, from setup to lifecycle management is codified, ensuring reproducibility across team members and CI environments.

Familiar workflows: By using containers, developers familiar with containerization can easily integrate AI/ML models, making the process more accessible.

Automated setups: Provides a straightforward clone-and-run experience for developers.

This approach leverages the strengths of both Hugging Face and Ollama, supported by the automation and encapsulation provided by the Testcontainers module, making powerful AI tools more accessible and manageable for developers across different ecosystems.

Conclusion

Integrating AI models into applications need not be a daunting task. By leveraging Ollama and Testcontainers, developers can seamlessly incorporate Hugging Face models into their projects with minimal effort. This approach not only simplifies the setup of the development environment process but also ensures reproducibility and ease of use. With the ability to programmatically manage models and containerize them for consistent environments, developers can focus on building innovative solutions without getting bogged down by complex setup procedures.

The combination of Ollama’s support for Hugging Face models and Testcontainers’ robust container management capabilities provides a powerful toolkit for modern AI development. As AI continues to evolve and expand, these tools will play a crucial role in making advanced models accessible and manageable for developers across various fields. So, dive in, experiment with different models, and unlock the potential of AI in your applications today.

Stay current on the latest Docker news. Subscribe to the Docker Newsletter.

Learn more

Visit the Testcontainers website.

Get started with Testcontainers Cloud by creating a free account.

Read LLM Everywhere: Docker for Local and Hugging Face Hosting.

Learn how to Effortlessly Build Machine Learning Apps with Hugging Face’s Docker Spaces.

Get the latest release of Docker Desktop.

Quelle: https://blog.docker.com/feed/