Monat: Oktober 2025

Unlocking Local AI on Any GPU: Docker Model Runner Now with Vulkan Support

Running large language models (LLMs) on your local machine is one of the most exciting frontiers in AI development. At Docker, our goal is to make this process as simple and accessible as possible. That’s why we built Docker Model Runner, a tool to help you download and run LLMs with a single command.

Until now, GPU-accelerated inferencing with Model Runner was limited to CPU, NVIDIA GPUs (via CUDA), and Apple Silicon (via Metal). Today, we’re thrilled to announce a major step forward in democratizing local AI: Docker Model Runner now supports Vulkan!

This means you can now leverage hardware acceleration for LLM inferencing on a much wider range of GPUs, including integrated GPUs and those from AMD, Intel, and other vendors that support the Vulkan API.

Why Vulkan Matters: AI for Everyone’s GPU

So, what’s the big deal about Vulkan?

Vulkan is a modern, cross-platform graphics and compute API. Unlike CUDA, which is specific to NVIDIA GPUs, or Metal, which is for Apple hardware, Vulkan is an open standard that works across a huge range of graphics cards. This means if you have a modern GPU from AMD, Intel, or even an integrated GPU on your laptop, you can now get a massive performance boost for your local AI workloads.

By integrating Vulkan (thanks to our underlying llama.cpp engine), we’re unlocking GPU-accelerated inferencing for a much broader community of developers and enthusiasts. More hardware, more speed, more fun!

Getting Started: It Just Works

The best part? You don’t need to do anything special to enable it. We believe in convention over configuration. Docker Model Runner automatically detects compatible Vulkan hardware and uses it for inferencing. If a Vulkan-compatible GPU isn’t found, it seamlessly falls back to CPU.

Ready to give it a try? Just run the following command in your terminal:

docker model run ai/gemma3

This command will:Pull the Gemma 3 model.Detect if you have a Vulkan-compatible GPU with the necessary drivers installed.Run the model, using your GPU to accelerate the process.It’s that simple. You can now chat with a powerful LLM running directly on your own machine, faster than ever.

Join Us and Help Shape the Future of Local AI!

Docker Model Runner is an open-source project, and we’re building it in the open with our community. Your contributions are vital as we expand hardware support and add new features.Head over to our GitHub repository to get involved:https://github.com/docker/model-runnerPlease star the repo to show your support, fork it to experiment, and consider contributing back with your own improvements.

Learn more

Check out the Docker Model Runner General Availability announcement

Visit our Model Runner GitHub repo! Docker Model Runner is open-source, and we welcome collaboration and contributions from the community!

Get started with Model Runner with a simple hello GenAI application

Quelle: https://blog.docker.com/feed/

From the Captain’s Chair: Pradumna Saraf

Docker Captains are leaders from the developer community that are both experts in their field and are passionate about sharing their Docker knowledge with others. “From the Captain’s Chair” is a blog series where we get a closer look at one Captain to learn more about them and their experiences. 

Today, we are interviewing Pradumna Saraf. He is an Open Source Developer with a passion for DevOps. He is also a Golang developer and loves educating people through social media and blogs about various DevOps tools like Docker, GitHub Actions, Kubernetes, etc. He has been a Docker Captain since 2024.

Can you share how you first got involved with Docker?

If I remember correctly, I was learning about databases, more specifically, MongoDB. Until that time, I had no idea there was something called Docker. I was trying to find a way to get the database up and running locally, and then I came to know from a YouTube video about how Docker is the most common and efficient way for running these kinds of applications locally, and then I skipped learning about databases and dived deep into learning Docker.

What inspired you to become a Docker Captain?

The community. Docker has always been working towards making the developer life easier and listening to the community and users, whether it’s an open source offering or an enterprise, and I wanted to be part of this community. Before even joining the Captains program, I was advocating for Docker by sharing my learning via social media, blogs, etc, and educating people because I was passionate and really loved the potential of Docker. Becoming a Captain felt natural, as I was already doing the stuff, so it was great to get the recognition.

What are some of your personal goals for the next year?

Writing more technical content, of course! Also, giving more in-person talks at international conferences. I also want to get back to contributing and helping open source projects grow.

If you weren’t working in tech, what would you be doing instead?

That’s an interesting question. I love tech. It’s hard to imagine my life without tech because getting into it was not a decision; it was a passion that was inside of me before I could spell technology. But still, if I were not in tech, I might be a Badminton or a Golf player.

Can you share a memorable story from collaborating with the Docker community?

Yes, there was a meetup in Docker Bangalore, India, where Ajeet (DevRel at Docker), a good friend of mine, and I collaborated, and he invited me to deliver a talk on Docker extensions. It was really nice meeting the community, having conversations over pizza about how various people and companies are using Docker in their workflow and bottlenecks.

What’s your favorite Docker product or feature right now, and why?

I am really biased towards Docker Compose. My favourite feature right now is being able to define models in a Docker Compose YAML file and start/stop an AI model with the same Docker Compose commands. Apart from that, I really like the standalone Docker Model Runner (DMR).

Can you walk us through a tricky technical challenge you solved recently?

I was working on an authorization project, where I was verifying users with the right set of permissions and letting them access the resource, and interestingly, Docker had a key role in that project. The role of Docker was a Policy Decision Point (PDP), which was running inside a container and listening to external requests, and was responsible for validating if the entity/user/request is authorized to access the particular resource with the right permissions. This was a particularly unique application of Docker, where I used it as a decision point. Docker made it easy to run, keeping it separate from the main app and making it scalable with almost zero downtime. It showed Docker can also be used for important services like authorization.

What’s one Docker tip you wish every developer knew?

Using multi-stage builds. It helps keep your images small, clean, secure, and production-ready. It’s such a simple thing, but it can make a huge difference. I have seen an image go from 1.7 GB to under 100 MB. Bonus: It will also make your pull and push faster, saving CI cost and making your overall deployment faster.

If you could containerize any non-technical object in real life, what would it be and why?

My age. I’d containerize age so I could choose how old I want to be. If I want to feel young, I will run Docker with an image with the age version of 20, and if I want to think more mature, I will run Docker with an image with the age version of 40.

Where can people find you online? (talks, blog posts, or open source projects, etc.)

People can find social media platforms like Twitter (X), LinkedIn, BlueSky, Threads, etc. For my open source work, people can find me on GitHub. I have many Docker-related projects. Apart from that, if people are more into blogs and conferences, they can find me on my blog and sessionize profile. Or just Google “Pradumna Saraf”.

Rapid Fire Questions

Cats or Dogs?

Cats

Morning person or night owl?

Night Owl

Favorite comfort food?

Dosa

One word friends would use to describe you?

Helpful

A hobby you picked up recently?

Learning more about aircraft and the aviation industry.
Quelle: https://blog.docker.com/feed/