Kategorie: Kubernetes

Safer Docker Hub Pulls via a Sonatype-Protected Proxy

Why a “protected repo”?

Modern teams depend on public container images, yet most environments lack a single, auditable control point for what gets pulled and when. This often leads to three operational challenges:

Inconsistent or improvised base images that drift across teams and pipelines.

Exposure to new CVEs when tags remain unchanged but upstream content does not.

Unreliable workflows due to rate limiting, throttling, or pull interruptions.

A protected repository addresses these challenges by evaluating images at the boundary between public sources and internal systems, ensuring only trusted content is available to the build process. Routing upstream pulls through a Nexus Repository Docker proxy that authenticates to Docker Hub and caches approved layers and creates a security and reliability checkpoint. Repository Firewall inspects image layers and their components against configured policies and enforces the appropriate action, such as allow, quarantine, or block, based on the findings. This provides teams a standard, dependable entry point for base images. Approved content is cached to accelerate subsequent pulls, while malware and high-severity vulnerabilities are blocked before any layer reaches the developer’s environment.Combining this workflow with curated sources such as Docker Official Images or Docker Hardened Images provides a stable, vetted baseline for the entire organization.

Docker Hub authentication (PAT/OAT) quick setup

Before configuring a Nexus Docker proxy, set up authenticated access to Docker Hub. Authentication prevents anonymous-pull rate limits and ensures that shared systems do not rely on personal developer credentials. Docker Hub supports two types of access tokens, and for proxies or CI/CD systems the recommended option is an Organization Access Token (OAT).

Choose the appropriate token type

Personal Access Token (PAT): Use a PAT when authentication is tied to an individual account, such as local development or small teams.

Tied to a single user account

Required for CLI logins when the user enables two-factor authentication

Not recommended for shared infrastructure

Organization Access Token (OAT) (recommended): Use an OAT when authentication is needed for systems that serve multiple users or teams.

Associated with an organization rather than an individual

Suitable for CI/CD systems, build infrastructure, and Nexus Docker proxies

Compatible with SSO and 2FA enforcement

Supports granular permissions and revocation

Requires a Docker Hub Team or Business plan

Create an access token

To create a Personal Access Token (PAT):

Open Docker Hub account settings (clink on your hub avatar in the top right corner).

Select “Personal access tokens”.

Click on “Generate new token”.

Define token Name, Expiration and Access permissions.

Choose “Generate” and save the value immediately, as it cannot be viewed again.

To create an Organization Access Token (OAT):

Sign in to Docker Home and select your organization.

Select Admin Console, then Access tokens.

Select Generate access token.

Expand the Repository drop-down and assign only the required permissions, typically read/pull for proxies or CI systems.

Select Generate token. Copy the token that appears on the screen and save it. You won’t be able to retrieve the token once you exit the screen.

Recommended practices

Scope tokens to the minimum necessary permissions

Rotate tokens periodically

Revoke tokens immediately if they are exposed

Monitor last-used timestamps to confirm expected usage patterns

Step-by-step: create a Docker Hub proxy

The next step after configuring authentication is to make your protected repo operational by turning Nexus into your organization’s Docker Hub proxy. A Docker proxy repository in Nexus Repository provides  a single, policy-enforced registry endpoint that performs upstream pulls on behalf of developers and CI, caches layers locally for faster and more reliable builds, and centralizes access and audit trails so teams can manage credentials and image usage from one place.

To create the proxy:

As an administrator, navigate to the Settings view (gear icon).

Open Repositories and select Create repository.

Choose docker (proxy) as the repository type.

Configure the following settings:

Remote storage: https://registry-1.docker.io

Docker V1 API: Enabled

Index type: Select “Use Docker Hub”

Blob store and network settings as appropriate for your environment

Save the repository to finalize the configuration.

Provide a Clean Pull EndpointTo keep developer workflows simple, expose the proxy at a stable, organization-wide hostname. This avoids custom ports or per-team configurations and makes the proxy a transparent drop-in replacement for direct Docker Hub pulls.Common examples include:

docker-proxy.company.com

hub.company.internal

Use a reverse proxy or ingress controller to route this hostname to the Nexus proxy repository.

Validate Connectivity

Once the proxy is exposed, verify that it responds correctly and can authenticate to Docker Hub.Run:

docker login docker-proxy.company.comdocker pull docker-proxy.company.com/dhi/node:24

A successful pull confirms that the proxy is functioning correctly, upstream connectivity is working, and authenticated access is in place.

Turn on Repository Firewall for containers

Once the Docker proxy is in place, enable Repository Firewall so images are inspected before they reach internal systems. Repository Firewall enforces policy at download time, stopping malware and high-severity vulnerabilities at the registry edge, reducing the blast radius of newly disclosed issues and cutting remediation work for engineering teams.

To enable Firewall for the proxy repository:

As an administrator, navigate to the Settings view (gear icon).

Navigate to Capabilities under the System menu.

Create a ‘Firewall Audit and Quarantine’ capability for your Docker proxy repository.

Configure your policies to quarantine new violating components and protect against introducing risk.

Inform your development teams of the change to set expectations.

Understanding “Quarantine” vs. “Audit”Repository Firewall evaluates each image as it is requested:

Quarantine – Images that violate a policy are blocked and isolated. They do not reach the developer or CI system. The user receives clear feedback indicating the reason for the failure.

Audit – Images that pass the policies are served normally and cached. This improves performance and makes the proxy a consistent, reliable source of trusted base images.

Enabling Repository Firewall gives you immediate, download-time protection and the telemetry to operate it confidently. Start with conservative policies (quarantine on malware, and on CVSS ≥ 8), monitor violations and cache hit rate, tune thresholds based on real-world telemetry, and move to stricter block enforcement once false positives are resolved and teams are comfortable with the workflow.

What a blocked pull looks like

After enabling Repository Firewall and configuring your baseline policies, any pull that fails those checks is denied at the registry edge and no image layers are downloaded. By default Nexus returns a non-descriptive 404 to avoid exposing policy or vulnerability details, though you can surface a short, internal-facing failure message.As an example, If Firewall is enabled and your CVSS threshold policy is configured correctly, the following pull should fail with a 404 message. 

docker pull docker-proxy.company.com/library/node:20

This confirms that:

The request is passing through the proxy.

Repository Firewall is inspecting the image metadata.

Policy violations are blocked before any image layers are downloaded.

In the Firewall UI, you can open the proxy repository and view the recorded violations. The details can include detected CVEs, severity information, and the policy that triggered the denial. This provides administrators with visibility and confirms that enforcement is functioning as expected.

Additionally, the Quarantined Containers dashboard lists every image that Repository Firewall has blocked, showing the triggering policy and severity so teams can triage with full context. Administrators use this view to review evidence, add remediation notes, and release or delete quarantined items; note that malware is quarantined by default while other violations are quarantined only when their rules are set to Fail at the Proxy stage.

Fix forward: choose an approved base and succeed

Once Policy Enforcement is validated, the next step is to pull a base image that complies with your organization’s security rules. This shows what the normal developer experience looks like when using approved and trusted content.

Pull a compliant tag through the proxy:

docker pull docker-proxy.company.com/dhi/node:24

This request passes the Repository Firewall checks, and the image is pulled successfully. The proxy caches each layer locally so that future pulls are faster and no longer affected by upstream rate limits or registry availability.If you repeat the pull, the second request is noticeably quicker because it is served directly from the cache. This illustrates the everyday workflow developers should expect: trusted images, predictable performance, and fewer interruptions.

Get started: protect your Docker pulls

A Sonatype-protected Docker proxy gives developers one policy-compliant registry endpoint for image pulls. Layers are cached for speed, policy violations surface with actionable guidance, and teams work with vetted base images with the same Docker CLI workflows they already rely on. When paired with trusted sources such as Docker Hardened Images, this pattern delivers predictable baselines with minimal developer friction.Ready to try this pattern? Check the following pages:

Sonatype Nexus Repository basic documentation

Integration with Docker Hub

Register for Nexus Repository trial here

Quelle: https://blog.docker.com/feed/

From the Captain’s Chair: Igor Aleksandrov

Docker Captains are leaders from the developer community that are both experts in their field and are passionate about sharing their Docker knowledge with others. “From the Captain’s Chair” is a blog series where we get a closer look at one Captain to learn more about them and their experiences.

Today we are interviewing Igor Aleksandrov. Igor is the CTO and co-founder of JetRockets, a Ruby on Rails development agency based in NYC, bringing over 20 years of software engineering experience and a deep commitment to the Rails ecosystem since 2008. He’s an open-source contributor to projects like the Crystal programming language and Kamal, a regular conference speaker sharing expertise on different topics from container orchestration to migration from React to Hotwire.

Can you share how you first got involved with Docker? What inspired you to become a Docker Captain?

Looking back at my journey to becoming a Docker Captain, it all started with a very practical problem that many Rails teams face: dependency hell. 

By 2018, JetRockets had been building Ruby on Rails applications for years. I’d been working with Rails since version 2.2 back in 2009, and we had established solid development practices. But as our team grew and our projects became more complex, we kept running into the same frustrating issues:

“It works on my machine” became an all-too-common phrase during deployments

Setting up new developer environments was a time-consuming process fraught with version mismatches

Our staging and production environments occasionally behaved differently despite our best efforts

Managing system-level dependencies across different projects was becoming increasingly complex

We needed a unified way to manage application dependencies that would work consistently across development, staging, and production environments.

Unlike many teams that start with Docker locally and gradually move to production, we decided to implement Docker in production and staging first. This might sound risky, but it aligned perfectly with our goal of achieving true environment parity.

We chose our first Rails application to containerize and started writing our first Dockerfile. Those early Dockerfiles were much simpler than the highly optimized ones we create today, but they solved our core problem: every environment now ran the same container with the same dependencies.

Even though AWS Beanstalk has never been a developer friendly solution, the goal was reached – we had achieved true environment consistency, and the mental overhead of managing different configurations across environments had virtually disappeared.

That initial Docker adoption in 2018 sparked a journey that would eventually lead to me becoming a Docker Captain. What began with a simple need for dependency management evolved into deep expertise in container optimization, advanced deployment strategies with tools like Kamal, and ultimately contributing back to the Docker community.

Today, I write extensively about Rails containerization best practices, from image slimming techniques to sophisticated CI/CD pipelines. But it all traces back to that moment in 2019 when we decided to solve our dependency challenges with Docker.

What are some of your personal goals for the next year?

I want to speak at more conferences and meetups, sharing the expertise I’ve built over the years. Living in the Atlanta area, I would like to become more integrated into the local tech community. Atlanta has such a vibrant IT scene, and I think there’s a real opportunity to contribute here. Whether that’s organizing Docker meetups, participating in Rails groups, or just connecting with other CTOs and technical leaders who are facing similar challenges.

If you weren’t working in tech, what would you be doing instead?

If I weren’t working in tech, I think I’d be doing woodworking. There’s something deeply satisfying about creating things with your hands, and woodworking offers that same creative problem-solving that draws me to programming – except you’re working with natural materials and traditional tools instead of code.

I truly enjoy working with my hands and seeing tangible results from my efforts. In many ways, building software and building furniture aren’t that different – you’re taking raw materials, applying craftsmanship and attention to detail, and creating something functional and beautiful.

If not woodworking, I’d probably pursue diving. I’m already a PADI certified rescue diver, and I truly like the ocean. There’s something about the underwater world that’s entirely different from our digital lives – it’s peaceful, challenging, and always surprising. Getting my diving instructor certification and helping others discover that underwater world would be incredibly rewarding.

Can you share a memorable story from collaborating with the Docker community?

One of the most rewarding aspects of being a Docker Captain is our regular Captains meetings, and honestly, I enjoy each one of them. These aren’t just typical corporate meetings – they’re genuine collaborations with some of the most passionate and knowledgeable people in the containerization space.

What makes these meetings special is the diversity of perspectives. You have Captains from completely different backgrounds – some focused on enterprise Kubernetes deployments, others working on AI, developers like me optimizing Rails applications, and people solving problems I’ve never even thought about.

What’s your favorite Docker product or feature right now, and why?

Currently, I’m really excited about the Build Debugging feature that was recently integrated into VS Code. As someone who spends a lot of time optimizing Rails Dockerfiles and writing about containerization best practices, this feature has been a game-changer for my development workflow.

When you’re crafting complex multi-stage builds for Rails applications – especially when you’re trying to optimize image size, manage build caches, and handle dependencies like Node.js and Ruby gems – debugging build failures used to be a real pain.

Can you walk us through a tricky technical challenge you solved recently?

Recently, I was facing a really frustrating development workflow issue that I think many Rails developers can relate to. We had a large database dump file, about 150GB, that we needed to use as a template for local development. The problem was that restoring this SQL dump into PostgreSQL was taking up to an hour every time we needed to reset our development database to a clean state.

For a development team, this was killing our productivity. Every time someone had to test a migration rollback, debug data-specific issues, or just start fresh, they’d have to wait an hour for the database restore. That’s completely unacceptable.

Initially, we were doing what most teams do: running pg_restore against the SQL dump file directly. But with a 150GB database, this involves PostgreSQL parsing the entire dump, executing thousands of INSERT statements, rebuilding indexes, and updating table statistics. It’s inherently slow because the database engine has to do real work.

I realized the bottleneck wasn’t the data itself – it was the database restoration process. So I wrote a Bash script that takes an entirely different approach:

Create a template volume: Start with a fresh Docker volume and spin up a PostgreSQL container

One-time restoration: Restore the SQL dump into this template database (this still takes an hour, but only once)

Volume snapshot: Use a BusyBox container to copy the entire database volume at the filesystem level

Instant resets: When developers need a fresh database, just copy the template volume to a new working volume

The magic is in step 4. Instead of restoring from SQL, we’re essentially copying files at the Docker volume level. This takes seconds instead of an hour because we’re just copying the already-processed PostgreSQL data files.

Docker volumes are just filesystem directories under the hood. PostgreSQL stores its data in a very specific directory structure with data files, indexes, and metadata. By copying the entire volume, we’re getting a perfect snapshot of the database in its “ready to use” state.

The script handles all the orchestration – creating volumes, managing container lifecycles, and ensuring the copied database starts up cleanly. What used to be a one-hour reset cycle is now literally 5-10 seconds. Developers can experiment freely, test destructive operations, and reset their environment without hesitation. It’s transformed how our team approaches database-dependent development.

What’s one Docker tip you wish every developer knew?

If something looks weird in your Dockerfile, you are doing it wrong. This is the single most important lesson I’ve learned from years of optimizing Rails Dockerfiles. I see this constantly when reviewing other developers’ container setups – there’s some convoluted RUN command, a bizarre COPY pattern, or a workaround that just feels off.

Your Dockerfile should read like clean, logical instructions. If you find yourself writing something like:

RUN apt-get update && apt-get install -y wget &&
wget some-random-script.sh && chmod +x some-random-script.sh &&
./some-random-script.sh && rm some-random-script.sh

…you’re probably doing it wrong.

The best Dockerfiles are almost boring in their simplicity and clarity. Every line should have a clear purpose, and the overall flow should make sense to anyone reading it. If you’re adding odd hacks, unusual file permissions, or complex shell gymnastics, step back and ask why.

This principle has saved me countless hours of debugging. Instead of trying to make unusual things work, I’ve learned to redesign the approach. Usually, there’s a cleaner, more standard way to achieve what you’re trying to do.

If you could containerize any non-technical object in real life, what would it be and why?

If I could containerize any non-technical object, it would definitely be knowledge itself. Imagine being able to package up skills, experiences, and expertise into portable containers that you could load and unload from your mind as needed. As someone who’s constantly learning new technologies and teaching others, I’m fascinated by how we acquire and transfer knowledge. Currently, if I want to dive deep into a new programming language like I did with Crystal, or master a deployment tool like Kamal, it takes months of dedicated study and practice.

But what if knowledge worked like Docker containers? You could have a “Ruby 3.3 expertise” container, a “Advanced Kubernetes” container, or even a “Woodworking joinery techniques” container. Need to debug a complex Rails application? Load the container. Working on a diving certification course? Swap in the marine biology knowledge base.

The real power would be in the consistency and portability – just like how Docker containers ensure your application runs the same way everywhere, knowledge containers would give you the same depth of understanding regardless of context. No more forgetting syntax, no more struggling to recall that one debugging technique you learned years ago.

Plus, imagine the collaborative possibilities. Experienced developers could literally package their hard-earned expertise and share it with the community. It would democratize learning in the same way Docker democratized deployment.

Of course, the human experience of learning and growing would be lost, but from a pure efficiency standpoint? That would be incredible.

Where can people find you online? (talks, blog posts, or open source projects, etc)

I am always active in X (@igor_alexandrov) and on LinkedIn. I try to give at least 2-3 talks at tech conferences and meetups each year, and besides this, I have my personal blog.

Rapid Fire Questions

Cats or Dogs?

Dogs

Morning person or night owl?

Both

Favorite comfort food?

Dumplings

One word friends would use to describe you?

Perfectionist

A hobby you picked up recently?

Cycling

Quelle: https://blog.docker.com/feed/

Docker Model Runner now included with the Universal Blue family

Running large language models (LLMs) and other generative AI models can be a complex, frustrating process of managing dependencies, drivers, and environments. At Docker, we believe this should be as simple as docker model run.

That’s why we built Docker Model Runner, and today, we’re thrilled to announce a new collaboration with Universal Blue. Thanks to the fantastic work of these contributors,  Docker Model Runner is now included in OSes such as Aurora and Bluefin, giving developers a powerful, out-of-the-box AI development environment.

What is Docker Model Runner?

For those who haven’t tried it yet, Docker Model Runner is our new “it just works” experience for running generative AI models.

Our goal is to make running a model as simple as running a container.

Here’s what makes it great:

Simple UX: We’ve streamlined the process down to a single, intuitive command: docker model run <model-name>.

Broad GPU Support: While we started with NVIDIA, we’ve recently added Vulkan support. This is a big deal—it means Model Runner works on pretty much any modern GPU, including AMD and Intel, making AI accessible to more developers than ever.

vLLM: Perform high-throughput inference with an NVIDIA GPU

The Perfect Home for Model Runner

If you’re new to it, Universal Blue is a family of next-generation, developer-focused Linux desktops. They provide modern, atomic, and reliable environments that are perfect for “cloud-native” workflows.

As Jorge Castro who leads developer relations at Cloud Native Computing Foundation explains, “Bluefin and Aurora are reference architectures for bootc, which is a CNCF Sandbox Project. They are just two examples showing how the same container pattern used by application containers can also apply to operating systems. Working with AI models is no different – one common set of tools, built around OCI standards.”

The team already ships Docker as a core part of its developer-ready experience. By adding Docker Model Runner to the default installation (specifically in the -dx mode for developers), they’ve created a complete, batteries-included AI development environment.

There’s no setup, no config. If you’re on Bluefin/Aurora, you just open a terminal and start running models.

Get Started Today

If you’re running the latest Bluefin LTS, you’re all set when you turn on developer mode. The Docker engine and Model Runner CLI are already installed and waiting for you. Aurora’s enablement instructions are documented here.

You can run your first model in seconds:

This command will download the model (if not already cached) and run it, ready for you to interact with.

If you’re on another Linux, you can get started just as easily. Just follow the instructions on our GitHub repository.

What’s Next?

This collaboration is a fantastic example of community-driven innovation. We want to give a huge shoutout to the greater bootc enthusiast community for their forward-thinking approach and for integrating Docker Model Runner so quickly.

This is just the beginning. We’re committed to making AI development accessible, powerful, and fun for all developers.

How You Can Get Involved

The strength of Docker Model Runner lies in its community, and there’s always room to grow. We need your help to make this project the best it can be. To get involved, you can:

Star the repository: Show your support and help us gain visibility by starring the Docker Model Runner repo.

Contribute your ideas: Have an idea for a new feature or a bug fix? Create an issue to discuss it. Or fork the repository, make your changes, and submit a pull request. We’re excited to see what ideas you have!

Spread the word: Tell your friends, colleagues, and anyone else who might be interested in running AI models with Docker.

We’re incredibly excited about this new chapter for Docker Model Runner, and we can’t wait to see what we can build together. Let’s get to work!
Quelle: https://blog.docker.com/feed/

Develop and deploy voice AI apps using Docker

Voice is the next frontier of conversational AI. It is the most natural modality for people to chat and interact with another intelligent being. However, the voice AI software stack is complex, with many moving parts. Docker has emerged as one of the most useful tools for AI agent deployment.

In this article, we’ll explore how to use open-source technologies and Docker to create voice AI agents that utilize your custom knowledge base, voice style, actions, fine-tuned AI models, and run on your own computer. It is based on a talk I recently gave at the Docker Captains Summit in Istanbul.

Docker and AI

Most developers consider Docker the “container store” for software. The Docker container provides a reliable and reproducible environment for developing software locally on your own machine and then shipping it to the cloud. It also provides a safe sandbox to isolate, run, and scale user-submitted software in the cloud. For complex AI applications, Docker provides a suite of tools that makes it easy for developers and platform engineers to build and deploy.

The Docker container is a great tool for running software components and functions in an AI agent system. It can run web servers, API servers, workflow orchestrators, LLM actions or tool calls, code interpreters, simulated web browsers, search engines, and vector databases.

With the NVIDIA Container Toolkit, you can access the host machine’s GPU from inside Docker containers, enabling you to run inference applications such as LlamaEdge that serve open-source AI models inside the container.

The Docker Model Runner runs OpenAI-compatible API servers for open-source LLMs locally on your own computer.

The Docker MCP Toolkit provides an easy way to run MCP servers in containers and make them available to AI agents.

The EchoKit platform provides a set of Docker images and utilizes Docker tools to simplify the deployment of complex AI workflows.

EchoKit

The EchoKit consists of a server and a client. The client could be an ESP32-based hardware device that can listen for user voices using a microphone, stream the voice data to the server, receive and play the server’s voice response through a speaker. EchoKit provides the device hardware specifications and firmware under open-source licenses. To see it in action, check out the following video demos.

EchoKit tells the story about the Diana exhibit at the MET museum

EchoKit recommends BBQ in a Texas accent

EchoKit helps a user practice for the US Civics test

You can check out the GitHub repo for EchoKit.

The AI agent orchestrator

The EchoKit server is an open-source AI service orchestrator focused on real-time voice use cases. It starts up a WebSocket server that listens for streaming audio input and returns streaming audio responses. It ties together multiple AI models, including voice activity detection (VAD), automatic speech recognition (ASR), large language models (LLM), and text-to-speech (TTS), using one model’s output as the input for the next model.

You can start an EchoKit server on your local computer and configure the EchoKit device to access it over the local WiFi network. The “edge server” setup reduces network latency, which is crucial for voice AI applications.

The EchoKit team publishes a multi-platform Docker image that you can use directly to start an EchoKit server. The following command starts the EchoKit server with your own config.toml file and runs in the background.

docker run –rm
-p 8080:8080
-v $(pwd)/config.toml:/app/config.toml
secondstate/echokit:latest-server &amp;

The config.toml file is mapped into the container to configure how the EchoKit server utilizes various AI services in its voice response workflow. The following is an example of config.toml. It starts the WebSocket server on port 8080. That’s why in the Docker command, we map the container’s port 8080 to the same port on the host. That allows the EchoKit server to be accessible through the host computer’s IP address. The rest of the config.toml specifies how to access the ASR, LLM, and TTS models to generate a voice response for the input voice data.

addr = "0.0.0.0:8080"
hello_wav = "hello.wav"

[asr]
platform = "openai"
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hellon你好n(noise)n(bgm)n(silence)n"

[llm]
platform = "openai_chat"
url = "https://api.groq.com/openai/v1/chat/completions"
api_key = "gsk_XYZ"
model = "openai/gpt-oss-20b"
history = 20

[tts]
platform = "elevenlabs"
url = "wss://api.elevenlabs.io/v1/text-to-speech/"
token = "sk_xyz"
voice = "VOICE-ID-ABCD"

[[llm.sys_prompts]]
role = "system"
content = """
You are a comedian. Engage in lighthearted and humorous conversation with the user. Tell jokes when appropriate.

"""

The AI services configured for the above EchoKit server are as follows.

It utilizes Groq for ASR (voice-to-text) and LLM tasks. You will need to fill in your own Groq API key.

It utilizes ElevenLabs for streaming TTS (text-to-speech). You will need to fill in your own ElevenLabs API key.

Then, in the EchoKit device setup, you just need to point your device to the local EchoKit server.

ws://local-network-ip.address:8080/ws/

For more options on the EchoKit server configuration, please refer to our documentation!

The VAD server

The voice-to-text ASR is not sufficient by itself. It could hallucinate and generate nonsensical text if the input voice is not human speech (e.g., background noise, street noise, or music). It also would not know when the user has finished speaking, and the EchoKit server needs to ask the LLM to start generating a response.

A VAD model is used to detect human voice and conversation turns in the voice stream. The EchoKit team has a multi-platform Docker image that incorporates the open-source Silero VAD model. The image is much larger than the plain EchoKit server, and it requires more CPU resources to run. But it delivers substantially better voice recognition results. Here is the Docker command to start the EchoKit server with VAD in the background.

docker run –rm
-p 8080:8080
-v $(pwd)/config.toml:/app/config.toml
secondstate/echokit:latest-server-vad &amp;

The config.toml file for this Docker container also needs an additional line in the ASR section, so that the EchoKit server knows to stream incoming audio data to the local VAD service and act on the VAD signals. The Docker container runs the Silero VAD model as a WebSocket service inside the container on port 8000. There is no need to expose the container port 8000 to the host.

addr = "0.0.0.0:8080"
hello_wav = "hello.wav"

[asr]
platform = "openai"
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hellon你好n(noise)n(bgm)n(silence)n"
vad_url = "http://localhost:9093/v1/audio/vad"

[llm]
platform = "openai_chat"
url = "https://api.groq.com/openai/v1/chat/completions"
api_key = "gsk_XYZ"
model = "openai/gpt-oss-20b"
history = 20

[tts]
platform = "elevenlabs"
url = "wss://api.elevenlabs.io/v1/text-to-speech/"
token = "sk_xyz"
voice = "VOICE-ID-ABCD"

[[llm.sys_prompts]]
role = "system"
content = """
You are a comedian. Engage in lighthearted and humorous conversation with the user. Tell jokes when appropriate.

"""

We recommend using the VAD enabled EchoKit server whenever possible.

MCP services

A key feature of AI agents is to perform actions, such as making web-based API calls, on behalf of LLMs. For example, the “US civics test prep” example for EchoKit requires the agent to get exam questions from a database, and then generate responses that guide the user toward the official answer.

The MCP protocol is the industry standard for providing tools (function calls) to LLM agents. For example, the DuckDuckGo MCP server provides a search tool for LLMs to search the internet if the user asks for current information that is not available in the LLM’s pre-training data. The Docker MCP Toolkit provides a set of tools that make it easy to run MCP servers that can be utilized by EchoKit.

The command below starts a Docker MCP gateway server. The MCP protocol defines several ways for agents or LLMs to access MCP tools. Our gateway server is accessible through the streaming HTTP protocol at port 8011.

docker mcp gateway run –port 8011 –transport streaming

Next, you can add the DuckDuckGo MCP server to the gateway. The search tool provided by the DuckDuckGo MCP server is now available on HTTP port 8011.

docker mcp server enable duckduckgo

You can simply configure the EchoKit server to use the DuckDuckGo MCP tools in the config.toml file.

[[llm.mcp_server]]
server = "http://localhost:8011/mcp"
type = "http_streamable"
call_mcp_message = "Please hold on a few seconds while I am searching for an answer!"

Now, when you ask EchoKit a current event question, such as “What is the latest Tesla stock price?”, it will first call the DuckDuckGo MCP’s search tool to retrieve this information and then respond to the user.

The call_mcp_message field is a message the EchoKit device will read aloud when the server calls the MCP tool. It is needed since the MCP tool call could introduce significant latency in the response.

Docker Model Runner

The EchoKit server orchestrates multiple AI services. In the examples in this article so far, the EchoKit server is configured to use cloud-based AI services, such as Groq and ElevenLabs. However, many applications—especially in the voice AI area—require the AI models to run locally or on-premises for security, cost, and performance reasons.

Docker Model Runner is Docker’s solution to run LLMs locally. For example, the following command downloads and starts OpenAI’s open-source gpt-oss-20b model on your computer.

docker model run ai/gpt-oss

The Docker Model Runner starts an OpenAI-compatible API server at port 12434. It could be directly utilized by the EchoKit server via config.toml.

[llm]
platform = "openai_chat"
url = "http://localhost:12434/engines/llama.cpp/v1/chat/completions"
model = "ai/gpt-oss"
history = 20

At the time of this writing, the Docker Model Runner only supports LLMs. The EchoKit server still relies on cloud services, or local AI solutions such as LlamaEdge, for other types of AI services.

Conclusion

The complexity of the AI agent software stack has created new challenges in software deployment and security. Docker is a proven and extremely reliable tool for delivering software to production. Docker images are repeatable and cross-platform deployment packages. The Docker container isolates software execution to eliminate large categories of security issues.

With new AI tools, such as the Docker Model Runner and MCP Toolkit, Docker continues to address emerging challenges in AI portability, discoverability, and security.

The easiest, most reliable, and most secure way to set up your own EchoKit servers is to use Docker.

Learn more

Check out the GitHub repo for EchoKit

Explore the MCP Catalog: Discover containerized, security-hardened MCP servers.

Get started with the MCP Toolkit: Run MCP servers easily and securely.

Visit our Model Runner GitHub repo! Docker Model Runner is open-source, and we welcome collaboration and contributions from the community!

Quelle: https://blog.docker.com/feed/

Private MCP Catalogs and the Path to Composable Enterprise AI

Most discussions about Model Context Protocol infrastructure ask how to govern thousands of AI tools and monitor which MCP servers are running. This question is table stakes but undershoots the possibilities. A better question is how we can unleash MCP to drive developer creativity from a trusted foundation.

The first question produces a phone book of curated, controlled, static resources. The second points toward an AI playground where agents and developers interact and learn from each other. What if private catalogs of MCP servers become composable playlists that encourage mixing, reshaping, and myriad combinations of tool calls? This requires treating MCP catalogs as OCI artifacts, not databases.

Cloud-native computing created feedback loops where infrastructure became code, deployments became declarative, and operational knowledge became shareable artifacts. MCP catalogs need to follow the same path. OCI artifacts, immutable versioning, and container-native workflows provide the model because they represent a well-understood approach that balances trust with creative evolution.

Trust Boundaries That Expand and Learn

iTunes provided a store. Spotify provided a store plus algorithmic discovery, playlist sharing, and taste profiles that improved over time. Private MCP catalogs can enable the same evolution. Today, this means curated, verified collections. Tomorrow, this becomes the foundation for self-improving discovery systems.

Tens of thousands of MCP servers are scattered across GitHub, registries, and forums. Community registries like mcp.so, Smithery, Glama, and PulseMCP are attempting to organize this ecosystem, but provenance remains unclear and quality varies wildly. Private catalogs with tighter access controls offer centralized discovery, enhanced security through vetted servers, and visibility into which tools developers actually use. Organizations can build curated subsets of approved servers, add proprietary internal servers, and selectively import from community registries. This solves the phone book problem. 

When Output Becomes Input

The real opportunity is when the work agents do creates shareable artifacts plus organizational learning automatically. Your agent faces a complex problem analyzing customer churn across three data sources. The MCP gateway then constructs a profile capturing the tools, API keys, sequence of operations, and documentation about what worked. That profile becomes an OCI artifact in your registry.

Next month, another team faces a similar problem. Their agent pulls your profile as a starting point, adapts it, and pushes a refined version. The customer success team creates a churn profile combining data warehouse connectors, visualization tools, and notification servers. The sales team imports that profile, adds CRM connectors, and uses it to strategize on renewals. They publish their enhanced version back to the catalog. Teams stop rebuilding identical solutions and instead reuse or remix. Knowledge is captured, shared, and refined.

Why OCI Makes This Possible

Treating catalogs as immutable OCI artifacts lets agents pin to versions or profiles. Your production agents use catalog v2.3 while QA uses v2.4, and they do not drift. Without this, Agent A mysteriously fails because the database connector it relied on got silently updated with breaking changes. Audit trails become straightforward. You can prove which tools were available when incident X occurred. OCI-based catalogs are the only approach that makes catalogs and agents first-class infrastructure fully addressable with GitOps tooling.

OCI with containers delivers two benefits that matter for MCP. First, containers provide hermetic but customizable and context-rich security boundaries. The MCP server runs in a sandboxed container with explicit network policies, filesystem isolation, and resource limits. Secret injection happens through standard mechanisms with no credentials in prompts. This is key if MCP servers execute arbitrary code or have filesystem access.

Second, containers and the associated OCI versioning appends reusable governance tooling in just the right way, matching other governance tooling in your general container stack and workflow. Because catalogs are OCI artifacts, image scanning works the same. Signing and provenance use Cosign on catalogs just like images. Harbor, Artifactory, and other registries already have sophisticated access controls. Policy enforcement through OPA applies to catalog usage as it does to container deployments. Your FedRAMP-approved container registry handles MCP catalogs too. Your security team does not need to learn new tools.

From Phone Books and iTunes to Intelligent Platforms and Spotify

Organizations can evolve to dynamic discovery within trust boundaries. An MCP gateway allows the agent to query the catalog at runtime, select the appropriate tool, and instantiate only what it needs. With Docker’s Dynamic MCPs in the MCP Gateway, the agent can also call built-in tools like mcp-find and mcp-add to search curated catalogs, pull and start new MCP servers on demand, and drop them when they are no longer needed, instead of hard-coding tool lists and configs. Dynamic MCPs keep unused tools out of the model’s context, reduce token bloat, and let agents assemble just-in-time workflows from a much larger pool of MCP servers. The longer-term vision goes further. The gateway captures semantic intelligence around how users interact with MCPs, learns which tools combine effectively, and suggests relevant servers based on how similar problems were previously solved.

The longer-term vision goes further. The gateway captures semantic intelligence around how users interact with MCPs, learns which tools combine effectively, and suggests relevant servers based on how similar problems were previously solved. Teams both learn from and add to this knowledge feedback loop, Private catalog users discover new MCPs, mix MCPs in useful ways, and develop new ways of doing things,inspired by their own thoughts and by suggestions from the MCP gateway. This process also provides live reinforcement learning, imparting wisdom and context to the system that can benefit and everyone using the gateway.  This is organizational memory as infrastructure, emergent from actual agent work that blends human and machine intelligence in unlimited ways..

The container-native approach using private catalogs, dynamic MCP for runtime discovery, profiles as OCI artifacts, and sandboxed execution builds a composable, secure foundation for this future AI playground. How can we unleash MCP to drive developer creativity from a trusted foundation? Treat it like we treated containers but afford it the privileges that AI deserves as agentic, intelligent systems. Private MCP catalogs endowed with semantic intelligence and context understanding,  built atop OCI versioned infrastructure, running in safe agent sandboxes, are the first step toward that vision.

Quelle: https://blog.docker.com/feed/