Cloud Computing Köln

Leveraging Testcontainers for Complex Integration Testing in Mattermost Plugins

This post was contributed by Jesús Espino, Principal Engineer at Mattermost.

In the ever-evolving software development landscape, ensuring robust and reliable plugin integration is no small feat. For Mattermost, relying solely on mocks for plugin testing became a limitation, leading to brittle tests and overlooked integration issues. Enter Testcontainers, an open source tool that provides isolated Docker environments, making complex integration testing not only feasible but efficient. 

In this blog post, we dive into how Mattermost has embraced Testcontainers to overhaul its testing strategy, achieving greater automation, improved accuracy, and seamless plugin integration with minimal overhead.

The previous approach

In the past, Mattermost relied heavily on mocks to test plugins. While this approach had its merits, it also had significant drawbacks. The tests were brittle, meaning they would often break when changes were made to the codebase. This made the tests challenging to develop and maintain, as developers had to constantly update the mocks to reflect the changes in the code.

Furthermore, the use of mocks meant that the integration aspect of testing was largely overlooked. The tests did not account for how the different components of the system interacted with each other, which could lead to unforeseen issues in the production environment. 

The previous approach additionally did not allow for proper integration testing in an automated way. The lack of automation made the testing process time-consuming and prone to human error. These challenges necessitated a shift in Mattermost’s testing strategy, leading to the adoption of Testcontainers for complex integration testing.

Mattermost’s approach to integration testing

Testcontainers for Go

Mattermost uses Testcontainers for Go to create an isolated testing environment for our plugins. This testing environment includes the Mattermost server, the PostgreSQL server, and, in certain cases, an API mock server. The plugin is then installed on the Mattermost server, and through regular API calls or end-to-end testing frameworks like Playwright, we perform the required testing.

We have created a specialized Testcontainers module for the Mattermost server. This module uses PostgreSQL as a dependency, ensuring that the testing environment closely mirrors the production environment. Our module allows the developer to install and configure any plugin you want in the Mattermost server easily.

To improve the system’s isolation, the Mattermost module includes a container for the server and a container for the PostgreSQL database, which are connected through an internal Docker network.

Additionally, the Mattermost module exposes utility functionality that allows direct access to the database, to the Mattermost API through the Go client, and some utility functions that enable admins to create users, channels, teams, and change the configuration, among other things. This functionality is invaluable for performing complex operations during testing, including API calls, users/teams/channel creation, configuration changes, or even SQL query execution. 

This approach provides a powerful set of tools with which to set up our tests and prepare everything for verifying the behavior that we expect. Combined with the disposable nature of the test container instances, this makes the system easy to understand while remaining isolated.

This comprehensive approach to testing ensures that all aspects of the Mattermost server and its plugins are thoroughly tested, thereby increasing their reliability and functionality. But, let’s see a code example of the usage.

We can start setting up our Mattermost environment with a plugin like this:

pluginConfig := map[string]any{}
options := []mmcontainer.MattermostCustomizeRequestOption{
mmcontainer.WithPlugin("sample.tar.gz", "sample", pluginConfig),
}
mattermost, err := mmcontainer.RunContainer(context.Background(), options…)
defer mattermost.Terminate(context.Background()

Once your Mattermost instance is initialized, you can create a test like this:

func TestSample(t *testing.T) {
client, err mattermost.GetClient()
require.NoError(t, err)
reqURL := client.URL + "/plugins/sample/sample-endpoint"
resp, err := client.DoAPIRequest(context.Background(), http.MethodGet, reqURL, "", "")
require.NoError(t, err, "cannot fetch url %s", reqURL)
defer resp.Body.Close()
bodyBytes, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.Equal(t, 200, resp.StatusCode)
assert.Contains(t, string(bodyBytes), "sample-response")
}

Here, you can decide when you tear down your Mattermost instance and recreate it. Once per test? Once per a set of tests? It is up to you and depends strictly on your needs and the nature of your tests.

Testcontainers for Node.js

In addition to using Testcontainers for Go, Mattermost leverages Testcontainers for Node.js to set up our testing environment. In case you’re unfamiliar, Testcontainers for Node.js is a Node.js library that provides similar functionality to Testcontainers for Go. Using Testcontainers for Node.js, we can set up our environment in the same way we did with Testcontainers for Go. This allows us to write Playwright tests using JavaScript and run them in the isolated Mattermost environment created by Testcontainers, enabling us to perform integration testing that interacts directly with the plugin user interface. The code is available on GitHub.  

This approach provides the same advantages as Testcontainers for Go, and it allows us to use a more interface-based testing tool — like Playwright in this case. Let me show a bit of code with the Node.js and Playwright implementation:

We start and stop the containers for each test:

test.beforeAll(async () => { mattermost = await RunContainer() })
test.afterAll(async () => { await mattermost.stop(); })

Then we can use our Mattermost instance like any other server running to run our Playwright tests:

test.describe('sample slash command', () => {
test('try to run a sample slash command', async ({ page }) => {
const url = mattermost.url()
await login(page, url, "regularuser", "regularuser")
await expect(page.getByLabel('town square public channel')).toBeVisible();
await page.getByTestId('post_textbox').fill("/sample run")
await page.getByTestId('SendMessageButton').click();
await expect(page.getByText('Sample command result', { exact: true })).toBeVisible();
await logout(page)
});
});

With these two approaches, we can create integration tests covering the API and the interface without having to mock or use any other synthetic environment. Also, we can test things in absolute isolation because we consciously decide whether we want to reuse the Testcontainers instances. We can also reach a high degree of isolation and thereby avoid the flakiness induced by contaminated environments when doing end-to-end testing.

Examples of usage

Currently, we are using this approach for two plugins.

1. Mattermost AI Copilot

This integration helps users in their daily tasks using AI large language models (LLMs), providing things like thread and meeting summarization and context-based interrogation.

This plugin has a rich interface, so we used the Testcontainers for Node and Playwright approach to ensure we could properly test the system through the interface. Also, this plugin needs to call the AI LLM through an API. To avoid that resource-heavy task, we use an API mock, another container that simulates any API.

This approach gives us confidence in the server-side code but in the interface side as well, because we can ensure that we aren’t breaking anything during the development.

2. Mattermost MS Teams plugin

This integration is designed to connect MS Teams and Mattermost in a seamless way, synchronizing messages between both platforms.

For this plugin, we mainly need to do API calls, so we used Testcontainers for Go and directly hit the API using a client written in Go. In this case, again, our plugin depends on a third-party service: the Microsoft Graph API from Microsoft. For that, we also use an API mock, enabling us to test the whole plugin without depending on the third-party service.

We still have some integration tests with the real Teams API using the same Testcontainers infrastructure to ensure that we are properly handling the Microsoft Graph calls.

Benefits of using Testcontainers libraries

Using Testcontainers for integration testing offers benefits, such as:

Isolation: Each test runs in its own Docker container, which means that tests are completely isolated from each other. This approach prevents tests from interfering with one another and ensures that each test starts with a clean slate.

Repeatability: Because the testing environment is set up automatically, the tests are highly repeatable. This means that developers can run the tests multiple times and get the same results, which increases the reliability of the tests.

Ease of use: Testcontainers is easy to use, as it handles all the complexities of setting up and tearing down Docker containers. This allows developers to focus on writing tests rather than managing the testing environment.

Testing made easy with Testcontainers

Mattermost’s use of Testcontainers libraries for complex integration testing in their plugins is a testament to the power and versatility of Testcontainers.

By creating a well-isolated and repeatable testing environment, Mattermost ensures that our plugins are thoroughly tested and highly reliable.

Learn more

Subscribe to the Docker Newsletter. 

Visit the Testcontainers website.

Get started with Testcontainers Cloud by creating a free account.

Vote on what’s next! Check out our public roadmap.

Have questions? The Docker community is here to help.

New to Docker? Get started.

Quelle: https://blog.docker.com/feed/

A New Era at Docker: How We’re Investing in Innovation and Customer Relationships

I recently joined Docker in January as Chief Revenue Officer. My role is responsible for the entire customer journey, from your first interaction with Docker’s sales org to post-sales support and onboarding. As I speak with customers and hear stories about their journey with Docker over the past decade, I’m often reminded of the immense trust you’ve placed in us. Whether you’ve been with us from the days of Docker Swarm or have more recently started using Docker Desktop, your partnership has been invaluable in shaping who we are today. 

I want to take a moment to personally thank you for being part of our story, especially as we continue to evolve in a rapidly changing ecosystem.

We know that change can bring challenges. Over the years, as containers became the backbone of modern software development, Docker has evolved alongside them. This evolution has not always been easy and I understand that shifts in our product offerings, changes in pricing, and recent adjustments to our subscription plans have impacted many of you. Our priority now, as it always has been, is to deliver unrivaled value to you. 

We recognize that to continue innovating and addressing the complex needs of modern developers, we must continue to invest in Docker products and our relationships with customers like you. This investment isn’t just about tools and features; it’s about creating a holistic ecosystem — a unified suite — that makes your development process more productive, secure, and manageable at an enterprise scale, while building a go-to-market organization that is equipped to support our growing customer base. 

To that end, we’ve redefined our strategy to focus on a deeper, more meaningful engagement with you. We’re committed to building stronger relationships, listening carefully to your feedback, and ensuring that the solutions we bring to market truly address your pain points. By focusing on your needs, we’re working to make every interaction with Docker more valuable, whether it’s through enhanced support, new features, or better licensing management. If you’d like to discuss this with me further, I’m happy to schedule time. (Reach out by email or connect with your Account Executive to set this up.)

Additionally, we’ve made key investments in our enterprise suite of products that surrounds Docker Desktop. We understand that the demands of modern development extend beyond the individual developer’s experience. Docker is the only container-first platform built specifically for development teams, improving developer experience and productivity while meeting the security and control needs of modern enterprises. Docker offers a comprehensive suite of enterprise-ready tools, cloud services, trusted content, and a collaborative community that helps streamline workflows and maximize development efficiency.

As we continue to invest in both vectors above, we’re excited about what lies ahead in our product roadmap. Our aim is simple: to help your teams develop with confidence, knowing that Docker is a trusted partner invested in your success. I am personally dedicated to ensuring that our roadmap reflects your needs and that our solutions empower your teams to reach their full potential.

Thank you again for your continued trust and partnership. We wouldn’t be here without you, and I look forward to what we will achieve together.

Learn more

Read Announcing Upgraded Docker Plans: Simpler, More Value, Better Development and Productivity.

Subscribe to the Docker Newsletter. 

Quelle: https://blog.docker.com/feed/