morbyd

morbyd is a thin layer on top of the standard Docker Go client to easily build and run throw-away test Docker images and containers. And to easily run commands inside these containers.

In particular, morbyd hides the gory details of how to stream the output, and optionally input, of container and commands via Dockers API. You just use your io.Writers and io.Readers, for instance, to reason about the expected output.

This module makes heavy use of option functions. So you can quickly get a grip on Docker's slightly excessive knobs-for-everything API design. morbyd neatly groups the many With...() options in packages, such as run for "run container" and exec for "container execute". This design avoids stuttering option names that would otherwise clash across different API operations for common configuration elements, such as names, labels, and options.

• Session.BuildImage: options are in the build package, such as WithDockerfile, ...
• Session.Run a new container: options are in the run package, such as WithCommand, ...
• Container.Execute a command inside a container: options are in the exec package, such as WithCombinedOutput, ...
• Session.CreateNetwork: with options in the net and bridge, macvlan, ipvlan sub packages.

Features of morbyd

• testable examples for common tasks to get you quickly up and running. Please see the package documentation.

• option function design with extensive Go Doc comments that IDEs show upon option completion. No more pseudo option function "callbacks" that are none the better than passing the original Docker config type verbatim.

  • allows you to add your own (missing?) option functions, as all option-related types are exported.

• uses the official Docker Go client in order to benefit from its security fixes, functional upgrades, and all the other nice things to get directly from upstream.

• “auto-cleaning” that runs when creating a new test session and again at its end, removing all containers and networks especially tagged using session.WithAutoCleaning for the test.

• uses context.Context throughout the whole module, especially integrating well with testing frameworks (such as Ginkgo) that support automatic unit test context creation.

• extensive unit tests with large coverage. We even mock the Docker client in order to cover the "unhappy paths", also known as "error handling". In addition, we run go routine leak checks, courtesy of Gomega gleak.

Trivia

The module name morbyd is an amalgation of "Moby (Dock)" and morbid – ephemeral – test containers.

Usage

Run Container and Pick Up Its Output

package main

import (
    "context"

    "github.com/thediveo/morbyd"
    "github.com/thediveo/morbyd/exec"
    "github.com/thediveo/morbyd/run"
    "github.com/thediveo/morbyd/session"
)

func main() {
    ctx := context.TODO()
    // note: error handling left out for brevity
    //
    // note: enable auto-cleaning of left-over containers and
    // networks, both when creating the session as well as when
    // closing the session. Use a unique label either in form of
    // "key=" or "key=value".
    sess, _ := morbyd.NewSession(ctx, session.WithAutoCleaning("test.mytest="))
    defer sess.Close(ctx)

    // run a container and copy the container's combined output
    // of stdout and stderr to our stdout.
    cntr, _ := sess.Run(ctx, "busybox",
        run.WithCommand("/bin/sh", "-c", "while true; do sleep 1; done"),
        run.WithAutoRemove(),
        run.WithCombinedOutput(os.Stdout))

    // run a command inside the container and wait for this command
    // to finish.
    cmd, _ := cntr.Exec(ctx,
        exec.WithCommand("/bin/sh", "-c", "echo \"Hellorld!\""),
        exec.WithCombinedOutput(os.Stdout))
    exitcode, _ := cmd.Wait(ctx)
}

Deploy Container and Contact Its Published Service

package main

import (
    "context"

    "github.com/thediveo/morbyd"
    "github.com/thediveo/morbyd/exec"
    "github.com/thediveo/morbyd/run"
    "github.com/thediveo/morbyd/session"
)

func main() {
    ctx := context.TODO()
    // note: error handling left out for brevity
    //
    // note: enable auto-cleaning of left-over containers and
    // networks, both when creating the session as well as when
    // closing the session. Use a unique label either in form of
    // "key=" or "key=value".
    sess, _ := morbyd.NewSession(ctx, session.WithAutoCleaning("test.mytest="))
    defer sess.Close(ctx)

    cntr, _ := sess.Run(ctx, "busybox",
        run.WithCommand("/bin/sh", "-c", `echo "DOH!" > index.html && httpd -f -p 1234`),
        run.WithAutoRemove(),
        run.WithPublishedPort("127.0.0.1:1234"))

    svcAddrPort := container.PublishedPort("1234").Any().UnspecifiedAsLoopback().String()
    req, _ := http.NewRequest(http.MethodGet, "http://"+svcAddrPort+"/", nil)
    resp, _ := http.DefaultClient.Do(req.WithContext(ctx))
    defer resp.Body.Close()
    body, _ := io.ReadAll(resp.Body)
    fmt.Sprintf("%s\n", body)
}

Dealing with Container Output

safe.Buffer is the concurrency-safe drop-in sibling to Go's bytes.Buffer: it is essential in unit tests that reason about container output without setting off Go's race detector. The reason is that container output is handled on background Go routines and simultaneously polling an unguarded bytes.Buffer causes a race. All you need to do is replace bytes.Buffer with safe.Buffer (which is just a thin mutex'ed wrapper), for instance, in this test leveraging Gomega:

var buff safe.Buffer

// run a container that outputs a magic phrase and then
// keeps sleeping until the container gets terminated.
Expect(cntr.Exec(ctx,
  exec.Command("/bin/sh", "-c", "echo \"**FOO!**\" 1>&2; while true; do sleep 1; done"),
  exec.WithTTY(),
  exec.WithCombinedOutput(
    io.MultiWriter(
      &buff, timestamper.New(GinkgoWriter))))).To(Succeed())

// ensure we got the magic phrase
Eventually(buff.String).Should(Equal("**FOO!**\r\n"))

timestamper.New returns a writer object implementing io.Writer that time stamps each line of output. It has proven useful in debugging tests involving container output.

Alternatives

Why morbyd when there are already other much bigger and long-time battle-proven tools for using Docker images and containers in Go tests?

• for years, @ory/dockertest has served me well. Yet I eventually hit its limitations hard: for instance, dockertest cannot handle Docker's 100 CONTINUE API protocol upgrades, because of its own proprietary Docker client implementation. However, this functionality is essential in streaming container and command output and input – and thus only allowing diagnosing tests. Such issues are unresponded and unfixed. In addition, having basically to pass functions for configuration of Docker data structures is repeating the job of option functions at each and every dockertest call site.
• Testcontainers for Go as a much larger solution with a steep learning curve as well as some automatically installing infrastructure – while I admire this design, it is difficult to understand what exactly is happening. Better keep it simple.

Supported Go Versions

morbyd supports versions of Go that are noted by the Go release policy, that is, major versions N and N-1 (where N is the current major version).

Contributing

Please see CONTRIBUTING.md.

Copyright and License

morbyd is Copyright 2024 Harald Albrecht, and licensed under the Apache License, Version 2.0.

The package github.com/thediveo/morbyd/run/dockercli is Copyright 2013-2017 Docker, Inc. and licensed under the Apache License Version 2.0, with the elements listed below coming from the github.com/docker/cli module in order to work around import dependency versioning problems due to @docker/cli using a managed vendor/ directory, but not providing a go.mod and the associated guarantees:

• opts/mount.go; removed the logrus dependency.
• opts/network.go
• a subset of cli/compose/types/types.go: type ServiceVolumeConfig, with direct dependency types ServiceVolumeBind, ServiceVolumeVolume, ServiceVolumeTmpfs, and ServiceVolumeCluster.
• cli/compose/loader/volume.go