Most containerd services follow a client-server/service model. And the service is usually implemented as a grpc service where the underlying storage is a
bolt DB. So the workflow of a containerd request is usually:
user -> containerd client -> service client -> grpc service client -> grp service server -> server -> bolt DB.
On data representation side, instead of exposing metadata directly, containerd only exposes interfaces to users, which defines the set of operations a user is expected to call for an object (e.g. container/image). Then it wraps the metadata object in an internal client-side struct and implements the user-facing interface. This approach has the benefits of (1) abstracting away some implementation details (e.g., inter-service communication); (2) providing better instructions compared to giving all data to user directly.
This usually leads to class/struct changes along a containerd request:
user -> client struct/interface -> metadata struct -> grpc-generated struct -> metadata struct.
Container Service in containerd is responsible for managing and storing container metadata. The workflow from containerd client to the underlying DB involves multiple components and different (but similar)
container structs. This section gives a bottom-up view of the container service and related code path.
metadata.containerStore is the underlying container storage implemented using a
bolt DB(same as other containerd storage). It implements the CRUD operations for containers, which are defined as an interface
Going up, how is a container CRUD operation called (and executed by
metadata.containerStore)? The answer is through Container Service which is a
grpc service defined and auto-generated in this folder.
I’m also new to proto/grpc, so I won’t go into details here. :) I’ll revisit here when I have a better understanding of the grpc services in containerd.
Now that we know how a container is stored (in a
bolt DB) and how a container CRUD operation is delivered to the container storage (via a
grpc service). Next is to define what data we want to store as a container and what CRUD operation we want to support. They are located in the same
containers.Containerstruct defines the metadata for a container to be stored in
metadata.containerStore. It has a proto-equivalent
Messagedefinition used by
containers.Storeinterface defines the CRUD operations for containers (implemented by
metadata.containerStore). It also has a proto-equivalent
Servicedefinition used by
All previous steps and for containerd internals: (container) data definition, data storage, service communication. Next question is how a container CRUD request comes from users and is sent to/processed by
The answer to the 1st question is, same as other containerd services, via containerd client which has (
grpc) clients for all containerd services. As an example, this is the entry to ContainerService. It first creates a
grpc client to the
containerStore, which is wrapped into
containerd.NewRemoteContainerStore implements the same
containers.Store interface and acts as a wrapper abstracting the fact that the data is stored in an underlying
bolt DB (
metadata.containerStore) and accessed through a
grpc service client (
The last question is how to make a container (metadata) object
containers.Container visible to users. Instead of returning the metadata Container struct directly, containerd returns a
containerd.Container interface. Based on my understanding, it has the following benefits:
- Clearly defines the set of container operations/data that containerd wants to expose to users;
- Containerd provides the implementation so that users don’t need to write much process code based on the metadata
- Containerd handles the coordinations between multiple services. For example, users can call
container.Image(ctx)directly to get the container’s
imageinstead of calling ImageService.
Under the hook, containerd implements the interface as a non-export struct
containerd.container which includes a containerd client, the underlying metadata container (
containers.Container), and the container id.
Let’s end with an end-to-end digram showing the above steps for container service in containerd.
Personally I feel image service is more complicated than container service because:
- It has more functionalities other than simply (image) metadata management: image conversion between different formats, image import/export, image (un)compression, etc.
- It handles different specs: OCI, docker (v1.1, v1.2).
I haven’t looked into all features of image service, so this section only focues on the code path for image service itself other than other features related to images.
Image service also has an entry in containerd client,
containerd.Client.ImageService(), where you can get the grpc client wrapper (
containerd.NewImageStoreFromClient) for image service grpc client (
The grpc wrapper handles 2 things: (1) convert user requests to protobuf request messages and send to grpc client; (2) convert grpc responses (from grpc client) back to corresponding structs (e.g.,
image.Image) and return back to the caller.
The image service grpc client, similar to other grpc clients, is auto-generated based on its
grpc/proto definition (
images.proto) and responsible for interacting with the image service backend service (which is basically a
image.Image metadata store).
Similarly, the grpc service binding for image service (bind the grpc client and backend service) happens in
Finally, the image metadata store,
metadata.NewImageStore, is the backend of image service and is also where a grpc request is processed.
Image service has 3 main structs:
containerd.Image: client interface (and implementation) that wraps
imagemetadata and is returned by containerd client.
images.Image: metadata struct that contains image metadata and is stored by image metadata store.
Imageprotobuf message and its auto-generated
Imagestruct: this is mainly used for grpc and should be identical with
Additionally, image service also has the data store interface,
images.Store, that defines its main features (image CRUD). It is implemented both by the grpc client wrapper (to send a request) and by the image metadata store (to receive and process a request).