How to run apps in containers (cinit)
Usually it is best practice (and a core design principle of Docker) to run only one application inside a Docker container. After long discussions inside the VIS IT and their decision to break with this principle VSETH IT decided to explicitly break with this principle for maintenance reasons:
- Many developers in VSETH don't have a lot of experience with docker so multiple images per logical application makes their life harder
- To support applications containing multiple images (in potentially multiple repositories) clear standards for the interactions of these repositories would have been needed.
- Such interactions are hard to enforce inside a vibrant organisation like VSETH
- It has a big potential to get messy very fast if people don't stick to the rules
- People often have to understand applications that they are not familiar with. It was an important aspect to have a clear place where to look in order to understand a application.
- VSETH does not need to scale parts of applications independently (they are just idle most of the times anyway)
Given this it was decided that logical associated applications, e.g. the typical setup of a application server and a web server, are put into the same Docker container and even more important: Inside the same git repository.
Important
Inside SIP all programs that are needed to run one logical applications are put into the same git repository and in the same Docker image!
Since this core decision breaks with different assumptions it was needed to put a custom container init system (cinit) in place. This system is free software and can be found in its repo on gitlab.com.
cinit
Init program for UNIX processes. Original development was done
here.
This is the user manual for developers wanting to start their programs in a
cinit enabled container. See Integration for information about
building a cinit enabled container.
Configuration
cinit takes its configuration via YAML files. The minimal
configuration to start a program is:
programs: - name: myprogram path: /the/path
The full list of available options is:
programs: - name: Some descriptive name # The path of the binary to run path: /the/path # Arguments to pass (see below) args: - "-e" - "hello world\\n" # This will be "my_program_arg", see below - "{{ NAME }}_arg" # Set the current working directoy workdir: /some/path # See Program Types type: cronjob: timer: 1 2 3 4 5 # If none is given, root is used uid: 0 gid: 0 user: root group: root # Specify dependencies, see below before: - other-program after: - other-program # Emulate a pseudo-terminal for this program pty: false # Give capabilities to this program capabilities: [] # Pass environment variables env: - PWD: "/home/user" # If no value is given, it is forwarded from cinit - PASSWORD: - NAME: "my_program" # This will be rendered to "my_program_user" - PROGRAM_USER: "{{ NAME }}_user"
If a file is passed via command line it is the only file used. Passing a
directory makes cinit traverse it recursively and taking all found files as
configuration. If no path is given /etc/cinit.yml is used.
Many more examples can be found in the repository’s system-tests/usecases
.
Beware however that some of these examples test error situations. This is
usually indicated by the subdirectory names.
Arguments
Pass arguments to the program to run. You SHOULD only pass one whitespace-
separated word per list item. You SHOULD quote all arguments as they might
contain characters interpreted by the YAML parser.
As with environment variables these strings support templating. You have access
to all environment variables listed in the current program’s env
list.
Program types
Oneshot
A program to be executed once is of type oneshot
. The corresponding
representation in YAML is just type: oneshot
. This is the default if none
is given.
Notify
A program which can notify cinit is of type notify
. This notification works
bysd_notify
which means that any program built with notify-support of systemd will work on
cinit as well. Understood options are as follows:
-
READY=1
: cinit treats the program as started successfully. Programs
depending on it will be started. This is a huge difference tooneshot
programs.notify
programs can still run while dependent programs are started
as well. -
STOPPING=1
: cinit is informed that the program is about to exit. -
STATUS=...
: cinit will log the published status of the child and publish it
on the status socket (see below). -
MAINPID=<PID>
: cinit is informed to treat the passed PID as the main PID of
the program.
Cronjob
A program which is called periodically is of type cronjob
. The YAML
representation of this is nested:
type: cronjob: timer: 1 2 3 4 5
See man cron
for
a description of the time format. A cronjob MUST NOT have dependencies.
Cronjobs are not reentrant. This means if the timer specification wants the job
to be executed at a certain time while simultaneously the job is still running
from a previous run, it won’t be executed twice. Instead it will be rescheduled
to the next time according to the timer specification.
The implementation of cron timer specifications deviates from the man page
linked above by not differentiating between *
and the full range of valid
values given, e.g. 1-31
for the day. This is relevant for interactions
between date specifications and weekday specification.
Special string specifications as @monthly
are not supported.
User / Group
Specify a UNIX user and group under which to run the program. If none is given,
root is used. If an invalid name is given (e.g. because it doesn’t exist), this
is reported by cinit before any program is run. cinit does not create or
otherwise manage users or groups.
Environment
By default the following environment variables will be forwarded from the
cinit process to the programs and are always present:
HOME
LANG
LANGUAGE
LOGNAME
PATH
PWD
SHELL
TERM
USER
Additional parameters may be specified. If no value is given, cinit will
forward the value from its own environment. If the value is not present in
cinit’s environment, no value will be passed (instead of an empty one).
The values of the variables support templating by using the tera
library. Use {{ VAR }}
to
refer to another variable in the environment. Note that VAR
has to be
defined before it can be referenced! For a complete reference and many more
features follow the link to tera.
Capabilities
Processes can be restricted in what they are allowed to do. This can also
mean that non-root process get elevated capabilities. See
man 7 capabilities
for a list of all capabilities.
Dependencies
Programs are allowed to depend on each other via the before
and after
fields. Dependent processes will only be started once all their
dependencies completed. A process is considered completed if
-
it is of type
oneshot
and has terminated successfully -
it is of type
notify
and has either informed cinit that it has started
successfully (see above) or has terminated successfully, whichever
happens first.
Refer to other programs in the config via their name
field.
The direction of a dependency is taken from the view of the program currently
being configured. E.g. the configuration
- name: current before: - other
reads as “current
runs before other
”. Similarly
- name: current after: - other
reads as “current
runs after other
”
If the dependencies form a cycle, this is reported before any process is
started and cinit terminates.
Cronjobs are only allowed to depend on other programs. Depending on a cronjob is
not allowed. A cronjob will only run if its timer schedule demands it and all
dependencies have completed.
Advanced Features
Merging configuration
Each program is identified by a name. Several programs containing the same name
will be merged into a single program configuration. This is especially useful if
cinit is configured to read the configuration from a directory. The fields which
are allowed in more than only one place are listed below. All other fields are
only permitted in one location.
env
All list entries will be merged into a single list. The order of the sublists is
not defined. The order of entries within one list will be preserved. This has
some implications:
-
When specifying duplicate keys, the value is taken from one of the duplicates
but it’s not defined, which one. -
When using a key from one location in the template of a different key, the
result can be either the template without variable substitution or a
successful substitution. It is not guaranteed to be consistent.
before
and after
All list entries will be merged into a single list. The dependencies will be
treated according to this merged list. Duplicates are handled by cinit.
capabilities
All list entries will be merged into a single list. Capabilities will be granted
according to this merged list. Duplicates are handled by cinit.
args
All list entries will be merged into a single list. The list from the location
containing a path
entry is always put first. Apart from that there are no
guarantees on the order of the lists. The order of entries within one sublist
will be preserved. Arguments can use environment variables from any location
specifying environment variables. The same implications about duplicate
environment variables as in env
do apply.
type
Setting a type other than oneshot
is only allowed in the list entry which also
contains the path
(the so called primary list entry). If the primary is set to
a different value than oneshot
it is not possible to change it from a
different list entry.
pty
The logical disjunction of all flags is computed, with false
as the default if
none is given.
Usage
Init program for UNIX processes Usage: cinit [OPTIONS] Options: -c, --config <PATH> The config file or directory to run with [default: /etc/cinit.yml] -v, --verbose... Output information while running -h, --help Print help information -V, --version Print version information
Logging
cinit combines the log output of children with its own. Child output is always
logged at INFO level. The log format is as follows:
<TIMESTAMP> <LEVEL> [<NAME>] <MESSAGE>
-
TIMESTAMP
: This follows the patternYYYY-MM-DD'T'HH:MM:SS.mmm
.
Example:1970-11-23T13:25:44.567
. -
LEVEL
: One ofERROR
,WARN
,INFO
,DEBUG
,TRACE
. -
NAME
: This is either the stringcinit
or the name of a child as defined
in thename
attribute in the YAML config. -
MESSAGE
: The actual event being reported.
Programs SHOULD log to the standard file descriptors and SHOULD NOT log own
timestamps.
Specifying -v
twice gives messages up to the TRACE
level. Tracing messages
are part of the public API. Specifying -v
once gives messages up to
the DEBUG
level. This is the expected level for bug reports. Production
installations should not specify the -v
flag which gives messages up to theINFO
level.
Status Reporting
While running cinit holds an open UNIX Domain Socket at /run/cinit.socket
.
When connecting to the socket cinit reports information about its current
runtime status. The output format is similar to the configuration file format.
A program consists of the following runtime keys:
-
name
: The name of the program as given in the configuration file. -
state
: The current status of the program. One of: -
blocked
: The program is currently waiting for its dependencies to
terminate. sleeping
: The program is a cronjob waiting for the next scheduled
execution.starting
: The program of typenotify
was started and has not informed
cinit about successful startup yet.running
: The program is currently running.stopping
: The program of typenotify
has informed cinit about its
shutdown.done
: The program has terminated successfully.crashed
: The program has terminated with an error code.-
pid
: Optional. The process id of a running child. -
scheduled_at
: Optional. Next execution time of a cronjob. -
exit_code
: Optional. The returned exit code of a child that isdone
orcrashed
.
An example report looks like this:
programs: - name: 'program 1' state: 'done' exit_code: 0 - name: 'program 2' state: 'running' pid: 1409 - name: 'cronjob' state: 'sleeping' scheduled_at: '2019-01-03T17:41:39'
License
Copyright (C) 2019 The cinit developers.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included alongside this document.
Additional Information
See the project description (German) for more details about the decision process and the design principles.