Getting started with Raptor

What is Raptor?

Raptor¹ is a modern, fast, and easy-to-use system for building disk images, bootable isos, containers and much more - all from a simple, Dockerfile-inspired syntax.

It uses systemd-nspawn for sandboxing when building or running containers.

Theory of operation

Raptor builds layers from .rapt files. If you are familiar with Docker, this is similar to how Docker builds containers from a Dockerfile.

However, Raptor has a different scope, and can do considerably different things than Docker.

Syntax

Raptor uses a syntax similar to Dockerfile. Statements start with uppercase keywords, and are terminated by end of line.

All lines starting with # are treated as comments:

# This copies "foo" from the host to "/bar" inside the build target
COPY foo /bar

Raptor files

Before being parsed as raptor files, .rapt files are processed through minijinja, a powerful templating language.

1. According to wikipedia: “The word “raptor” refers to several groups of avian and non-avian dinosaurs which primarily capture and subdue/kill prey with their talons.“. Hopefully, this Raptor is less scary. ↩

Install guide

Currently, the only supported and recommended way to install raptor is by compiling from source. Pre-built binary packages might be available in the future.

New user?

1. Complete Install prerequisites
2. Complete Install Raptor from git
3. Enjoy Raptor 🦅

Install prerequisites

1. Install necessary packages:

   sudo apt-get update
   sudo apt-get install -y git musl-tools
   

2. Install Rust 1.90 or greater:

   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
   

3. Install musl target for rust:

   rustup target add x86_64-unknown-linux-musl
   

Now you are ready to install Raptor.

Install Raptor from git [stable]

1. Install prerequisites

2. Install raptor and falcon from git:

   cargo install \
       --target "x86_64-unknown-linux-musl" \
       --git "https://github.com/chrivers/raptor" \
       raptor falcon
   

Install Raptor from git [development branch]

This procedure is intended for developers, beta testers, and anyone else who want to try a specific branch build of Raptor. To test a branch named hypothetical:

1. Install prerequisites

2. Install raptor and falcon from git:

   cargo install \
       --target "x86_64-unknown-linux-musl" \
       --git "https://github.com/chrivers/raptor" \
       --branch "hypothetical" \
       raptor falcon
   

Making a Debian liveboot iso with Raptor

In this walkthrough, we will take a look at all the steps needed to make a customized Debian liveboot .iso image.

By putting the image on a USB stick or external hard drive, any computer can be booted from this external device, without affecting existing data on the computer.

Using this technique, you will be able to create custom images, containing the tools, programs and settings that you like, and build them consistently and easily with raptor.

This guide is presented as a three-step process:

• Build: Build raptor layers suitable for live booting.

• Generate: Generate a live boot iso from the layers

• Make: Use raptor-make to simplify the build process [optional]

Create a raptor file:

# Start from a docker iso
FROM docker://debian:trixie

# Set root password to "raptor"
RUN usermod -p "$1$GQf2tS9s$vu72NbrDtUcvvqnyAogrH0" root

# Update package sources, and install packages
RUN apt-get update
RUN apt-get install -qy systemd-sysv live-boot linux-image-amd64

Git clone (or otherwise download) the necessary build container recipes:

git clone https://github.com/chrivers/raptor-builders.git

Then use these to build liveboot.iso:

sudo raptor run -L rbuild raptor-builders '$rbuild.deblive' -I base -O liveboot.iso

After this command has run, you should have the file liveboot.iso available. Enjoy!

To learn how (and why) this works, read the next chapters of this guide.

Building a bootable target

In order to build a raptor target that can be turned into a bootable iso, there are a few requirements we need to consider:

• The distribution must be Debian-based (or, preferably, Debian)
• The package live-boot must be installed, since it contains the necessary tools and scripts
• A linux kernel package must be installed (e.g. linux-image-amd64).

Here is an example of a fairly minimal base layer, which can be turned into an iso:

# Start from a docker iso
FROM docker://debian:trixie

# Set root password to "raptor"
RUN usermod -p "$1$GQf2tS9s$vu72NbrDtUcvvqnyAogrH0" root

# Update package sources, and install packages
RUN apt-get update
RUN apt-get install -qy systemd-sysv live-boot linux-image-amd64

The kernel package can take a bit of time to install, so let’s start a new layer for further customization. This way, we don’t need to rebuild the base layer with every change we make to the upper layer:

# Start from previous layer
FROM base

# Update package sources, and install ssh server
RUN apt-get update
RUN apt-get install -qy openssh-server

Of course, these layers could easily be combined, but it is good to get in a habit of separating things into reasonable layers. This improves caching and makes builds faster, since more can be reused between builds.

Now we are ready to build the layers. Since ssh derives from base (which derives from a docker layer), we just have to request raptor to build ssh. Raptor automatically determines dependencies, and builds them as needed.

sudo raptor build ssh

You should now see raptor build the ssh layer, with all command output from the apt-get commands being shown in the terminal.

Once complete, you can quickly verify that the layer is complete, by running the same command again. This time it should very quickly display a message, indicating that each layer is complete:

$ sudo raptor build ssh
[*] Completed [675DE2C3A4D8CD82] index.docker.io-library-debian-trixie
[*] Completed [AB1DD71BFD07718B] base
[*] Completed [80E4F4E5B0E2F6BA] ssh

In the next chapter we will see how we can turn these layers into a bootable iso file.

Generating an iso file

In the last chapter we built some raptor layers that serve as the file content for the bootable iso we want to build.

These layers are very similar to layers in a docker file¹.

After building ssh (which implies the base target), their respective contents can be found in layers/....

Once built like this, it’s possible to “run” the container, like so:

$ sudo raptor run ssh id
[*] Completed [675DE2C3A4D8CD82] index.docker.io-library-debian-trixie
[*] Completed [AB1DD71BFD07718B] base
[*] Completed [80E4F4E5B0E2F6BA] ssh
uid=0(root) gid=0(root) groups=0(root)

Instead of id, an interactive command like sh or bash could be specified. This is very similar to how docker run works, except raptor containers are ephemeral by default. If no further options are specified, any changes made to the container will only be saved while the container is running.

This is all well and good, but we are interested in building a bootable iso from the layers, not just running them as a container.

It turns out that having a stack of layers available, is quite a powerful primitive. With the right commands, we could build all kinds of output from them; a .tar.gz file with a release archive, a bootable iso, or even a disk image for a machine.

In other words, the same input layers can be post-processed into a wide variety of output layers, depending on what you need.

To help get started with this, Raptor provides a standard set of build containers, in the companion project raptor-builders:

• [deblive] Builds Debian liveboot isos from Raptor layers.
• [disk-image] Builds disk images (for virtual or physical machines) from Raptor layers.

Since this walkthrough is focused on making a liveboot iso, we’ll use the deblive builder to convert layers to an iso file.

First, we need to clone (or otherwise download) the raptor-builders project:

git clone https://github.com/chrivers/raptor-builders

This builder uses mounts to access input, output and cache from outside the container (see MOUNT).

To build the layers from the last step into a debian liveboot iso, use the following command (assuming raptor-builders is checked out next to the directory containing ssh.rapt):

sudo raptor run                \
  -L book   book/example       \
  -L rbuild ../raptor-builders \
  -C liveboot-cache            \
  -I '$book.ssh'               \
  -O custom-liveboot.iso       \
  '$rbuild.deblive'

sudo raptor run                    \
  --link book   book/example       \
  --link rbuild ../raptor-builders \
  --cache liveboot-cache           \
  --input '$book.ssh'              \
  --output custom-liveboot.iso     \
  '$rbuild.deblive'

(See the section on module names to learn more about --link and the $rbuild notation.)

This build process works, but the required command is fairly long and complicated.

In the next chapter, we’ll take a look at raptor-make, and how it can simplify the process.

1. There’s a key difference between how Raptor and Docker handles layers. In Docker, each command (RUN, COPY, etc) creates a new layer. In Raptor, each .rapt file forms a new layer. ↩

Make

In the previous chapter we took a look at the final command used to build a debian liveboot iso:

sudo raptor run                    \
  --link book   book/example       \
  --link rbuild ../raptor-builders \
  --cache liveboot-cache           \
  --input '$book.ssh'              \
  --output custom-liveboot.iso     \
  '$rbuild.deblive'

As mentioned, this works, but it doesn’t exactly roll off the tongue.

Having to type this command every time we want to build the iso, is not a satisfying solution.

To solve this, the subcommand raptor make is used. It is a make-like system, where dependencies are specified in a configuration file.

For make, this is a Makefile, for Rust it’s Cargo.toml, and for raptor make we have Raptor.toml.

Building a Raptor.toml for your project is recommend, but not required. It is perfectly possible to start using raptor from the command line, and only write a Raptor.toml file when it feels right.

That being said, the format for Raptor.toml has been designed to be a smooth transition from a constructed command line. The smallest valid Raptor.toml file is an empty one.

Let’s start by adding the two linked packages (--link arguments):

[raptor.link]
book = "book/example"
rbuild = "../raptor-builders"

We have added two package links, but of course any number can be added as desired.

All sections named [raptor.*] are settings for Raptor. The section [raptor.link] is for specifying linked packages.

Next, we will define a run job, which accounts for almost all of the remaining command line arguments.

A Raptor.toml file can have any number of [run.*] sections, each with their own name. Each of those sections is a separate run job. Let’s use book-ssh for this example:

[raptor.link]
book = "book/example"
rbuild = "../raptor-builders"

# The name `book-ssh` is not special.
# Feel free to choose any name you like!
[run.book-ssh]
target = "$rbuild.deblive"
cache = "liveboot-cache"
input = "$book.ssh"
output = "custom-liveboot.iso"

[raptor.link]
book = "book/example"
...

[run.example-1]
# $book refers to the link name above
input = "$book.ssh"
...

[raptor.link]
example = "book/example"
...

[run.example-2]
# $example refers to the link name above
input = "$example.ssh"
...

Now that we have encoded all the arguments, we can use a much simpler raptor make command to run this build job:

sudo raptor make book-ssh

Module names

In Raptor, “module names” are the names Raptor files use to refer to other Raptor files.

This is the case both for the FROM instruction (where the modules have the .rapt extension) and INCLUDE (.rinc extension).

There are three types of module names; Relative, Absolute and Package names.

The difference between these forms is how they resolve to paths in the filesystem. For a detailed explanation of each, see the following sections.

Instancing

Each of these forms support instancing, which is a way to pass a single, simple parameter through the module name, by appending a @ followed by the value, to the module name.

All raptor files are either instanced (example@.rapt / example@.rinc) or not (example.rapt / example.rinc).

Non-instanced files cannot be referenced with an instanced name, and vice versa.

Users of systemd might recognize this pattern, which is used in the same way there.

To learn more, read the section on instancing.

Learn more

Each type of module name is described in more detail:

• Relative
• Absolute
• Package

Relative module names

The first, and arguably simplest form, is the Relative name. It is characterized by not having a dollar sign ($) in front of the first element, unlike the other two forms.

Relative module paths form a sequence of one or more elements, which are the names between the dots (.).

Each element before the last is viewed as a directory. The last element is the file name, appended with either .rapt for FROM instructions, or .rinc for INCLUDE instructions.

In other words, a.b.c.d becomes a/b/c/d.rapt (FROM) or a/b/c/d.rinc (INCLUDE).

Examples:

Absolute module names

Relative module names are simple, but lack the ability to point upwards in the directory heirarchy.

For example, suppose you want to organize a set of Raptor files like so:

~/project/base.rapt
~/project/hostname.rinc
~/project/target/common.rapt
~/project/target/frontend.rapt
~/project/target/database.rapt
~/project/lib/utils/tools.rinc

The base layer can INCLUDE hostname, but the frontend and database targets have no way to specify they want to build FROM the base layer!

This is where absolute module names are useful.

By prefixing a name with $. (dollar + dot) the name refers to the root of the current package.

We will go into more detail of what a package is, in the next section. For now, it is enough to know that when invoking raptor on a target, the root for that target is the directory that raptor was started from.

Package module names

In the previous section, we briefly mentioned packages.

Raptor is designed to work collaboratively, encouraging sharing of code between projects, and people. A Raptor package is simply a collection of useful Raptor files, typically distributed as a git repository.

This means we need a robust way to refer to Raptor files outside of our current project, as well as a way to tell Raptor how to find these files.

This is where package module names are used.

First, let us take a look at how to make Raptor aware of external code bases. This is called linking, analogous to how the term is used when building a program from several sources.

When invoking raptor, the -L option defines a linked raptor package:

sudo raptor -L name src ...

For example, imagine we are working on a web service, and we want to generate the deployment with Raptor. Now imagine this requires a database server, and that someone else is responsible for the Raptor code that controls the database.

Then we might have a file layout like so:

~/project/web/server.rapt
~/project/database/db-setup.rinc

We fully control ~/projects/web, but ~/project/database is a git repository we only pull changes from.

We would like to INCLUDE the db-setup module, but it exists outside our own repository.

This can be solved by declaring database as linked package:

$ cd ~/project/web
$ sudo raptor build server -L database ../database

Now, we can refer to the content of ../database by using $database.

Instancing

Raptor files can be instanced, which makes them work as a template.

Instanced files can be recognized by ending in @.rapt (for FROM) or @.rinc (for INCLUDE).

File names and syntax

The table above shows some examples of instanced and non-instanced Raptor files.

Using instancing

So far, we have seen how to create templated (instanced) Raptor files, and how to reference them to provide a value.

Now we will see how to use the provided value, so that instancing becomes useful. Let us start the simplest possible example

WRITE "We are instance {{instance}}\n" /root/build-stamp.txt

This writes a human-readable line of text to a file, including the instance id.

Now we can use it from another file:

# Will write "We are instance 1\n" to /root/build-stamp.txt
INCLUDE build-stamp@1

Strings

In raptor files, string values can be expressed as quoted strings, or in certain cases as so-called barewords:

RUN echo these are barewords

RUN "echo" "these" "are" "quoted" "string"

For example, the following two statements are equivalent:

RUN echo word1 word2

RUN echo "word1" "word2"

But the following two statements are not:

# This creates 1 file called "filename with spaces"
RUN touch "filename with spaces"

# This creates *3 files* called "filename", "with", and "spaces", respectively
RUN touch filename with spaces

String escaping

When using a quoted string, the backslash character (\) gains special meaning, and is known as the escape character.

When it is followed by certain other characters, the combined expression is replaced in the string:

A backslash followed by any other character will result in a parse error.

Expressions

Raptor supports a limited form of inline literal expressions (i.e. “values”), that can be specified when using the INCLUDE and RENDER instructions.

Booleans

Boolean values work identically to json (and many other languages):

Integers

Integer values are supported.

Be aware that raptor always uses i64 (signed 64-bit) integers.

This is unlike the typical json implementations, that uses f64 (64-bit floating-point) numbers, in two important ways:

A) The valid range for i64 integers is -9223372036854775808 to 9223372036854775807, inclusive. This should be sufficient for most applications. Any integer in this range will be represented exactly.

B) Floating-point (i.e. fractional) numbers are not supported. For example, 3.14 is not valid in raptor. Instead, consider passing such a value as a string.

Currently, alternate integer bases (i.e. hexadecimal or octal) are not supported.

Strings

String are supported, and work much like they do in json, or other common notations.

Lists

Lists are supported, with a syntax very similar to json. The only difference is that raptor allows an optional trailing comma after the last list element, while json does not.

[1, 2, 3]

[true, false]

[["a", "b"], 123]

Maps

Maps (also typically known as dicts or hashmaps) contain a set of (key, value) pairs.

The syntax is similar to json. Like lists, raptor allows an optional trailing comma after the last key-value pair.

{"answer": 42}

{"name": "Heart of Gold", "engine": "Improbability drive"}

File options

Several instructions (COPY, WRITE, RENDER, MKDIR) write files into the build target. They all supports common options that affect how the files are written.

Change mode: --chmod <mode>

The --chmod option specifies the mode bits (i.e. permissions) associated with the file. The mode is specified as 3 or 4 octal digits.

Examples:

# these are equivalent:
COPY --chmod  755 script.sh /root/script.sh
COPY --chmod 0755 script.sh /root/script.sh

# set suid bit:
COPY --chmod 4755 sudo /usr/bin/sudo

# user access only, read-only:
WRITE --chmod 0400 "secret" /etc/service/token

Change owner: --chown <owner>

The --chown option specifies the user and/or group to own the file.

Create parent directories: -p

The -p option instructs MKDIR to create parent directories as needed.

Importantly, it also makes MKDIR accept existing directories, including the last directory.

This is identical to the behavior of -p with the shell command mkdir.

# will fail if:
#   A) /foo is missing
# or:
#   B) /foo/bar already exists
MKDIR /foo/bar

# this will create:
#   /foo (if missing)
# and then
#   /foo/bar (if missing)
MKDIR -p /foo/bar

Mount types

When running containers, Raptor supports mounting various file resources into the container namespace.

In the simplest form, this means presenting a directory from the host environment, at a specified location in the container.

This is equivalent to how Docker uses volume mounts (-v / --volume).

However, Raptor supports more than just mounting directories:

Read more about the specific mount types:

• --simple
• --file
• --layers
• --overlay

Mount type --simple

This is the default mount type.

A --simple mount will mount a directory from the host into the container. Docker users are likely to be familiar with this concept.

Example

FROM docker://debian:trixie

MOUNT --simple list /input

CMD "ls -l /input"

This container can be run, to provide a file listing on the mounted directory:

$ sudo raptor run file-lister -M list /tmp
... <"ls" output would be shown here> ...

Mount type --file

This mount type requires that a single file is mounted.

Example

Let us expand on the earlier example, to make the file lister provide output to a file.

FROM docker://debian:trixie

MOUNT --simple input /input
MOUNT --file output /output

CMD "ls -l /input > /output"

Now that we have named the mounts input and output, we can use the shorthand notation for convenience:

$ sudo raptor run file-lister-output -I /etc -O /tmp/filelist.txt
...
$ sudo cat /tmp/filelist.txt
... <"ls" output would be shown here> ...

The above example would generate a file listing of /etc from the host, and place it in /tmp/filelist.txt. However, the execution of ls takes place in the container.

Mount type --layers

The --simple and --file mount types both present content from the host filesystem inside the container, and both have equivalents in Docker.

This is not the case for --layers, which is specific to Raptor.

When using a --layers mount, the input argument when running Raptor is not a file path, but a set of Raptor build targets.

Let us take a look at a target, that lists the files in a --layers mounts:

FROM docker://debian:trixie

MOUNT --layers input /input

CMD "ls -l /input"

Let’s try running this, using the previous file-lister.rapt target as input:

$ sudo raptor run layers-lister -I file-lister
total 12
drwxr-xr-x  2 root root 4096 Oct 17 13:37 file-lister-16689594BA5D2989
drwxr-xr-x 17 root root 4096 Sep 29 00:00 index.docker.io-library-debian-trixie-675DE2C3A4D8CD82
-rw-r--r--  1 root root  200 Oct 17 13:37 raptor.json

We see that each layer in the input is available as a directory, but also a raptor.json file.

Think of this file as a manifest of the contents in the --layers mount. It contains useful metadata about the inputs, including which targets have been specified, as well as the stacking order for each target:

{
  "targets": [
    "file-lister"
  ],
  "layers": {
    "file-lister": [
      "index.docker.io-library-debian-trixie-675DE2C3A4D8CD82",
      "file-lister-16689594BA5D2989"
    ]
  }
}

At first, this might seem like overly complicated. If we just needed the layer order, surely a simple text file would suffice?

In fact, multiple build targets can be specified at the same time:

$ sudo raptor run layers-lister -I file-lister -I file-lister-output
total 16
drwxr-xr-x  2 root root 4096 Oct 17 11:33 file-lister-16689594BA5D2989
drwxr-xr-x  2 root root 4096 Oct 17 11:31 file-lister-output-2CFDE4FEBD507157
drwxr-xr-x 17 root root 4096 Sep 29 00:00 index.docker.io-library-debian-trixie-675DE2C3A4D8CD82
-rw-r--r--  1 root root  381 Oct 17 11:45 raptor.json

We now have multiple build targets, and the Docker layer is shared between both inputs.

However, we can still make sense of this using raptor.json:

{
  "targets": [
    "file-lister",
    "file-lister-output"
  ],
  "layers": {
    "file-lister-output": [
      "index.docker.io-library-debian-trixie-675DE2C3A4D8CD82",
      "file-lister-output-2CFDE4FEBD507157"
    ],
    "file-lister": [
      "index.docker.io-library-debian-trixie-675DE2C3A4D8CD82",
      "file-lister-16689594BA5D2989"
    ]
  }
}

This mount type is useful for any build container that needs to work the contents of individual layers.

For example, the deblive builder¹ uses this mount type, in order to build Debian liveboot images.

1. From the raptor-builders companion project. ↩

Mount type --overlay

Like --layers, this is a Raptor-specific mount type.

Raptor targets are almost always built from a set of layers (i.e., there’s at least one FROM instruction).

When running a raptor container, this stack of layers is combined using overlayfs, which makes the kernel present them to the container as a single, unified filesystem.

For build containers that need to operate on this combined view of a build target, the --overlay mount type is available.

For example, the disk-image builder[^deblive] uses this mount type, in order to build disk images for virtual (or physical) machines.

FROM docker://debian:trixie

MOUNT --overlay input /input

CMD "ls -l /input"

$ sudo raptor run overlay-lister -I file-lister
total 60
lrwxrwxrwx  1 root root    7 Aug 24 16:20 bin -> usr/bin
drwxr-xr-x  2 root root 4096 Aug 24 16:20 boot
drwxr-xr-x  2 root root 4096 Sep 29 00:00 dev
drwxr-xr-x 27 root root 4096 Sep 29 00:00 etc
drwxr-xr-x  2 root root 4096 Aug 24 16:20 home
lrwxrwxrwx  1 root root    7 Aug 24 16:20 lib -> usr/lib
lrwxrwxrwx  1 root root    9 Aug 24 16:20 lib64 -> usr/lib64
drwxr-xr-x  2 root root 4096 Sep 29 00:00 media
drwxr-xr-x  2 root root 4096 Sep 29 00:00 mnt
drwxr-xr-x  2 root root 4096 Sep 29 00:00 opt
drwxr-xr-x  2 root root 4096 Sep 29 00:00 proc
drwx------  2 root root 4096 Sep 29 00:00 root
drwxr-xr-x  3 root root 4096 Sep 29 00:00 run
lrwxrwxrwx  1 root root    8 Aug 24 16:20 sbin -> usr/sbin
drwxr-xr-x  2 root root 4096 Sep 29 00:00 srv
drwxr-xr-x  2 root root 4096 Aug 24 16:20 sys
drwxrwxrwt  2 root root 4096 Sep 29 00:00 tmp
drwxr-xr-x 12 root root 4096 Sep 29 00:00 usr
drwxr-xr-x 11 root root 4096 Sep 29 00:00 var

Raptor make

Overall structure

Below is an example of the overall structure of a Raptor.toml file.

Note: All parts are optional, so you only need to define the parts you need.

[raptor.link]
name1 = "path/to/source1"
name2 = "path/to/source2"
...

[run.purple]
# ..run target here..

[run.orange]
# ..run target here..

[group.green]
# ..group here..

[group.yellow]
# ..group here..

Run target format

A run target ([run.<name>]) is the most commonly used feature in Raptor.toml.

The structure for a job named example is shown below, where each field is specified with its default value.

Note: Only the target field is required! Everything else can be specified as needed.

[run.example]
target = <required>

# Cache mounts
# (default is empty list)
#
# Note: A single element can be specified as a string instead of the list
cache = []

# Input mounts
# (default is empty list)
#
# Note: A single element can be specified as a string instead of the list
input = []

# Output mounts
# (default is empty list)
#
# Note: A single element can be specified as a string instead of the list
output = []

# Entrypoint arguments
entrypoint = []

# Command arguments
args = []

# BTreeMap<String, String>
env = {}

# State directory for container state
# (default is unset, meaning ephemeral containers)
#state_dir =

Group format

A group is used to collectively refer to a number of run and build jobs, by a single name.

[group.example]
# List of layers to build
#
# Default is empty list
build = []

# List of names for [run.<name>] targets to run
#
# Default is empty list
run = []

Grammar

The parsing starts from the first rule, <file>, and proceeds from there.

<file>         ::= <statement>*

<statement>    ::= <from>
                 | <mount>
                 | <render>
                 | <write>
                 | <mkdir>
                 | <copy>
                 | <include>
                 | <run>
                 | <env>
                 | <workdir>
                 | <entrypoint>
                 | <cmd>

<from>         ::= "FROM" <from-source> "\n"
<mount>        ::= "MOUNT" <mount-type>? <word> <path> "\n"
<render>       ::= "RENDER" <file-option>* <path> <path> <include-arg>* "\n"
<write>        ::= "WRITE" <file-option>* <value> <path> "\n"
<mkdir>        ::= "MKDIR" <mkdir-option>* <path> "\n"
<copy>         ::= "COPY" <file-option>* <path>+ <path> "\n"
<include>      ::= "INCLUDE" <module-name> <include-arg>* "\n"
<run>          ::= "RUN" <word>+ "\n"
<env>          ::= "ENV" <env-assign>+ "\n"
<workdir>      ::= "WORKDIR" <path> "\n"
<entrypoint>   ::= "ENTRYPOINT" <word>* "\n"
<cmd>          ::= "CMD" <word>* "\n"

<env-assign>   ::= <word> ( "=" <value> )?
<mount-type>   ::= "--file" | "--simple" | "--layers" | "--overlay"

<mkdir-option> ::= <file-option> | "-p"
<file-option>  ::= <file-chown> | <file-chmod>
<file-chown>   ::= "--chown" "="? <chown>
<file-chmod>   ::= "--chmod" "="? <chmod>
<chown>        ::= (<word> (":" <word>?)?) | (":" <word>?)
<chmod>        ::= /* built-in rule: 3 or 4 octal digits */

<include-arg>  ::= <word> ( "=" <expression> )?
<expression>   ::= <expr-lookup> | <expr-value>
<expr-lookup>  ::= <word> ("." <word>)*
<expr-value>   ::= <expr-list>
                 | <expr-map>
                 | <expr-string>
                 | <expr-number>
                 | <expr-bool>
<expr-list>    ::= "[" ( <expr-value>   ( "," <expr-value>   )* ","? )? "]"
<expr-map>     ::= "{" ( <expr-mapitem> ( "," <expr-mapitem> )* ","? )? "}"
<expr-mapitem> ::= <expr-value> ":" <expr-value>
<expr-string>  ::= /* built-in rule: see section on string escapes */
<expr-number>  ::= <digit>+
<expr-bool>    ::= "true" | "false"
<digit>        ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"

<module-name>  ::= <mn-package>? <mn-body> <mn-instance>?
<mn-package>   ::= "$" <word>? "."
<mn-body>      ::= <word> ( "." <word> )*
<mn-instance>  ::= "@" <word>

Instruction: FROM

FROM [<schema>://]<from-source>

The FROM instruction bases the current layer on top of some the specified layer.

This instruction is not required. If no FROM instruction is used, the target is building from an empty base, with no dependencies.

Multiple FROM instructions are not supported.

From sources

Raptor supports multiple options for from-source:

Raptor sources

When no schema is specified, the from-source is assumed to be the module name of another raptor layer.

# Dockerfile
FROM base

However, unlike docker files, raptor can point to raptor files in other directories, or even other packages. See module names for an overview.

Examples

# This will depend on `base.rapt`
FROM base

# This will depend on `library/debian.rapt`
FROM library.debian

Docker sources

To use a docker image as the basis for a raptor layer, specify the name of the docker image, prefixed with docker://, e.g:

FROM docker://debian:trixie

There are multiple (optional) parts in a docker reference, which has a surprisingly intricate syntax.

Raptor supports the entire grammar for docker references, so anything that docker pull will accept, should work with FROM docker:// in raptor.

Instruction RUN

RUN <command> [...<arg>]

The RUN instruction executes the given command inside the build namespace.

# enable the foo service
RUN systemctl enable foo.service

# BROKEN: This will call "cat" with 3 arguments
RUN cat /etc/hostname "|" md5sum
#   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this will not work

Instead, /bin/sh can be called explicitly:

# This will produce the md5sum of /etc/hostname
RUN /bin/sh -c "cat /etc/hostname | md5sum"

Instruction ENV

ENV <key>=<value> [...<key=value>]

The ENV instruction sets one or more environment variables inside the build namespace.

Example:

ENV CFLAGS="--with-sprinkles"
ENV API_TOKEN="acbd18db4cc2f85cedef654fccc4a4d8" API_USER="user@example.org"

The ENV instruction affects all instructions that come after it, in the same layer.

It is possible to overwrite a value that has been set previously:

FROM docker://busybox

ENV HITCHHIKER="Arthur Dent"
RUN sh -c "echo $HITCHHIKER" # outputs "Arthur Dent"

ENV HITCHHIKER="Ford Prefect"
RUN sh -c "echo $HITCHHIKER" # outputs "Ford Prefect"

Instruction WORKDIR

WORKDIR <path>

The WORKDIR instruction changes the current working directory inside the build namespace. This affects all subsequent relative paths, including the RUN instruction.

The workdir is not inherited through FROM.

The initial workdir is always /.

Example

# This will copy "program" to "/bin/program" (since initial directory is "/")
COPY program bin/program

# This creates /foo
RUN /bin/sh -c "touch foo"



# Switch to /usr
WORKDIR /usr

# The same command will now copy "program" to "/usr/bin/program"
COPY program bin/program



# Switch to /tmp
WORKDIR /tmp

# This creates /tmp/foo
RUN /bin/sh -c "touch foo"

Instruction WRITE

WRITE [<file-options>] <value> <path>

The WRITE instruction writes a fixed string to the given path.

A file can be added to the build output with COPY, but sometimes we just need to write a short value, and COPY might feel like overkill.

Using WRITE, we can put values directly into files:

WRITE "hello world" hello.txt

WRITE "hostname\n" /etc/hostname

The same file options as COPY and RENDER are accepted:

WRITE --chmod 0600 --chown service:root "API-TOKEN" /etc/service/token.conf

Instruction MKDIR

MKDIR [<file-options>] <path>

The MKDIR instruction creates an empty directory inside the build target.

This is roughly equivalent to the following command:

RUN mkdir /foo

However, using RUN mkdir requires the mkdir command to be available and executable inside the build target. This is not always the case, especially when building things from scratch.

Example

MKDIR /data

MKDIR -p --chown babelfish:daemon /usr/local/translate/

Instruction COPY

COPY [<file-options>] <source> [...<source>] <destination>

The COPY instruction takes one or more source files, and copies them to the destination.

If multiple source files are specified, the destination MUST BE a directory.

Instruction INCLUDE

INCLUDE <module-name> [...<key>=<value>]

The INCLUDE instruction points to a Raptor include file (.rinc) to be executed. When using INCLUDE, any number of local variables can be passed to the included target.

For example, if we have previously made the file lib/install-utils.rinc that installs some useful programs, we can use that file in build targets:

# Note: We use module name notation when including files
#
# The file is called `lib/install-utils.rinc`, which makes
#    the module name `lib.install-utils`
INCLUDE lib.install-utils

We can also make the component accept parameters, to make powerful, flexible components:

# hypothetical library function to update "/etc/hostname"
INCLUDE lib.set-hostname hostname="server1"

In the above example, we set the hostname of a server using an included component.

INCLUDE setup-thing username=username password=password

INCLUDE setup-thing username password

Instruction RENDER

RENDER [<file-options>] <source> <destination> [...<include-arg>]

The RENDER instruction renders a file from a template, and writes it to the specified destination. It accepts the same key=value arguments as INCLUDE. These arguments are made available in the template.

Example:

RENDER widgetfactory.tmpl /etc/widgetd/server.conf host="example.org" port=1234

The short form name (meaning name=name) is also supported here.

For example, in a component where host and port are available in the environment:

RENDER widgetfactory.tmpl /etc/widgetd/server.conf host port

Instruction MOUNT

MOUNT [--<mount-type>] <name> <destination>

By using MOUNT, targets can specify certain resources (files, directories, layers, etc) that should be made available when running the container.

Raptor mounts are identified by a name, which is used when running the container, to declare what to mount.

When running a raptor container, a mount input is specified with the -M <name> <source> command line option.

For example, suppose we have a simple container (disk-usage.rapt) that just calculates the disk space used by a mount:

FROM docker://debian:trixie

# Declare the mount "input", and place it at /input
MOUNT input /input

# Calculate the disk space used in /input
CMD "du -sh /input"

If we try to run this, we will get an error message:

$ sudo raptor run disk-usage
[*] Completed [675DE2C3A4D8CD82] index.docker.io-library-debian-trixie
[*] Completed [A24E97B01374CFEF] disk-usage
[E] Raptor failed: Required mount [input] not specified

As we can see, the container builds correctly, but fails because the input mount is not specified.

To fix this, we specify the input mount:

sudo raptor run disk-usage -I /tmp
[*] Completed [675DE2C3A4D8CD82] index.docker.io-library-debian-trixie
[*] Completed [4BDD0649E00CA728] disk-usage
128M    /input

We could have specified -I /tmp as -M input /tmp, but the short form usually makes the command easier to follow.

Mount type

The example have just looked will probably feel familiar to Docker users, since it is very similar to docker volumes, which are also bind mounts from the host to the container namespace.

However, Raptor mounts are more advanced than that.

For example, in order to build a Debian liveboot iso, we need to provide the Debian live-boot scripts with a set of squashfs files, generated from the individual layers of one or more raptor targets.

Docker does not easily provide access to container layers, as it is seen as more of an implementation detail.

In contrast, Raptor considers layers a first-class object, and makes them available using the MOUNT instruction.

To learn more about different mount types, please see the mount types section.

Instruction ENTRYPOINT

Instruction CMD