Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Intro

Noxa is a tool to ease the configuration of multi-host NixOS configurations. Starting with the motivation, design goals, and design decisions, this handbook guides you through setting up a multi-host NixOS configuration.

Motivation

NixOS is a Linux distribution configuration solution composed of modules and packages NixOS Manual. While nixpkgs provides many modules for configuring NixOS machines, it lacks fundamental support for configuration beyond the scope of a single host.

In practice, some hosts share configuration settings. Imagine that you configure four different NixOS machines and would like to set up a Wireguard network between them. In the end, you will configure networking.wireguard.interfaces.<name> on each host, but there are several open questions:

  1. How do you manage secrets? For some secrets, how do you share public interface keys with each host or handle connection keys shared between two hosts?
  2. Some hosts will have public IP addresses and a direct connection should be established, while others take on a client role and do not own a public IP address. The trait of having a public IP address will ultimately belong to the configuration of each host, while each other host needs to configure their connection settings accordingly.

To summarize, there are configuration settings shared among different hosts and do not belong to any specific one, and there are configuration values owned by hosts but accessed by others to configure their NixOS configuration.

Currently, nixpkgs does not support these multi-host configuration scenarios and multi-host dependency.

Goal

The goal of Noxa is to fill this gap and provide a framework to support the configuration of hosts that depend on each other.

Contributing

Contributions are welcome. Suggest new features and bugs via issue. Pull requests are welcome.

Other frameworks already aim to implement multi-host NixOS configuration; the list is provided below. Why another framework? 1. Because of the following design goal: Plug-and-play, new hosts added, should have minimal configuration already configured according to the existing hosts. 2. Learning of Nix language and ecosystem.

List of other multi-host configuration framework (feel free to add others):

Options

This section is the option specification of Noxa. The specification is split into the following parts:

age.secrets

Extension of the age (agenix) secrets module to provide secrets for multi-host NixOs configurations.

Type: attribute set of (submodule)

Declared by:

age.secrets.<name>.hosts

The hosts that have access to this secret.

Type: unique list of string

Default:

[
  "<hostname>"
]

Example:

[
  "host1"
  "host2"
]

Declared by:

age.secrets.<name>.ident

The name of the secret.

This is the name of the secret, e.g. “wg-interface-key”.

Type: string

Example: "wg-interface-key"

Declared by:

age.secrets.<name>.identifier

A unique identifier for the secret, derived from the module and name. This may be used to name the secret.

Type: string (read only)

Example: "host:noxa.wireguard.interfaces.some-interface::wg-interface-key"

Declared by:

age.secrets.<name>.module

The owning module of that secret.

Typically this is the name of module declaring the secret, e.g. “noxa.wireguard.interfaces.<name>”.

Type: string

Example: "services.openssh"

Declared by:

noxa.secrets.enable

Enables the secrets module, multi-host secret management.

Type: boolean

Default: true

Declared by:

noxa.secrets.def

A list of secrets that are managed by the noxa secrets module.

Each secret is either a host specific secret or a shared secret. Host specific secrets are only available on the host that owns them, while shared secrets are available on all hosts that declare them.

The options provided will be passed to the agenix module, by using the identifier as the name of the secret. The identifier is derived from the module and name of the secret, e.g. “host:noxa.wireguard.interfaces.some-interface::wg-interface-key” or “shared:noxa.wireguard.interfaces.some-interface:host1,host2:wg-preshared-connection-key”.

Type: list of (submodule)

Default: [ ]

Declared by:

noxa.secrets.def.*.generator.dependencies

Other secrets on which this secret depends. See agenix-rekey documentation.

Type: null or (list of unspecified value) or attribute set of unspecified value

Default: null

Example: [ config.age.secrets.basicAuthPw1 nixosConfigurations.machine2.config.age.secrets.basicAuthPw ]

Declared by:

noxa.secrets.def.*.generator.script

Generator script, see agenix-rekey documentation.

Type: null or string or function that evaluates to a(n) string

Default: null

Declared by:

noxa.secrets.def.*.generator.tags

Optional list of tags that may be used to refer to secrets that use this generator.

See agenix-rekey documentation for more information.

Type: null or (list of string)

Default: null

Example:

[
  "wireguard"
]

Declared by:

noxa.secrets.def.*.hosts

The hosts that have access to this secret.

Type: unique list of string

Default:

[
  "<hostname>"
]

Example:

[
  "host1"
  "host2"
]

Declared by:

noxa.secrets.def.*.ident

The name of the secret.

This is the name of the secret, e.g. “wg-interface-key”.

Type: string

Example: "wg-interface-key"

Declared by:

noxa.secrets.def.*.identifier

A unique identifier for the secret, derived from the module and name. This may be used to name the secret.

Type: string (read only)

Example: "host:noxa.wireguard.interfaces.some-interface::wg-interface-key"

Declared by:

noxa.secrets.def.*.module

The owning module of that secret.

Typically this is the name of module declaring the secret, e.g. “noxa.wireguard.interfaces.<name>”.

Type: string

Example: "services.openssh"

Declared by:

noxa.secrets.def.*.rekeyFile

The path to the rekey file for this secret. This is used by the agenix-rekey module to rekey the secret.

Type: absolute path (read only)

Declared by:

noxa.secrets.hostSecretsPath

The path where host secrets are stored. This is the path where noxa will look for (encrypted) host specific secrets.

This directory contains encrypted secrets for each host. Secrets in this directory are host specific, at least the secret part of the secret is owned by a single host and only published to that host.

An example secret would be the private wireguard key for an interface. Still the public key might be shared with other hosts.

ATTENTION: Since this path is copied to the nix store, it must not contain any secrets that are not encrypted.

Type: absolute path

Declared by:

noxa.secrets.options.enable

Enables the ‘simple’ options, by providing settings proxy, a user can set the options, inside the noxa.secrets.options module that will provide sensible defaults for the agenix and agenix-rekey module.

If this is set to false, the user must set-up the agenix and agenix-rekey modules manually.

Type: boolean

Default: true

Declared by:

noxa.secrets.options.hostPubkey

The public key of the host that is used to encrypt the secrets for this host.

Type: null or string

Default: null

Declared by:

noxa.secrets.options.masterIdentities

A list of identities that are used to decrypt encrypted secrets for rekeying.

Type: list of (submodule)

Declared by:

noxa.secrets.options.masterIdentities.*.identity

The identity that is used to encrypt and store secrets as .age files. This must be an absolute path, given as string to not publish keys to the nix store.

This is the private key file used.

Type: null or string

Declared by:

noxa.secrets.options.masterIdentities.*.pubkey

The identity that is used to encrypt and store secrets as .age files. This is the age public key of the identity, used to encrypt the secrets.

This is the public key file used.

Type: null or string

Declared by:

noxa.secrets.options.rekeyDirectory

The directory where the rekey files are stored. This is used by the agenix-rekey module to rekey the secrets. This directory must be writable by the user that runs the agenix-rekey module and added to the git repo.

It is recommended to use $\{config.networking.hostName} to create a unique directory for each host.

Type: absolute path

Declared by:

noxa.secrets.secretsPath

The path where all secrets are stored. Subfolders are created for host specific and shared secrets.

Type: null or absolute path

Declared by:

noxa.secrets.sharedSecretsPath

The path where secrets shared between several hosts are stored. This is the path where noxa will look for (encrypted) shared secrets.

This directory contains encrypted secrets that are shared between several hosts. Secrets in this directory are not host specific, they are not owned by a single host, but an group of hosts.

An example secret would be the pre-shared symmetric key for a wireguard interface peer.

Since this path is used by multiple hosts, it is recommended to set this path once for all hosts, instead of setting it per host.

ATTENTION: Since this path is copied to the nix store, it must not contain any secrets that are not encrypted.

Type: absolute path

Declared by:

noxa.sshHostKeys.generate

Generates SSH host keys on boot even if the openssh service is not enabled.

Type: boolean

Default: false

Declared by:

noxa.sshHostKeys.hostKeysPrivate

List of SSH private host keys, accessible during runtime.

Type: list of string (read only)

Declared by:

noxa.sshHostKeys.impermanencePathOverride

Override the storage location for the ssh keys. Since some modules, like the noxa.secrets module, depend on the keys being stored on a mounted disk during configuration activation, and not expose functionality of systemd orderings, this option can be used to override the storage location of the keys; useful when using impermanence setups.

Type: null or string

Default: null

Declared by:

noxa.wireguard.enable

Enables the WireGuard module, which a cross-host VPN setup utility for wireguard.

Type: boolean

Default: false

Declared by:

noxa.wireguard.interfaces

A set of WireGuard interfaces to configure. Each interface is defined by its name and contains its private key, public key, and listen port.

Type: lazy attribute set of (submodule)

Default: { }

Declared by:

noxa.wireguard.interfaces.<name>._deviceAddress

The device address assigned to the peer, based on the network address and device number. This is automatically generated.

Type: null or IPv4 address or IPv6 address (read only)

Declared by:

noxa.wireguard.interfaces.<name>.advertise.keepAlive

The keep alive interval remote peers should use when communicating with this interface.

Type: null or signed integer

Default: null

Declared by:

noxa.wireguard.interfaces.<name>.advertise.server

Options for wireguard servers. If a wireguard interface is regarded as a server (e.g. since it has a public IP address), it may advertise its service via the server.advertise option.

If set, all peers that would like to connect to that peer will use the advertised listen port and address as means of directly connecting to the server.

Further, if server.defaultGateway is set, all peers that do not advertise listen port and address will be reached via the server marked as default gateway. Therefore, only one interface may be marked as default gateway at any time.

Type: null or (submodule)

Default: null

Declared by:

noxa.wireguard.interfaces.<name>.advertise.server.defaultGateway

If set, this server will be the default gateway for clients.

Type: boolean

Default: false

Declared by:

noxa.wireguard.interfaces.<name>.advertise.server.firewallAllow

If set, the nixos firewall will allow incoming connections to the advertised listen port.

Type: boolean

Default: true

Declared by:

noxa.wireguard.interfaces.<name>.advertise.server.listenAddress

The address this server will listen on for incoming connections.

Type: IPv4 address or IPv6 address

Declared by:

noxa.wireguard.interfaces.<name>.advertise.server.listenPort

The port this server will listen on for incoming connections.

Type: signed integer

Declared by:

noxa.wireguard.interfaces.<name>.autostart

Specifies whether to autostart the WireGuard interface.

Only relevant if the backend is set to wg-quick.

Type: boolean

Default: true

Declared by:

noxa.wireguard.interfaces.<name>.backend

The backend to use for WireGuard config generation.

  • wireguard: Uses the networking.wireguard.interfaces module to generate the configuration.
  • wg-quick: Uses the networking.wg-quick.interfaces module to generate the configuration.

Type: one of “wireguard”, “wg-quick”

Default: "wireguard"

Declared by:

noxa.wireguard.interfaces.<name>.deviceAddresses

List of ip addresses to assign to this interface. The server will forward traffic to these addresses.

Type: list of (IPv4 address or IPv6 address)

Declared by:

noxa.wireguard.interfaces.<name>.deviceNumber

If set, a new address will be assigned to the peer with the following schema:

The network prefix defined in noxa.wireguard.interfaces.<name>.networkAddress will be used as the base for the device address, and the value of noxa.wireguard.interfaces.<name>.deviceNumber will be used in the device part. For example, if the network prefix is 1.2.0.0/16 and the device number is 11, the address added to the peer will be 1.2.0.11/32.

The device number might be a single number passed into the last component of the address.

Type: null or signed integer

Default: null

Example: "3"

Declared by:

noxa.wireguard.interfaces.<name>.gatewayOverride

If set, this interface will use the specified hostname as the gateway for connecting to the wireguard network.

Type: null or string

Default: null

Declared by:

noxa.wireguard.interfaces.<name>.keepAlive

The default keep alive interval this interface will use when communicating with remote peers.

If the remote end uses advertise.keepAlive, the minimum value of both will be used.

Type: null or signed integer

Default: null

Declared by:

noxa.wireguard.interfaces.<name>.kind.isClient

This interface has the role of a client, meaning it does not advertise other peers to connect to it. Instead, it connects to other peers, initiating the connection.

Type: boolean (read only)

Default: true

Declared by:

noxa.wireguard.interfaces.<name>.kind.isGateway

This interface is a server and is marked as the default gateway for clients. This means that clients will use this interface to reach other peers that do not advertise their listen port and address.

Type: boolean (read only)

Default: false

Declared by:

noxa.wireguard.interfaces.<name>.kind.isServer

This interface has the role of a server, meaning it advertises its listen port and address to peers.

Type: boolean (read only)

Default: false

Declared by:

noxa.wireguard.interfaces.<name>.networkAddress

The network IP addresses. On clients, traffic of this network will be routed through the WireGuard interface.

Type: IPv4 address, normalized network address, or (IPv6 address, normalized network address)

Declared by:

noxa.wireguard.routes

A set of intermediary connection information, automatically computed from the nixos configurations.

Type: lazy attribute set of (submodule) (read only)

Declared by:

noxa.wireguard.routes.<name>.neighbors

A set of connections for this interface, automatically computed from the nixos configurations.

Type: lazy attribute set of (submodule) (read only)

Declared by:

noxa.wireguard.routes.<name>.neighbors.<name>.keepAlive

The keep-alive interval for this connection, in seconds. If set to null, no keep-alive is configured.

Type: null or signed integer (read only)

Declared by:

noxa.wireguard.routes.<name>.participants.clients

A list of clients for this interface. Automatically populated.

Type: list of string (read only)

Declared by:

noxa.wireguard.routes.<name>.participants.gateways

A list of gateways for this interface. Automatically populated.

Type: list of string (read only)

Declared by:

noxa.wireguard.routes.<name>.participants.servers

A list of servers for this interface. Automatically populated.

Type: list of string (read only)

Declared by:

noxa.wireguard.routes.<name>.peers

A list of peers for this interface. Automatically populated.

Type: list of (submodule) (read only)

Declared by:

noxa.wireguard.routes.<name>.peers.*.target

The target hostname of the connection.

Type: string (read only)

Declared by:

noxa.wireguard.routes.<name>.peers.*.via

The hostname of the peer this connection is routed through.

Type: null or string (read only)

Declared by:

noxa.wireguard.secrets

A set of wireguard secrets. When using the .interfaces options, this set is automatically populated. Each peer will own its own set of secrets

Type: lazy attribute set of (submodule)

Default: { }

Declared by:

noxa.wireguard.secrets.<name>.presharedKeyFiles

The pre-shared key file for each peer (by hostname) of the wireguard interface

Type: lazy attribute set of string (read only)

Declared by:

noxa.wireguard.secrets.<name>.privateKeyFile

The private key file of the wireguard interface

Type: string (read only)

Declared by:

noxa.wireguard.secrets.<name>.publicKey

The public key the wireguard interface

Type: string (read only)

Declared by:

noxa.overlays

A list of overlays to apply to this nixos configuration. The inputs being the final output and the build results from the stage-1 evaluation of the NixOS configuration.

Evaluated by the noxa.lib.nixos-instantiate function.

Type of each entry in the list is a function of kind {final, prev, stageOne} -> { ... }.

  • final: the final output of the NixOS configuration, which is the result of the stage-2 evaluation.
  • prev: the previous output of the previous overlay, view of the stage-2 evaluation.
  • stageOne: the output of the stage-1 evaluation, which is the result of the first evaluation of the NixOS configuration.

Type: list of anything

Default: [ ]

Declared by: