Intro
Noxa is a tool to ease the configuration of multi-host NixOS configurations. Starting with the motivation, design goals, and design decisions, this handbook guides you through setting up a multi-host NixOS configuration.
Motivation
NixOS is a Linux distribution configuration solution composed of modules and packages NixOS Manual. While nixpkgs provides many modules for configuring NixOS machines, it lacks fundamental support for configuration beyond the scope of a single host.
In practice, some hosts share configuration settings. Imagine that you configure four different NixOS machines and would like to set up a WireGuard network between them. In the end, you will configure networking.wireguard.interfaces.<name> on each host, but there are several open questions:
- Is specifying each network redundantly on each host necessary? Boiling it down: specifying a network once globally should be enough.
- How do you manage secrets efficiently? Preshared connection keys, for example, are owned not by one but by two hosts. Should you specify them on several hosts redundantly, or would it be sufficient to declare the network topology globally?
To summarize, some configuration settings are shared among different hosts and do not belong to any specific one. For example, the existence of a WireGuard network and its configuration fundamentally does not belong to the configuration of a single NixOS host. Further, one host's configuration might depend on another's configuration.
Currently, nixpkgs does not support these multi-host configuration scenarios and multi-host dependencies.
Goal
The goal of Noxa is to fill this gap and provide a framework to support the configuration of multiple hosts that:
- Potentially depend on each other
- Depend on a global configuration
Whats contained in Noxa:
- The module system scaffolding to set up a noxa configuration.
- Noxa modules covering standard use-cases.
- Nixos modules that configure the specified hosts according to the global configuration from noxa modules
How does it work
At the highest level, Noxa uses the module system to provide a configuration environment. A Noxa module can extend the functionality of your configuration (like with standard NixOS modules).
The difference to the "normal" NixOS module system being the provided configuration options. Hosts are, for example, defined using the configuration value nodes.<name>.***. NixOS
configuration is then supplied via the nodes.<name>.configuration option.
For example, the following Noxa module declares two hosts, while hostB depends on the configuration values of hostA.
{config, ...}: {
# Declaration of `hostA`
nodes.hostA.configuration = {
services.openssh.enable = true;
/* Other NixOS configuration options */
};
# Declaration of `hostB`
nodes.hostB.configuration = {
imports = [
./some-nixos-module.nix
];
# Access config values from other hosts
networking.hostName =
"${config.nodes.hostA.configuration.networking.hostName}-copy";
};
imports = [
./some-other-noxa-module.nix
];
/* Other Noxa configuration options */
}
For a reference, which options are available to noxa modules checkout options.
Contributing
Contributions are welcome. Suggest new features and bugs via issues. Pull requests are welcome.
Related projects
Other frameworks already aim to implement multi-host NixOS configuration; the list is provided below. Why another framework? Learning the Nix language and ecosystem.
List of other multi-host configuration frameworks (feel free to add others):
Getting Started
todo
Options
You found the documentation/reference of all options and configuration values noxa exposes to the user.
Documentation by Module
| Module | Link | Description |
|---|---|---|
| Noxa Nodes | Options | Define and manage hosts; per-host/node configuration. |
| NixOS Secrets | Options | Manage secrets across multiple hosts. |
| Noxa Wireguard | Options | Declare Wireguard networks. |
| NixOS Wireguard | Options | Configure hosts according to globally defined Wireguard networks. |
age.secrets
Extension of the age (agenix) secrets module to provide
secrets for multi-host NixOs configurations.
Type: attribute set of (submodule)
Declared by:
age.secrets.<name>.hosts
The hosts that have access to this secret.
Type: unique list of string
Default:
[
"<noxa-host-id>"
]
Example:
[
"host1"
"host2"
]
Declared by:
age.secrets.<name>.ident
The name of the secret.
This is the name of the secret, e.g. “wg-interface-key”.
Type: string
Example:
"wg-interface-key"
Declared by:
age.secrets.<name>.identifier
A unique identifier for the secret, derived from the module and name. This may be used to name the secret.
Type: string (read only)
Example:
"host:noxa.wireguard.interfaces.some-interface::wg-interface-key"
Declared by:
age.secrets.<name>.module
The owning module of that secret.
Typically this is the name of module declaring the secret, e.g. “noxa.wireguard.interfaces.<name>”.
Type: string
Example:
"services.openssh"
Declared by:
noxa.secrets.enable
Enables the secrets module, multi-host secret management.
Type: boolean
Default:
true
Declared by:
noxa.secrets.def
A list of secrets that are managed by the noxa secrets module.
Each secret is either a host specific secret or a shared secret. Host specific secrets are only available on the host that owns them, while shared secrets are available on all hosts that declare them.
The options provided will be passed to the agenix module, by using the identifier as the name of the secret.
The identifier is derived from the module and name of the secret, e.g.
“host:noxa.wireguard.interfaces.some-interface::wg-interface-key” or “shared:noxa.wireguard.interfaces.some-interface:host1,host2:wg-preshared-connection-key”.
Type: list of (submodule)
Default:
[ ]
Declared by:
noxa.secrets.def.*.generator.dependencies
Other secrets on which this secret depends. See agenix-rekey documentation.
Type: null or (list of unspecified value) or attribute set of unspecified value
Default:
null
Example:
[ config.age.secrets.basicAuthPw1 nixosConfigurations.machine2.config.age.secrets.basicAuthPw ]
Declared by:
noxa.secrets.def.*.generator.script
Generator script, see agenix-rekey documentation.
Type: null or string or function that evaluates to a(n) string
Default:
null
Declared by:
noxa.secrets.def.*.generator.tags
Optional list of tags that may be used to refer to secrets that use this generator.
See agenix-rekey documentation for more information.
Type: null or (list of string)
Default:
null
Example:
[
"wireguard"
]
Declared by:
noxa.secrets.def.*.hosts
The hosts that have access to this secret.
Type: unique list of string
Default:
[
"<noxa-host-id>"
]
Example:
[
"host1"
"host2"
]
Declared by:
noxa.secrets.def.*.ident
The name of the secret.
This is the name of the secret, e.g. “wg-interface-key”.
Type: string
Example:
"wg-interface-key"
Declared by:
noxa.secrets.def.*.identifier
A unique identifier for the secret, derived from the module and name. This may be used to name the secret.
Type: string (read only)
Example:
"host:noxa.wireguard.interfaces.some-interface::wg-interface-key"
Declared by:
noxa.secrets.def.*.module
The owning module of that secret.
Typically this is the name of module declaring the secret, e.g. “noxa.wireguard.interfaces.<name>”.
Type: string
Example:
"services.openssh"
Declared by:
noxa.secrets.def.*.rekeyFile
The path to the rekey file for this secret. This is used by the agenix-rekey module to rekey the secret.
Type: absolute path (read only)
Declared by:
noxa.secrets.hostSecretsPath
The path where host secrets are stored. This is the path where noxa will look for (encrypted) host specific secrets.
This directory contains encrypted secrets for each host. Secrets in this directory are host specific, at least the secret part of the secret is owned by a single host and only published to that host.
An example secret would be the private wireguard key for an interface. Still the public key might be shared with other hosts.
ATTENTION: Since this path is copied to the nix store, it must not contain any secrets that are not encrypted.
Type: absolute path
Declared by:
noxa.secrets.options.enable
Enables the ‘simple’ options, by providing settings proxy, a user can set the options, inside the noxa.secrets.options module
that will provide sensible defaults for the agenix and agenix-rekey module.
If this is set to false, the user must set-up the agenix and agenix-rekey modules manually.
Type: boolean
Default:
true
Declared by:
noxa.secrets.options.hostPubkey
The public key of the host that is used to encrypt the secrets for this host.
Type: null or string
Default:
null
Declared by:
noxa.secrets.options.masterIdentities
A list of identities that are used to decrypt encrypted secrets for rekeying.
Type: list of (submodule)
Declared by:
noxa.secrets.options.masterIdentities.*.identity
The identity that is used to encrypt and store secrets as .age files. This must be an absolute path, given as string to not publish keys to the nix store.
This is the private key file used.
Type: null or string
Declared by:
noxa.secrets.options.masterIdentities.*.pubkey
The identity that is used to encrypt and store secrets as .age files. This is the age public key of the identity, used to encrypt the secrets.
This is the public key file used.
Type: null or string
Declared by:
noxa.secrets.options.rekeyDirectory
The directory where the rekey files are stored. This is used by the agenix-rekey module to rekey the secrets.
This directory must be writable by the user that runs the agenix-rekey module and added to
the git repo.
It is recommended to use $\{noxaHost} to create a unique directory for each host.
Type: absolute path
Declared by:
noxa.secrets.secretsPath
The path where all secrets are stored. Subfolders are created for host specific and shared secrets.
Type: null or absolute path
Declared by:
noxa.secrets.sharedSecretsPath
The path where secrets shared between several hosts are stored. This is the path where noxa will look for (encrypted) shared secrets.
This directory contains encrypted secrets that are shared between several hosts. Secrets in this directory are not host specific, they are not owned by a single host, but an group of hosts.
An example secret would be the pre-shared symmetric key for a wireguard interface peer.
Since this path is used by multiple hosts, it is recommended to set this path once for all hosts, instead of setting it per host.
ATTENTION: Since this path is copied to the nix store, it must not contain any secrets that are not encrypted.
Type: absolute path
Declared by:
noxa.sshHostKeys.generate
Generates SSH host keys on boot even if the openssh service is not enabled.
Type: boolean
Default:
false
Declared by:
noxa.sshHostKeys.hostKeysPrivate
List of SSH private host keys, accessible during runtime.
Type: list of string (read only)
Declared by:
noxa.sshHostKeys.impermanencePathOverride
Override the storage location for the ssh keys. Since some modules, like the noxa.secrets module,
depend on the keys being stored on a mounted disk during configuration activation, and not
expose functionality of systemd orderings, this option can be used to override the
storage location of the keys; useful when using impermanence setups.
Type: null or string
Default:
null
Declared by:
noxa.wireguard.enable
Enables the WireGuard module, which a cross-host VPN setup utility for wireguard.
Type: boolean
Default:
false
Declared by:
noxa.wireguard.interfaces
A set of WireGuard interfaces to configure. Each interface is defined by its name and contains its private key, public key, and listen port.
Type: lazy attribute set of (submodule)
Default:
{ }
Declared by:
noxa.wireguard.interfaces.<name>.advertise.keepAlive
The keep alive interval remote peers should use when communicating with this interface.
Type: null or signed integer
Default:
null
Declared by:
noxa.wireguard.interfaces.<name>.advertise.server
Options for wireguard servers. If a wireguard interface is regarded as a server (e.g. since it has a public IP address), it may advertise its service via the server.advertise option.
If set, all peers that would like to connect to that peer will use the advertised listen port and address as means of directly connecting to the server.
Further, if server.defaultGateway is set, all peers that do not advertise listen port and address will be reached via the server marked as default gateway. Therefore, only one interface may be marked as default gateway at any time.
Type: null or (submodule)
Default:
null
Declared by:
noxa.wireguard.interfaces.<name>.advertise.server.defaultGateway
If set, this server will be the default gateway for clients.
Type: boolean
Default:
false
Declared by:
noxa.wireguard.interfaces.<name>.advertise.server.firewallAllow
If set, the nixos firewall will allow incoming connections to the advertised listen port.
Type: boolean
Default:
true
Declared by:
noxa.wireguard.interfaces.<name>.advertise.server.listenAddress
The address this server will listen on for incoming connections.
Type: IPv4 address or IPv6 address
Declared by:
noxa.wireguard.interfaces.<name>.advertise.server.listenPort
The port this server will listen on for incoming connections.
Type: signed integer
Declared by:
noxa.wireguard.interfaces.<name>.autostart
Specifies whether to autostart the WireGuard interface.
Only relevant if the backend is set to wg-quick.
Type: boolean
Default:
true
Declared by:
noxa.wireguard.interfaces.<name>.backend
The backend to use for WireGuard config generation.
wireguard: Uses thenetworking.wireguard.interfacesmodule to generate the configuration.wg-quick: Uses thenetworking.wg-quick.interfacesmodule to generate the configuration.
Type: one of “wireguard”, “wg-quick”
Default:
"wireguard"
Declared by:
noxa.wireguard.interfaces.<name>.deviceAddresses
List of ip addresses to assign to this interface. The server will forward traffic to these addresses.
Type: list of (IPv4 address or IPv6 address)
Declared by:
noxa.wireguard.interfaces.<name>.gatewayOverride
If set, this interface will use the specified hostname as the gateway for connecting to the wireguard network.
Type: null or string
Default:
null
Declared by:
noxa.wireguard.interfaces.<name>.keepAlive
The default keep alive interval this interface will use when communicating with remote peers.
If the remote end uses advertise.keepAlive, the minimum value of both will be used.
Type: null or signed integer
Default:
null
Declared by:
noxa.wireguard.interfaces.<name>.kind.isClient
This interface has the role of a client, meaning it does not advertise other peers to connect to it. Instead, it connects to other peers, initiating the connection.
Type: boolean (read only)
Default:
true
Declared by:
noxa.wireguard.interfaces.<name>.kind.isGateway
This interface is a server and is marked as the default gateway for clients. This means that clients will use this interface to reach other peers that do not advertise their listen port and address.
Type: boolean (read only)
Default:
false
Declared by:
noxa.wireguard.interfaces.<name>.kind.isServer
This interface has the role of a server, meaning it advertises its listen port and address to peers.
Type: boolean (read only)
Default:
false
Declared by:
noxa.wireguard.interfaces.<name>.networkAddress
The network IP addresses. On clients, traffic of this network will be routed through the WireGuard interface.
Type: IPv4 address, normalized network address, or (IPv6 address, normalized network address)
Declared by:
noxa.wireguard.routes
A set of intermediary connection information, automatically computed from the nixos configurations.
Type: lazy attribute set of (submodule) (read only)
Declared by:
noxa.wireguard.routes.<name>.neighbors
A set of connections for this interface, automatically computed from the nixos configurations.
Type: lazy attribute set of (submodule) (read only)
Declared by:
noxa.wireguard.routes.<name>.neighbors.<name>.keepAlive
The keep-alive interval for this connection, in seconds.
If set to null, no keep-alive is configured.
Type: null or signed integer (read only)
Declared by:
noxa.wireguard.routes.<name>.participants.clients
A list of clients for this interface. Automatically populated.
Type: list of string (read only)
Declared by:
noxa.wireguard.routes.<name>.participants.gateways
A list of gateways for this interface. Automatically populated.
Type: list of string (read only)
Declared by:
noxa.wireguard.routes.<name>.participants.servers
A list of servers for this interface. Automatically populated.
Type: list of string (read only)
Declared by:
noxa.wireguard.routes.<name>.peers
A list of peers for this interface. Automatically populated.
Type: list of (submodule) (read only)
Declared by:
noxa.wireguard.routes.<name>.peers.*.target
The target hostname of the connection.
Type: string (read only)
Declared by:
noxa.wireguard.routes.<name>.peers.*.via
The hostname of the peer this connection is routed through.
Type: null or string (read only)
Declared by:
noxa.wireguard.secrets
A set of wireguard secrets. When using the .interfaces options,
this set is automatically populated. Each peer will own its own
set of secrets
Type: lazy attribute set of (submodule)
Default:
{ }
Declared by:
noxa.wireguard.secrets.<name>.presharedKeyFiles
The pre-shared key file for each peer (by hostname) of the wireguard interface
Type: lazy attribute set of string (read only)
Declared by:
noxa.wireguard.secrets.<name>.privateKeyFile
The private key file of the wireguard interface
Type: string (read only)
Declared by:
noxa.wireguard.secrets.<name>.publicKey
The public key the wireguard interface
Type: string (read only)
Declared by:
defaults
Default options applied to all nodes.
Type: submodule
Default:
{ }
Declared by:
defaults.build.toplevel
Build this node’s configuration into a NixOS system package.
Alias to config.system.build.toplevel.
Type: package (read only)
Declared by:
defaults.build.vm
Build this node’s configuration into a VM testing package.
Alias to config.system.build.vm.
Type: package (read only)
Declared by:
defaults.nixpkgs
The nixpkgs version to use when building this node.
By default, if not explicitly set, it uses the same version than the Noxa flake itself.
Type: absolute path
Default:
"<nixpkgs>"
Declared by:
defaults.reachable.allowHostConfiguration
Allow the host to configure its own reachable addresses. If set to false, values can only be set on the Noxa module level.
Type: boolean
Default:
false
Declared by:
defaults.reachable.internet
List of external IP addresses this host is reachable at via using a public IP address.
Type: list of (IPv4 address or IPv6 address)
Default:
[ ]
Declared by:
defaults.reachable.wireguardNetwork
List of IP addresses this host is reachable at via WireGuard (specified via name).
Type: attribute set of list of (IPv4 address or IPv6 address)
Default:
{ }
Declared by:
defaults.specialArgs
Special arguments passed to the host modules.
Type: attribute set of anything
Default:
{ }
Declared by:
nodeNames
A list of node names managed by Noxa. Due to the architecture of Noxa, noxa modules might unwillingly create new nodes, this list contains the name of all nodes that are currently managed by Noxa. Noxa modules can check this list to see if a node was created by themselves.
The user must set this to the listOf all nodes they want to manage, otherwise if you
don't care, set this to `attrNames config.nodes`.
Type: list of string
Default:
[ ]
Declared by:
nodes
A set of nixos hosts managed by Noxa.
Type: attribute set of (submodule)
Default:
{ }
Declared by:
nodes.<name>.build.toplevel
Build this node’s configuration into a NixOS system package.
Alias to config.system.build.toplevel.
Type: package (read only)
Declared by:
nodes.<name>.build.vm
Build this node’s configuration into a VM testing package.
Alias to config.system.build.vm.
Type: package (read only)
Declared by:
nodes.<name>.nixpkgs
The nixpkgs version to use when building this node.
By default, if not explicitly set, it uses the same version than the Noxa flake itself.
Type: absolute path
Default:
"<nixpkgs>"
Declared by:
nodes.<name>.reachable.allowHostConfiguration
Allow the host to configure its own reachable addresses. If set to false, values can only be set on the Noxa module level.
Type: boolean
Default:
false
Declared by:
nodes.<name>.reachable.internet
List of external IP addresses this host is reachable at via using a public IP address.
Type: list of (IPv4 address or IPv6 address)
Default:
[ ]
Declared by:
nodes.<name>.reachable.wireguardNetwork
List of IP addresses this host is reachable at via WireGuard (specified via name).
Type: attribute set of list of (IPv4 address or IPv6 address)
Default:
{ }
Declared by:
nodes.<name>.specialArgs
Special arguments passed to the host modules.
Type: attribute set of anything
Default:
{ }
Declared by:
wireguard
Type: attribute set of (submodule)
Default:
{ }
Declared by:
wireguard.<name>.allowNodesToJoin
If set to true, nodes may join this network by declaring noxa.wireguard.interfaces.<interface>.
If set to false, nodes may only join this network if declared in this network configuration.
Type: boolean
Default:
false
Declared by:
wireguard.<name>.autoConfigureGateway
If set to true, the gateways for this network will be automatically configured based
on the nodes.<name>.reachable.internet attribute.
Type: boolean
Default:
true
Declared by:
wireguard.<name>.members
Configuration of the wireguard members.
Type: attribute set of (submodule)
Declared by:
wireguard.<name>.members.<name>.advertise.keepAlive
The keep alive interval remote peers should use when communicating with this interface.
If set to any other value than null, the value will be applied to the nodes configuration.
Type: null or signed integer
Default:
null
Declared by:
wireguard.<name>.members.<name>.advertise.server
Options for wireguard servers. If a wireguard interface is regarded as a server (e.g. since it has a public IP address), it may advertise its service via the server.advertise option.
If set, all peers that would like to connect to that peer will use the advertised listen port and address as means of directly connecting to the server.
Further, if server.defaultGateway is set, all peers that do not advertise listen port and address will be reached via the server marked as default gateway. Therefore, only one interface may be marked as default gateway at any time.
If set to any other value than null, the value will be applied to the node configuration.
Type: null or (submodule)
Default:
null
Declared by:
wireguard.<name>.members.<name>.advertise.server.defaultGateway
If set, this server will be the default gateway for clients.
Type: null or boolean
Default:
null
Declared by:
wireguard.<name>.members.<name>.advertise.server.firewallAllow
If set, the nixos firewall will allow incoming connections to the advertised listen port.
If set to any other value than null, the value will be applied to the node configuration.
Type: null or boolean
Default:
null
Declared by:
wireguard.<name>.members.<name>.advertise.server.listenAddress
The address this server will listen on for incoming connections.
Type: null or IPv4 address or IPv6 address
Default:
null
Declared by:
wireguard.<name>.members.<name>.advertise.server.listenPort
The port this server will listen on for incoming connections.
Type: null or signed integer
Default:
null
Declared by:
wireguard.<name>.members.<name>.autostart
Specifies whether to autostart the WireGuard interface.
Only relevant if the backend is set to wg-quick.
If set to any other value than null, the value will be applied to the node configuration.
Type: null or boolean
Default:
null
Declared by:
wireguard.<name>.members.<name>.backend
The backend to use for WireGuard config generation.
wireguard: Uses thenetworking.wireguard.interfacesmodule to generate the configuration.wg-quick: Uses thenetworking.wg-quick.interfacesmodule to generate the configuration.
If set to any other value than null, the value will be applied to the node configuration.
Type: null or one of “wireguard”, “wg-quick”
Default:
"wireguard"
Declared by:
wireguard.<name>.members.<name>.deviceAddresses
List of ip addresses to assign to this interface. The server will forward traffic to these addresses.
Type: null or ((list of (IPv4 address or IPv6 address)) or (IPv4 address or IPv6 address) convertible to it)
Default:
null
Declared by:
wireguard.<name>.members.<name>.gatewayOverride
If set, this interface will use the specified node as the gateway for connecting to the wireguard network.
If set to any other value than null, the value will be applied to the nodes configuration.
Type: null or string
Default:
null
Declared by:
wireguard.<name>.members.<name>.keepAlive
The default keep alive interval this interface will use when communicating with remote peers.
If the remote end uses advertise.keepAlive, the minimum value of both will be used.
If set to any other value than null, the value will be applied to the nodes configuration.
Type: null or signed integer
Default:
null
Declared by:
wireguard.<name>.members.<name>.onlySpecifiedDeviceAddresses
If set to true, device addresses set by the node configuration are rejected. If set to false, the node might add additional device addresses to the interface.
Type: boolean
Default:
true
Declared by:
wireguard.<name>.networkAddress
The network address of the wireguard network.
Type: IPv4 address, normalized network address, or (IPv6 address, normalized network address)
Declared by: