r00t2
/
aif-ng
1
0
Fork 0
Code Issues Pull Requests Projects Releases Activity
aif-ng/docs/MANUAL.adoc

49 KiB
Raw Blame History

AIF-NG User Manual

Table of Contents

Preface

About the Author

I am a GNU/Linux Systems/Network Administrator/Engineer — I wear a lot of hats. I have a lot of side projects to keep me busy when I’m not working at ${dayjob}, mostly to assist in other side projects and become more efficient and proficient at those tasks. “Shaving the yak,” [1] indeed.

I got frustrated at the lack of options for installing Arch from a network or automated deployment environment and decided I needed a tool to do that for me.

What is AIF-NG?

AIF-NG (Arch Installation Framework, Next Generation) is a means to automatically install Arch Linux. Think of it as something akin to Kickstart.

AIF (classic) was written entirely in bash, required compilation, wasn’t flexible enough, and is obsolete/no longer maintained. So I rewrote it in Python3 and give it a more basic yet flexible structure.

The client (aifclient.py) is a single script and gets its configuration from a combination of an XML file and kernel paramaters (which tell it where to find the former and how to access it).

AIF-NG is intended mainly for system administrators but if you find yourself turning up a lot of Arch Linux installations in other environments, you may find it useful.

What it’s Not

AIF-NG is not intended to be a complete turnup solution. Instead, it’s useful to build up from baremetal and configure a system to a point where you can use another management tool (such as Ansible, Chef, Puppet, SaltStack, and others).

Though if you’re really gung-ho about it, I suppose you could use the post-script feature to fully turn up a box.

It is also not a magic bullet. It will not make an Arch Linux installation easier, nor is it designed to do that. Don’t file bug reports for this. It’s designed to make it faster. I recommend you follow the manual installation process several times first so you’re comfortable with the process and understand what’s happening behind the scenes. (If you find it too hard to understand, you may be interested in Antergos instead.)

Copyright/Licensing

The AIF-NG code is GPLv3-licensed. This means that you can use it for business reasons, personal reasons, modify it, etc. Please be sure to familiarize yourself with the full set of terms. You can find the full license in docs/LICENSE.

GPLv3

This document, and all other associated author-generated documentation, are released under the Creative Commons CC-BY-SA 4.0 copyright. It’s essentially the GPL for non-software, so similar terms apply.

CC-BY-SA_4.0

Getting Started

Downloading

If it isn’t in your distro’s repositories (It is in Arch’s AUR! Both tagged release and git master.), you can still easily get rolling. Simply visit the project’s source code web interface and download a tarball under the Download column:

cgit

If you know the tag of the commit you want, you can use curl:

curl -sL -o aif.tar.xz https://git.square-r00t.net/AIF-NG/snapshot/AIF-NG-0.01-BETA.tar.xz

or wget:

wget -O aif.tar.xz https://git.square-r00t.net/AIF-NG/snapshot/AIF-NG-0.01-BETA.tar.xz

You can use https://git.square-r00t.net/AIF-NG/snapshot/AIF-NG-master.tar.xz for the URL if you want the latest working version. If you want a snapshot of a specific commit, you can use e.g. https://git.square-r00t.net/AIF-NG/snapshot/AIF-NG-0e3b4572f9bc460741fe5cd3108b22fad89bfc71.tar.xz and so on.

Alternatively, you can use git. Git most definitely should be in your distro’s repositories.

Tip
 If you’re new to git and want to learn more, I highly recommend the book Pro Git. It is available for free download (or online reading).

You can clone via https:

git clone https://git.square-r00t.net/AIF-NG

or native git protocol:

git clone git://git.square-r00t.net/aif-ng.git AIF-NG

The git protocol is much faster, but at a cost of lessened security.

Note
 I also have a mirror at GitHub, but I don’t like GitHub very much and since it’s a mirror repository, it’s possible it will be out of date. For this reason, it’s recommended that you use the resources above.

Prerequisites

This is a list of software you’ll need available to use the AIF-NG client.

Tip
 Your distro’s package manager should have most if not all of these available, so it’s unlikely you’ll need to install from source.
Note
 Some versions may be higher than actually needed.

Necessary

These are needed for using AIF-NG.

These are no required Python modules, at least for the client; it will work fine with just the standard library for Python 3.

Optional

While not strictly necessary, these will greatly enhance your AIF-NG usage. I’ve included some reasons why you might want to install them.

Python modules:

  • LXML

    • Recommended for more complete XML processing, the aifverify.py utility, etc.

Starting an Install

First, aifclient.py (/usr/bin/aifclient in AUR packages) must be configured to start at boot after networking has initiated in the host environment. This can be done very easily with a oneshot systemd unit file.

However, this will do nothing on its own. This is a security measure; you can very easily destroy the host’s installation if you attempt to run AIF-NG with an inappropriate configuration. For this reason, AIF-NG will exit if it is not enabled via the kernel commandline/boot parameters (mkinitcpio hooks may be provided in future updates to the AUR packages to assist in creating more lightweight install environments).

Configure your bootloader to add the following options as necessary:

Parameter Purpose

aif

This enables AIF-NG; without this, a run will never be initiated — note that aif and aif=True are the same, and it can be explicitly disabled by setting aif=False

aif_url

The URI to your XML configuration file (see below)

aif_auth

(see below)

aif_username

(see below)

aif_password

(see below)

Some notes on auth and URIs

  • aif_url can be an HTTP/HTTPS URL, an FTP/FTPS URI, or a file:// URI. e.g.:

    • aif_url=http://aif.square-r00t.net/aif.xml

    • aif_url=https://aif.square-r00t.net/aif.xml

    • aif_url=ftp://ftp.domain.tld/bootstrap/aif.xml

    • aif_url=ftps://secure.ftp.domain.tld/bootstrap/aif.xml

    • aif_url=file:///srv/aif/aif.xml

  • If aif_url is an HTTP/HTTPS URL, then aif_user is the username to use with the 401 (RFC 7235) auth (via aif_auth).

    • If aif_url is an FTP/FTPS URI, then aif_user will be the FTP user.

    • The same behavior applies for aif_password.

Building a compatible LiveCD

The default Arch install CD does not have AIF installed (hopefully, this will change someday). You have two options for using AIF-NG.

Recommended

The recommended option is to use BDisk (the author should look familiar ;) and per the documentation, you would simply create the following modifications (remember to replace <BDisk directory> with your actual BDisk directory):

  1. mkdir -p <BDisk directory>/overlay/etc/systemd/system/multi-target.wants

  2. ln -s /etc/systemd/system/aif.service <BDisk directory>/overlay/etc/systemd/system/multi-target.wants/aif.service

    1. (NOTE: This is not a typo; the symlink will resolve to the correct place during the build)

  3. printf '[Unit]\nDescription=AIF-NG Client Service\nAfter=livecdfix.service\n\n[Service]\nType=oneshot\nExecStart=/usr/bin/aif\n\n[Install]\nWantedBy=multi-user.target\n' > <BDisk directory>/overlay/etc/systemd/system/aif.service

    1. (NOTE: This is all one line.)

    2. (NOTE: We use a custom aif.service instead of the AUR package provided one because of how BDisk handles bringing up the network.)

  4. echo "aif-git" > <BDisk directory>/extra/pre-build.d/root/packages.both

  5. If you want automatic root login on TTY1 like the Arch install ISO (optional):

    1. mkdir -p <BDisk directory>/overlay/etc/systemd/system/getty\@tty1.service.d

    2. printf '[Service]\nType=idle\nExecStart=\nExecStart=-/usr/bin/agetty --autologin root --noclear %%I 38400 linux\n' > <BDisk directory>/overlay/etc/systemd/system/getty\@tty1.service.d/autologin.conf

      1. (NOTE: This is all one line.)

Remember to also create a build.ini file. You can find a compatible one here (but remember to tailor it to your particular paths and needs first!).

Make any further customizations as you wish, then start the build.

Quickest

For convenience, I’ve already built a LiveCD that will auto-start AIF. Note, however, that it is configured to my personal preferences (it installs python3, installs apacman (and configures it and pacman to my tastes), sets up a more strict SSH configuration, and installs my SSH pubkeys.), so you may want to use the recommended method above instead.

The full environment

A full ISO build is here (GPG signatures are available in SIG and ASC format; make sure you verify it).

It has a full GNU/Linux environment that you can use, and works on both UEFI and BIOS systems. It boots to a non-passworded root login, but AIF will be running in the background. SSH is installed and configured for key-based authentication only, but is not enabled by default.

The iPXE environment

If you would like to boot over the network, I have an iPXE ISO here (GPG signatures are available in SIG and ASC format; make sure you verify it).

You will need at least 2GB of RAM, as it loads entirely into memory.

It also boots to a full GNU/Linux environment that you can use, and works on both UEFI and BIOS systems. It boots to a non-passworded root login, but AIF will be running in the background. SSH is installed and configured for key-based authentication only, but is not enabled by default.

Logging

Currently, only one method of logging is enabled, and is always enabled. It can be found on the host and guest at /root/aif.log.<UNIX epoch timestamp>. Note that after the build finishes successfully, it will remove the host’s log (as it’s just a broken symlink at that point). You will be able to find the full log in the guest after the install, however.

Debugging

Sometimes it’s useful to get a little more information, or to start an installation from within an already-booted environment and you didn’t remember (or weren’t able to) change the kernel parameters. If this is the case, simply export the DEBUG environment variable (it can be set to anything, it doesn’t matter) — if this is done, the arguments will be read from /tmp/cmdline instead. e.g.:

export DEBUG=true
cp /proc/cmdline /tmp/.
chmod 600 /tmp/cmdline
sed -i -e '1s/$/ aif aif_url=https:\/\/aif.square-r00t.net\/aif.xml/' /tmp/cmdline

It will also write the full configuration (after parsing) to the logfile.

Writing an XML Configuration File

I’ve included a sample aif.xml file with the project which is fully functional. However, it’s not ideal — namely because it will add my personal SSH pubkeys to your new install, and you probably don’t want that. However, it’s fairly complete so it should serve as a good example. If you want to see the full set of supported configuration elements, take a look at the most up-to-date aif.xsd. For explanation’s sake, however, we’ll go through it here. The directives are referred to in XPath syntax within the documentation text for easier context (but not the titles).

<aif>

The /aif element is the root element. It serves as a container for all the configuration data. The only attributes it contains are for formatting and verification of the containing XML.

<storage>

The /aif/storage element contains disk, disk/part, and mount elements.

<disk>

The /aif/storage/disk element holds information about disks on the system, and within this element are one (or more) part elements. Note that any disk elements specified here will be entirely reformatted; operate under the assumption that ANY and ALL pre-existing data on the specified device will be IRREVOCABLY LOST.

Attribute Value

device

The disk to format (e.g. /dev/sda)

diskfmt

gpt or msdos
<part>

The /aif/storage/disk/part element holds information on partitioning that it’s parent disk element should have.

Attribute Value

num

The partition number (positive integer)

start

The amount of the total disk size to start the partition at (see below)

stop

The amount of the total disk size to end the partition at (see below)

fstype

The partition type. Must be in gdisk format (see below)

The start and stop attributes can be in the form of:

  • A percentage of the total disk size, indicated by a percentage sign ("10%")

  • A size, indicated by the abbreviation ("300KiB", "10GB", etc.)

  • A raw sector size, if no suffix is provided (sector sizes are typically 512 bytes but this can vary depending on disk) (1024)

  • One can also specify modifiers ("+10%", "-400MB", etc.). A positive modifier indicates from the beginning of the start of the disk and a negative modifier specifies from the end of the disk (the default, if none is specified, is to use the previously defined partition’s end as the start for the new partition, or to use the beginning of the usable disk space as the start if no previous partition is specified, and to add the size to the start until the stop is reached)

Note
 The following is a table for your reference of partition types. Note that it may be out of date, so reference the link above for the most up-to-date table.
fstype Formatting type

0700

Microsoft basic data

0c01

Microsoft reserved

2700

Windows RE

3000

ONIE config

3900

Plan 9

4100

PowerPC PReP boot

4200

Windows LDM data

4201

Windows LDM metadata

4202

Windows Storage Spaces

7501

IBM GPFS

7f00

ChromeOS kernel

7f01

ChromeOS root

7f02

ChromeOS reserved

8200

Linux swap

8300

Linux filesystem

8301

Linux reserved

8302

Linux /home

8303

Linux x86 root (/)

8304

Linux x86-64 root (/

8305

Linux ARM64 root (/)

8306

Linux /srv

8307

Linux ARM32 root (/)

8400

Intel Rapid Start

8e00

Linux LVM

a500

FreeBSD disklabel

a501

FreeBSD boot

a502

FreeBSD swap

a503

FreeBSD UFS

a504

FreeBSD ZFS

a505

FreeBSD Vinum/RAID

a580

Midnight BSD data

a581

Midnight BSD boot

a582

Midnight BSD swap

a583

Midnight BSD UFS

a584

Midnight BSD ZFS

a585

Midnight BSD Vinum

a600

OpenBSD disklabel

a800

Apple UFS

a901

NetBSD swap

a902

NetBSD FFS

a903

NetBSD LFS

a904

NetBSD concatenated

a905

NetBSD encrypted

a906

NetBSD RAID

ab00

Recovery HD

af00

Apple HFS/HFS+

af01

Apple RAID

af02

Apple RAID offline

af03

Apple label

af04

AppleTV recovery

af05

Apple Core Storage

bc00

Acronis Secure Zone

be00

Solaris boot

bf00

Solaris root

bf01

Solaris /usr & Mac ZFS

bf02

Solaris swap

bf03

Solaris backup

bf04

Solaris /var

bf05

Solaris /home

bf06

Solaris alternate sector

bf07

Solaris Reserved 1

bf08

Solaris Reserved 2

bf09

Solaris Reserved 3

bf0a

Solaris Reserved 4

bf0b

Solaris Reserved 5

c001

HP-UX data

c002

HP-UX service

ea00

Freedesktop $BOOT

eb00

Haiku BFS

ed00

Sony system partition

ed01

Lenovo system partition

ef00

EFI System

ef01

MBR partition scheme

ef02

BIOS boot partition

f800

Ceph OSD

f801

Ceph dm-crypt OSD

f802

Ceph journal

f803

Ceph dm-crypt journal

f804

Ceph disk in creation

f805

Ceph dm-crypt disk in creation

fb00

VMWare VMFS

fb01

VMWare reserved

fc00

VMWare kcore crash protection

fd00

Linux RAID
Note
 Automatic formatting is currently only enabled for the following (subject to further configuration in later versions):
fstype Formatted as

ef00

vFAT32 (mkfs.vfat -F 32)

ef01

"

ef02

"

8200

GNU/Linux swap (mkswap)

8300

ext4

8301

"

8302

"

8303

"

8304

"

8305

"

8306

"

8307

"

<mount>

The /aif/storage/mount element specifies mountpoints for each disk's partition.

Attribute Value

source

The device to mount

target

Where it should be mounted to in the filesystem (on the host system, not the new installation); if swap, it will be handled as swapspace instead

order

The order in which it should be mounted. These should be unique positive integers.

fstype

The filesystem type; usually this is not required but if you need to manually specify the type of filesystem, this will allow you to do it

opts

The mount options; provide the string exactly as it would be provided to mount(8)'s -o option

<network>

The /aif/network element specifies network configuration(s). It contains iface ("interface") elements.

Attribute Value

hostname

The hostname of the new installation

<iface>

The /aif/network/iface element specifies various network configurations. Currently only ethernet is supported, and only limited support for IPv6 is available (but future improvements/flexible capabilities are planned).

Attribute Value

device

The interface name (in Predictable Interface Naming) (e.g. ens3); can be auto (see below)

address

The address to be assigned to the interface (in CIDR format); can be auto (see below)

netproto

One of ipv4, ipv6, or both

gateway

The gateway address for the interface/protocol pairing; only used if address is not auto

resolvers

The DNS resolver addresses, if you wish/need to manually specify them; pass as a comma-separated list

If "auto" is specified for device, the system will configure the first (and only the first) interface it finds with an active link with the provided address information.

If "auto" is specified for address, then DHCP (or DHCPv6, depending on the configuration of netproto).

Note
 Setting netproto to "both" is really only useful if "auto" is specified for address.

<system>

The /aif/system element is for handling general system configuration. It contains the users, users/user, users/user/home, users/user/xgroup, and service elements.

Attribute Value

timezone

The timezone for the installed system (can be independent of the host system)

locale

The locale of the installed system (e.g. en_US.UTF-8); if a short version is used (e.g. en), then all locales starting with that prefix will be enabled (multiple explicit locale support is in the TODO)

chrootpath

The path on the host that will serve as the chroot path. This should be where your new install’s / (root filesystem partition) is mounted at in mounts

kbd

The keyboard layout (if not US)

reboot

If we should reboot the system after the install (in order to boot to the newly-installed system, assuming your boot order is set correctly). Boolean, accepts 1/true or 0/false.

<users>

The /aif/system/users element is used to specify users you wish to create (if any). It contains the user, user/home, and user/xgroup elements.

Attribute Value

rootpass

A properly hashed-and-salted password. See below
Note
 To generate a proper hashed/salted password, you may want to reference this section from BDisk's user manual (another project of mine). You can use this python script to generate one. If you specify an empty string, the password will be BLANK (i.e. you can log in with just the username). This is very insecure. If you specify a ! instead of a salted hash, TTY login will be disabled (though it will still be possible to log in via other means such as SSH pubkey auth — assuming you configure it beforehand. This has some added security benefits).
<user>

The /aif/system/users/user element specifies user(s) to create. It contains xgroup and home elements.

Attribute Value

name

The username/login name

sudo

If (full) sudo access should be granted to this user (boolean; must be one of 1/true or 0/false)

password

The salted/hashed password (see above)

comment

A comment (typically, the user’s real/full name)

uid

The UID of the user; if specified, must be a positive integer

group

The primary group of the user (the default is to create a new group named after that user)

gid

The GID to use for the primary group; must be a positive integer
<xgroup>

The /aif/system/users/user/xgroup elements specifies one (or more) "eXtra groups" (i.e. non-primary) that AIF-NG should add the user to.

Attribute Value

name

The group name

create

If the group should be created (boolean; must be one of 1/true or 0/false)

gid

The GID to use (if creating); must be a positive integer and not be taken by an existing group
<home>

The /aif/system/users/user/home element contains information for a user's home directory. It can be only specified once per user, but it is optional.

Attribute Value

path

The path for the home directory; useful if you don’t want it to be /home/<username>

create

If the home directory should be created (boolean; must be one of 1/true or 0/false)

<service>

The /aif/system/service element holds information about services that should explicitly be enabled/disabled on boot.

Attribute Value

name

The service name. It can be shortform (sshd) or long form (git-daemon.socket); if the shortform is provided, ".service" is assumed

status

A boolean that specifies if the service should be enabled (1/true) or disabled (0/false)

<pacman>

The /aif/pacman element contains the repos, repos/repo, mirrorlist, mirrorlist/mirror, software, and software/packages elements.

Attribute Value

command

The command to use to install a package

If you configured an alternate package utility (using a execution="pkg" script entry), you can specify the command here. Note that it should be configured/called with necessary options to avoid the necessity of user involvement (since that’s the entire point of AIF-NG). e.g.:

<aif ... >
  ...
    <pacman command="apacman --needed --noconfirm --noedit --skipinteg -S">
  ...
</aif>

<repos>

The /aif/pacman/repos element contains one (or more) repo element(s).

<repo>

The /aif/pacman/repos/repo elements specify information for configuring the installed system’s /etc/pacman.conf (specifically, the repositories).

Attribute Value

name

The name of the repository

enabled

A boolean that specifies if the repository should be enabled (1/true) or disabled (0/false)

siglevel

The siglevel of the repository (e.g. Optional TrustedOnly); can be default (in which the pacman.conf default siglevel will be used)

mirror

The URI for the mirror; if it begins with file://, we will use it as an Include = instead of a Server = (make sure it is a full/absolute path and it exists on the newly installed system)
<mirrorlist>

The /aif/pacman/mirrorlist element contains elements that should be in /etc/pacman.d/mirrorlist. It is optional; if it isn’t specified, the default distributed mirrorlist will be used instead.

<mirror>

The /aif/pacman/mirrorlist/mirror elements are mirrorlist entries.

<software>

The /aif/pacman/software element contains one (or more) package element(s) that describe software to install. It is optional.

<package>

The /aif/pacman/software/package element holds information about software to be installed.

Attribute Value

name

The name of the package (e.g. openssh)

repo

Optional, but you can specify which repository to install the package from (in the case of multiple repositories providing the same package)

<bootloader>

The /aif/bootloader element specifies a bootloader to install.

Attribute Value

type

The bootloader to use; currently, the only supported values are grub and systemd (for systemd-boot) but more options may be available in the future

efi

If used for (U)EFI support; note that the install environment must be booted in UEFI mode and that systemd(-boot) only supports EFI and that it is a boolean (1/true or 0/false)

target

This should be the absolute path (from within the newly installed system) to your ESP (if efi is true); otherwise the disk/partition to install the bootloader to (if you’re using BIOS mode)

<scripts>

The /aif/scripts element contains one or more script elements.

<script>

The /aif/scripts/script elements specify scripts to be run at different stages during the install process. This is useful if you need to set up SSH pubkey authentication, for example, or configure mdadm so you can use that as a disk.

Attribute Value

uri

The URI to the script; can be an HTTP/HTTPS reference, an FTP/FTPS reference, or a local file reference (file:///path/to/file).

order

A unique positive integer used to order the scripts during the run; note that e.g. pre- and post-scripts are executed at different points, so you can use the same order as long as it’s in different execution points

authtype

Same behavior as aif_auth but for fetching this script (see also further notes on this)

user

Same behavior as aif_user but for fetching this script (see also further notes on this)

password

Same behavior as aif_password but for fetching this script (see also further notes on this)

execution

(see below)

There are several script types availabe for execution. Currently, these are:

  • pre

  • pkg

  • post

pre scripts are run (in specified order) before the disks are even formatted. pkg scripts are run (in specified order) right before the packages are installed (this allows you to configure an alternate packager such as apacman) — these are run inside the chroot of the new install. post scripts are run inside the chroot like pkg, but are executed very last thing, just before the reboot.

Further Information

Here you will find further info and other resources relating to AIF-NG.

FAQ

"Eww, why XML?"

Because it’s the superior format for this:

  • It supports in-spec validation of data values and data types, formatting of data levels, required data objects and at certain occurrence levels, etc. (unlike JSON, YAML, INI, etc.). Both in and out of channel.

    • This means it’s MUCH easier for code/language/project/etc.-agnostic software to create, generate, and validate a configuration profile.

  • It supports inclusion via XInclude, letting you standardize your configuration snippets across multiple configuration profiles (unlike JSON, YAML, INI, etc.).

  • It supports sane nesting (unlike INI).

  • It supports attributes to data objects (unlike JSON, YAML, INI, etc.).

  • While certainly not used as extensively as it could be in this particular project, it supports namespacing — and referential namespacing at that, providing a URI to get more info about a certain namespace. JSON, YAML, INI, etc. all do not.

  • It is not whitespace-sensitive to denote significance/levels of objects (unlike YAML and, in some cases, INI). This allows for whitespace compression (commonly referred to as "minifying") while still being able to completely retain whitespace inside data’s content.

    • And as a result, it requires MUCH less escaping and post-parsing cleanup like e.g. JSON and YAML do.

  • and so on.

Trust me. XML is superior, especially when needing to represent something as complex as an entire OS install. Sorry not sorry to all the bigmad webdevs and DevOps-y people out there. JSON and YAML actually do suck.

"How do I make AIF-NG operate entirely offline?"

This is cooked right in, but takes a little extra work.

1.) First you’ll need to locally clone the supporting XSD (XML schemas) that AIF-NG uses to verify the configuration file:

/var/tmp/aif/xml

mkdir -p /var/tmp/aif
cd /var/tmp/aif
git clone https://git.square-r00t.net/XML xml

The path you clone it to isn’t important as long as you’re consistent below.

2.) Then edit your AIF-NG configuration file to source this directory for XML verification:

aif.xml (before)

     xsi:schemaLocation="https://aif-ng.io/ http://schema.xml.r00t2.io/projects/aif.xsd"

aif.xml (after)

     xsi:schemaLocation="https://aif-ng.io/ file:///var/tmp/aif/xml/schema/projects/aif.xsd"

The XSD files use includes with relative paths, so the rest of that is automagic.

3.) Use local file:// URIs in the rest of your AIF-NG configuration file. e.g.:

<tarball>file:///var/tmp/aif/bootstrap.tar.gz</tarball>

and

<signatureFile>file:///var/tmp/aif/bootstrap.tar.gz.sig</signatureFile>

etc.

Obviously you need to download those files to their respective destinations first, however.

4.) Lastly, ensure you only use local pacman mirrors in your config. This gets tricky because the chroot will not have a way to access the hosts filesystem without creating e.g. a bind mount beforehand.

As long as:

  • No remote locations are specified in your AIF-NG configuration file…​

  • and it is completely and well defined…​

  • and your scripts don’t make remote calls,

then it shouldn’t try to perform any remote operations.

Note that if you specified a GPG verification, you’ll need to use a local exported key file for the public key (keyFile); if you use a keyID, then AIF-NG will try to fetch the key from keyservers.

"I specified start sector as 0 for a GPT-labeled disk but it starts at sector 2048 instead. What gives?"

GPT requires 33 sectors for the table at the beginning (and 32 sectors at the end) for the actual table. That plus an extra (usually) 512 bytes at the beginning for something called a Protective MBR (this prevents disk utilities from overwriting the GPT label automatically in case they only recognize "msdos" labels and assume the disk is not formatted yet).

Most disks these days use something called Advanced Format. These align their sectors to factors of 8, so sector 34 can’t be used - it’d have to be sector 40. Additionally, various other low-level disk interactions (e.g. RAID stripe sizes) require a much larger boundary between partitions. If you’re interested in a little more detail, you may find this interesting (specifically this section, paragraph starting with You may also ask why the first partition starts from LBA 2048…​).

TL;DR: "It’s the safest way to make sure your disk doesn’t suffer massive degradation in performance, your RAID doesn’t eat partitions, etc." Don’t worry, it typically only ends up being about 1MB of "wasted" space surrounding partitions. I’ve written plaintext documentation larger than 1MB.

"Why do partitions take start/stop attributes but LVs take size?"

Using start/stop attributes makes sense for disk partitions because they operate on actual geometry (positions on-disk); that is, this lets you create a "gap" between partitions on the disk which can be helpful if you want to do any modifications to the partition table afterwards (this is also why partitions are processed in the order they’re specified).

LVM (LVs, in particular), however, aren’t consecutive. There is no concept of a "start" and "stop" for an LV; LVM uses chunks called "(physical) extents" rather than sectors, and VGs don’t have geometry since they’re essentially a pool of blocks. This is also why the modifiers like - and + aren’t allowed for LV sizes - they’re position-based.

"How can I use a whole disk as an MDADM member?"

TL;DR: You don’t. You just don’t.

The long-winded answer: it’s a terrible idea. I’m not here to criticize how you want to structure your install, but I’m definitely going to try to prevent some dumb mistakes from being made. This is one of them.

It can cause a whole slew of issues:, including but not limited to:

  • Inflexible disk replacement. Disk geometry (low-level formatting, etc.) can vary wildly across vendors and models. When you have to replace a disk in your degraded RAID array, you’re going to be in for a nasty surprise (loss of performance, incompatible size, etc.) when one vendor aligned their e.g. 1TB disk to 512 blocks and the other to 128 blocks (because there are some dumb vendors out there). If you try to replace a disk in a RAID-1 with mismatched size, even by a couple blocks, you’re gonna have a bad time.

  • Your motherboard may arbitrarily wipe out the RAID superblocks. (source) source source

  • It can cause some weird issues with e.g. LVM on top of the array. source source

  • You can’t put a bootloader or EFI System Partition on the disk.

"How do I specify packages from the AUR?"

You’d have to build the package(s), set up a repository, serve it via e.g. nginx, and add it as a repo (/aif/pacman/repos/repo) first. Then you can specify the package as normal as a /aif/pacman/software/package item.

"Why can’t the network settings in <network> be applied during install?"

Simply put, a logical race condition. In order for probably 90+% of AIF-NG deploys to bootstrap, they fetch their XML configuration via a network URI (rather than a file URI). This means it needs a network connection that pre-exists in the install environment (LiveCD, LiveUSB, PXE/iPXE, etc.) before it even knows what network configuration you want the persistent environment to have.

Granted, this is a moot point if you’re using a file:// URI for the XML configuration, but this is not a very flexible means regardless. The installation host itself is outside the scope of AIF-NG.

If you desire the configuration to be applied during the install, you can do it yourself in an /aif/scripts/pre/script or /aif/scripts/pkg/script script. The fetched XML file can be found at /var/tmp/AIF.xml in the install environment.

If you wish to SSH into the install environment to check the status/progress of the install, it is recommended that you set up a static lease (if using DHCP) or use SLAAC (if using IPv6) beforehand and configure your install environment beforehand. Remember, AIF-NG only installs Arch Linux; it tries very hard to not interact with the install environment.

"Why isn’t enabling/disabling automatic DNS resolvers/routes/addresses working?"

This is going to be highly unpredictable based on the networking provider you choose. This is a limitation of underlying network provider intercompatibility, resolver libraries, there being no way to tell DHCP/DHCP6/SLAAC clients to only fetch information about a network and not assign a lease, and technology architecture. This may be changed in the future, but because of how DNS servers are handled via DHCP/RDNSS and glibc (and the fact that IPv4 resolver addresses can serve IPv6 — e.g. AAAA — records and vice versa) and inherent limitations in some network providers like netctl, I wouldn’t hold your breath.

"I’m using netctl as my network provider, and-"

I’ma let you finish, but netctl is a really simple network provider. I mean REALLY simple. As such, a lot of things don’t work at all feasibly, and probably might not ever. It’s great for simple and flat configurations (i.e. all static everything, all automatic everything, etc.) and I even use it on my own machines where I can, but it just simply doesn’t make allowances for more complex setups. (This is why init scripts were replaced by systemd for init, remember? Script-and-shell-based utilities, such as netctl — seriously, the entire thing’s written in Bash — just can’t handle more complex jobs reliably.)

If you need more advanced functionality but don’t want a lot of cruft or bloat, I recommend networkd as your network provider. It requires no extra packages (other than wpa_supplicant, if you’re using wireless) because it’s part of the systemd package (which is part of the most basic install of Arch) and handles more advanced configurations a lot more reliably.

"How do I specify WEP for a wireless network?"

You can’t. WEP’s pretty broken. I understand some legacy networks may still use it, but I’m incredibly uncomfortable supporting it.

If absolutely necessary, you can manually configure it yourself via a /aif/scripts/post/script script (or just configure it once you boot the newly-installed system).

"Then why do you allow connecting to open wireless networks in the config?"

Because captive portals are a thing. Authing to them, however; that’s out of my scope.

"How do I configure connecting to a WPA2 Enterprise network?"

You can’t, currently; support is only stubbed out for now. If absolutely necessary, you can manually configure it yourself via a /aif/scripts/post/script script.

This hopefully will be changed in the future, however, as I’m interested in adding support. For now, open and WPA/WPA2 PSK only are considered supported.

"How do I use my own GnuPG homedir instead of letting AIF-NG create one automatically?"

I can pretty easily add support for this — it’s stubbed in already. But there are a couple reasons it doesn’t really make sense to do so:

  • Being that most people are probably using this from a LiveCD/LiveUSB/PXE/whatever, it’s highly unlikely they’ll even have a static GnuPG homedir available.

  • Even if they did, AIF-NG has no real way of running a passphrase prompt. It’s intended to be run automatically, non-interactively, and daemonized. You’d have to have a passphrase-less private key for it to work.

    • Why? Because it needs to be able to sign and trust the key ID you specified to get an accurate validity reading of the signature. If the private key has a passphrase, this is required for the operation to complete. If a custom homedir with a passphrased private key was specified, the signature’s signer’s public key would already need to be imported into the keyring, signed, AND trusted (with a sufficiently high enough level).

"Why do I have to specify a URI or key ID for a GPG key but can include a raw text block for a GPG signature?"

Because keys are (generally speaking) intended to be publicly fetchable in some form or another. signatures are not (necessarily); they’re more geared towards being file objects. I definitely recommend using signatureFile instead, though, even if it’s just to a local .sig/.asc file.

"Why don’t you support WKD for GPG key fetching?"

Because I didn’t. If there is interest, I can add support for it but please don’t request it unless you plan on actually using it.

Bug Reports/Feature Requests

Note
 It is possible to submit a bug or feature request without registering in my bugtracker. One of my pet peeves is needing to create an account/register on a bugtracker simply to report a bug! The following links only require an email address to file a bug (which is necessary in case I need any further clarification from you or to keep you updated on the status of the bug/feature request — so please be sure to use a valid email address).

Bugs

If you encounter any bugs in AIF-NG (for the actual agent), you can file a bug report here.

If you encounter any bugs in the configuration file tool, you can file a bug report here.

If you encounter any bugs (inaccurate information, typos, misformatting, etc.) in this documentation, you can file a bug report here.

Feature Requests

If you have any features you’d like to see or you think would help AIF-NG become even more useful, please file a feature request here.

If you have any features you’d like to see in the configuration file tool, you can file a feature requests here.

If you have any suggestions on how to improve this documentation or feel it’s missing information that could be useful, please file a feature request here.

Patches

I gladly welcome patches, but I deplore using GitHub (even though I have a mirror there). For this reason, please follow the same patch/pull request process for the Linux kernel and email it to bts@square-r00t.net.

Alternatively, you may attach a patch to a bug report/feature request.

Contact the Author

If you have any questions, comments, or concerns, you can use the following information to get in touch with me.

I am available via email. If you use GPG, you can find my pubkey and other related info here (and on most keyservers).

I occasionally write howto articles, brief tips, and other information in my dev blog.

I am on IRC as r00t^2, and am usually in the Sysadministrivia channel on Freenode. Which reminds me, I run a podcast called Sysadministrivia.

I am on Twitter as @brentsaner, though I don’t tweet very often. (I usually tweet from my podcast’s twitter.)

1. See http://catb.org/jargon/html/Y/yak-shaving.html