working on the boot problem. also, documentation

bdisk/bdisk.py

@@ -21,7 +21,7 @@ if __name__ == '__main__':
     prep.buildChroot(conf, keep = False)
     prep.prepChroot(conf)
     arch = conf['build']['arch']
-    #bGPG.killStaleAgent(conf)
+    bGPG.killStaleAgent(conf)
     for a in arch:
         bchroot.chroot(conf['build']['chrootdir'] + '/root.' + a, 'bdisk.square-r00t.net')
         bchroot.chrootUnmount(conf['build']['chrootdir'] + '/root.' + a)

docs/TODO

@@ -22,6 +22,8 @@
 -There *has* to be a better way of handling package installation in the chroots.
 -maybe remove lxde, firefox, chrome and replace with enlightenment/midori?
 -custom repo? https://brainwreckedtech.wordpress.com/2013/01/27/making-your-own-arch-linux-repository/
+--https://wiki.archlinux.org/index.php/Building_32-bit_packages_on_a_64-bit_system
+--include arch repo clone script
 -implement better "additional" packages list. specify for path in build.ini- these should be more easily changed by end users. DON'T TOUCH iso.pkgs.lst since those are necessary for booting.
 
 

docs/manual/FURTHER.adoc

@@ -5,6 +5,7 @@
 Here you will find further info, other resources, and such relating to BDisk.
 --
 
+include::further/PASSWORDS.adoc[]
 include::further/BUGS.adoc[]
 include::further/CONTACT.adoc[]
 

docs/manual/further/PASSWORDS.adoc

@@ -0,0 +1,80 @@
+== Passwords
+NOTE: If you're specifying passwords, be sure to use a https://www.schneier.com/blog/archives/2014/03/choosing_secure_1.html[strong password^]!
+
+=== `build.ini` Password Value Examples
+Passwords work a little interestingly in BDisk. These aspects all apply to both <<__code_root_password_code,the root password>> and <<__code_passwowrd_code,the user password>> (if you enable a regular user).
+
+CAUTION: DO *NOT* USE A PLAINTEXT PASSWORD IN THE `build.ini`! This is _by design_; plaintext passwords are much more insecure. If you use a plaintext password, it *will not work*.
+
+WARNING: Remember to <<_escaping_the_salted_hash,escape your hash>> before placing it in your `build.ini`!
+
+.Password Value Scheme
+[frame="topbot",options="header,footer"]
+|======================
+|If you have...|BDisk will...
+|the string `BLANK`|give the user a blank password, allowing you to just hit `<Enter>` to log in
+|nothing set|lock the account (e.g. no non-SSH login is possible)
+|a properly hashed, salted, and escaped string|set the account to the password used to generate that hash.
+||
+|======================
+
+.Password Value Examples
+[frame="topbot",options="header,footer"]
+|======================
+|If the value is...|Then BDisk...
+|`root_password = BLANK`|will let you log into the TTY as the root user by just hitting the `<Enter>` key.
+|`root_password =`|will not allow the root user to log into the TTY at all.
+|`root_password = <some salted, hashed, escaped string created from 'test'>`|will let you log into the root user on a TTY with the password `test`.
+||
+|======================
+
+
+NOTE: I specify "TTY login" because SSH login may still be possible. By default, SSH will allow password logins for non-root users (root user SSH password login is prohibited by default; only pubkey login for root is allowed.) -- this can be overridden, however, by customization.
+
+=== Generating a Password Salt/Hash
+First, if you are not familiar with a http://man7.org/linux/man-pages/man3/crypt.3.html#NOTES[salted hash^] that GNU/Linux uses, you may want to learn about it.
+
+That said, there are utilities in `extra/bin/` that should generate a salted hash for you. Currently only `hashgen.py` is distributed, but additions/examples for other languages are welcome.
+
+....
+ $ ./hashgen.py 
+ 
+ What password would you like to hash/salt?
+ (NOTE: will NOT echo back!)
+ 
+ 
+ Your salted hash is:
+ 	$6$t92Uvm1ETLocDb1D$BvI0Sa6CSXxzIKBinIaJHb1gLJWheoXp7WzdideAJN46aChFu3hKg07QaIJNk4dfIJ2ry3tEfo3FRvstKWasg/
+ 
+....
+
+The password `test` was used above. In `crypt(3)`-salted hashes, there are specific sections separated by USD dollar symbols (`$`). The first section (containing `6`) marks the *hash algorithm* -- in this case, _SHA512_. (The http://man7.org/linux/man-pages/man3/crypt.3.html#NOTES[crypt man page^] mentions all supported hash types and their corresponding ID.) The next section, `t92Uvm1ETLocDb1D`, is the *salt*. The last section is the *hash*. How salted hashes work is an original piece of data is given (in our case, the word `test`). This data is then sent through a one-way cryptographic process that generates a new string that makes it difficult to know what the original data was. THEN a salt is added- a random string- and the process repeats. In our format, this is done _5000_ times in a row. When you log in with your password, the salt is fetched and the same process is done again- predictably, the data that process goes through should then match the salted hash string stored in the password system (in this case, the https://linux.die.net/man/5/shadow[`/etc/shadow`] file).
+
+Whew! Got all that? Good.
+
+=== Escaping the Salted Hash
+One last thing, and this is *very* important -- failure to perform this step will cause all sorts of strange Python errors -- is to escape the salted hash. Thankfully, however, this is a lot easier than it sounds.
+
+So we have our salted hash: `$6$t92Uvm1ETLocDb1D$BvI0Sa6CSXxzIKBinIaJHb1gLJWheoXp7WzdideAJN46aChFu3hKg07QaIJNk4dfIJ2ry3tEfo3FRvstKWasg/`. In order to get it into a usable format, we need to make sure the configuration parsing won't try to read sections of it as variables. To do this, we do something called *escaping*.
+
+All you need to do is take the salted hash and replace every `$` you see -- there should be exactly three -- with `$$`. That's it! Count them to be sure; you should now have *6* `$` symbols present instead of three. Once you've escaped the salted hash, you're ready to roll.
+
+=== Cheating/The Easy Way
+Feeling overwhelmed? There's an easy way to do all of this.
+
+First, while logged into your local computer, change your password to what you want ether `root_password` or `password` to be:
+
+ passwd
+
+NOTE: Remember, changing your password won't echo the password back on the screen for security reasons!
+
+Then get your shadow entry. This has to be done with sudo, as only the root user has access to the hashed passwords on the system. The following command will combine all steps necessary; the string it returns will be a string you can use directly in your `build.ini`.
+
+ sudo grep "^${SUDO_USER}:" /etc/shadow | awk -F':' '{print $2}' | sed -e 's/\$/$$/'
+
+Don't forget to change your password back to what it was before!
+
+ passwd
+
+That's it!
+

docs/manual/user/BUILDINI.adoc

@@ -73,3 +73,180 @@ We'll go into more detail for each section below.
  user =
  path =
  iso = no
+
+=== `[bdisk]`
+This section controls some basic branding and information having to do with the end product.
+
+==== `name`
+This value is a "basic" name of your project. It's not really shown anywhere end-user visible, but we need a consistent name that follows some highly constrained rules:
+
+. Alphanumeric only
+. 8 characters total (or less)
+. No whitespace
+. ASCII only
+. Will be converted to uppercase if it isn't already
+
+==== `uxname`
+This value is used for filenames and the like. I highly recommend it be the same as `<<__code_name_code,name>>` (in lowercase) but it doesn't need to be. It also has some rules:
+
+. Alphanumeric only
+. No whitespace
+. ASCII only
+. Will be converted to lowercase if it isn't already
+
+==== `pname`
+This string is used for "pretty-printing" of the project name; it should be a more human-readable string.
+
+. *Can* contain whitespace
+. *Can* be mixed-case, uppercase, or lowercase
+. ASCII only
+
+==== `ver`
+The version string. If this isn't specified, we'll try to guess based on the current git commit and tags in `<<__code_basedir_code,build:basedir>>`.
+
+. No whitespace
+
+==== `dev`
+The name of the developer or publisher of the ISO, be it an individual or organization. For example, if you are using BDisk to build an install CD for your distro, this would be the name of your distro. The same rules as `<<__code_pname_code,pname>>` apply.
+
+. *Can* contain whitespace
+. *Can* be mixed-case, uppercase, or lowercase
+. ASCII only
+
+==== `email`
+An email address to use for git syncing messages, and/or GPG key generation.
+
+==== `desc`
+What this distribution/project is used for.
+
+. *Can* contain whitespace
+. *Can* be mixed-case, uppercase, or lowercase
+. ASCII only
+
+==== `uri`
+What is this project's URI (website, etc.)? Alternatively, your personal site, your company's site, etc.
+
+. Should be a valid URI understood by curl
+
+
+==== `root_password`
+The escaped, salted, hashed string to use for the root user.
+
+Please see <<_passwords,the section on passwords>> for information on this value. In the <<_example,example above>>, the string `$$6$$t92Uvm1ETLocDb1D$$BvI0Sa6CSXxzIKBinIaJHb1gLJWheoXp7WzdideAJN46aChFu3hKg07QaIJNk4dfIJ2ry3tEfo3FRvstKWasg/` is created from the password `test`. I cannot stress this enough, do not use a plaintext password here nor just use a regular `/etc/shadow` file/`crypt(3)` hash here. Read the section. I promise it's short.
+
+==== `user`
+*Default: no*
+
+This setting specifies if we should create a regular (non-root) user in the live environment.
+
+NOTE: If enabled, this user has full sudo access.
+
+[options="header"]
+|======================
+2+^|Accepts (case-insensitive) one of:
+^m|yes ^m|no
+^m|true ^m|false
+^m|1 ^m|0
+|======================
+
+=== `[user]`
+This section of `build.ini` controls aspects about `bdisk:user`. It is only used if <<__code_user_code,`bdisk:user`>> is enabled.
+
+==== `username`
+What username should the user have? Standard *nix username rules apply:
+
+. ASCII only
+. 32 characters or less
+. Alphanumeric only
+. Lowercase only
+. No whitespace
+. Cannot start with a number
+
+==== `name`
+What comment/description/real name should be used for the user? For more information on this, see the https://linux.die.net/man/5/passwd[passwd(5) man page^]'s section on *GECOS*.
+
+. ASCII only
+
+==== `password`
+The escaped, salted, hashed string to use for the non-root user.
+
+Please see <<_passwords,the section on passwords>> for information on this value. In the <<_example,example above>>, the string `$$6$$t92Uvm1ETLocDb1D$$BvI0Sa6CSXxzIKBinIaJHb1gLJWheoXp7WzdideAJN46aChFu3hKg07QaIJNk4dfIJ2ry3tEfo3FRvstKWasg/` is created from the password `test`. I cannot stress this enough, do not use a plaintext password here nor just use a regular `/etc/shadow` file/`crypt(3)` hash here. Read the section. I promise it's short.
+
+=== `[build]`
+This section controls some aspects about the host and things like filesystem paths, etc.
+
+==== `mirror`
+A mirror that hosts the bootstrap tarball. It is *highly* recommended you use an Arch Linux https://wiki.archlinux.org/index.php/Install_from_existing_Linux#Method_A:_Using_the_bootstrap_image_.28recommended.29[bootstrap tarball^] as the build process is highly specialized to this (but <<_bug_reports_feature_requests,patches/feature requests>> are welcome for other built distros). You can find a list of mirrors at the bottom of Arch's https://www.archlinux.org/download/[download page^].
+
+. No whitespace
+. Must be accessible remotely/via a WAN-recognized address
+. Must be a domain/FQDN only; no paths (those come later!)
+
+==== `mirrorproto`
+What protocol should we use for <<_mirror,the mirror>>?
+
+|======================
+^s|Must be (case-insensitive) one of: ^.^m|http ^.^m|https ^.^m|ftp
+|======================
+
+==== `mirrorpath`
+What is the path to the tarball directory on the <<__code_mirror_code,`mirror`>>?
+
+. Must be a complete path (e.g. `/dir1/subdir1/subdir2`)
+. No whitespace
+
+==== `mirrorfile`
+What is the filename for the tarball found in the path specified in <<__code_mirrorpath_code,`mirrorpath`>>? If left blank, we will use the sha1 <<__code_mirrorchksum_code,checksum>> file to try to guess the most recent file.
+
+==== `mirrorchksum`
+The path to a sha1 checksum file of the bootstrap tarball.
+
+. No whitespace
+. Must be the full path
+. Don't include the mirror domain or protocol
+
+==== `mirrorgpgsig`
+*[optional]* +
+*default: (no GPG checking done)* +
+*requires: <<_optional,_gpg/gnupg_>>* +
+*requires: <<__code_gpgkey_code,`gpgkey`>>*
+
+If the bootstrap tarball file has a GPG signature, we can use it for extra checking. If it's blank, GPG checking will be disabled.
+
+If you specify just `.sig` (or use the default and don't specify a <<__code_mirrorfile_code,`mirrorfile`>>), BDisk will try to guess based on the file from the sha1 <<__code_mirrorchksum_code,checksum>> file. Note that this must evaluate to a full URL. (e.g. `${mirrorproto}://${mirror}${mirrorpath}somefile.sig`)
+
+==== `gpgkey`
+*requires: <<_optional,_gpg/gnupg_>>*
+
+What is a key ID that should be used to verify/validate the <<__code_mirrorgpgsig_code,`mirrorgpgsig`>>?
+
+. Only used if <<__code_mirrorgpgsig_code,`mirrorgpgsig`>> is set
+. Can be in "short" form (e.g. _7F2D434B9741E8AC_) or "full" form (_4AA4767BBC9C4B1D18AE28B77F2D434B9741E8AC_), with or without the _0x_ prefix.
+
+==== `gpgkeyserver`
+*default: blank (GNUPG-bundled keyservers)* +
+*requires: <<_optional,_gpg/gnupg_>>*
+
+What is a valid keyserver we should use to fetch <<__code_gpgkey_code,`gpgkey`>>?
+
+. Only used if <<__code_mirrorgpgsig_code,`mirrorgpgsig`>> is set
+. The default (blank) is probably fine. If you don't specify a personal GPG config, then you'll most likely want to leave this blank.
+. If set, make sure it is a valid keyserver URI (e.g. `hkp://keys.gnupg.net`)
+
+==== `gpg`
+Should we sign our release files? See the gpg section.
+
+[options="header"]
+|======================
+2+^|Accepts (case-insensitive) one of:
+^m|yes ^m|no
+^m|true ^m|false
+^m|1 ^m|0
+|======================
+
+==== `dlpath`
+Where should the release files be saved? Note that many other files are created here as well.
+
+. No whitespace
+. Will be created if it doesn't exist
+

docs/manual/user/GETTING_STARTED.adoc

@@ -63,7 +63,7 @@ NOTE: If you do not wish to install any of these or cannot install them, be sure
 * http://gcc.gnu.org[gcc (multilib)^] (>=6.x)
 ** Needed for building iPXE.
 * http://gcc.gnu.org[gcc-libs (multilib)^] (>=6.x)
-** (Same as gcc.)
+** (Same as _gcc_.)
 * https://git-scm.com/[git^]
 ** For autodetection of version, automatically making commits for your project, etc.
 * https://www.gnupg.org/[gpg/gnupg^] (>=2.1.11)

extra/pre-build.d/etc/initcpio/install/livecd

@@ -6,6 +6,11 @@ build()
 	add_module 'loop'
 	add_module 'overlay'
 
+	add_file "/etc/passwd"
+	add_file "/etc/shadow"
+	add_file "/etc/group"
+	add_file "/etc/gshadow"
+
         add_binary "/usr/bin/sed"
         add_binary "/usr/bin/pkill"
         add_binary "/usr/bin/curl"

extra/pre-build.d/etc/mkinitcpio.conf

@@ -49,10 +49,9 @@ FILES="/usr/bin/pkill"
 ##   NOTE: If you have /usr on a separate partition, you MUST include the
 #    usr, fsck and shutdown hooks.
 #HOOKS="base udev autodetect modconf block filesystems keyboard fsck"
-#HOOKS="base udev memdisk archiso_shutdown archiso modconf net ssh archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
-#HOOKS="base udev memdisk archiso_shutdown archiso-custom modconf net ssh archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_http_custom archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
-HOOKS="base udev memdisk archiso_shutdown archiso modconf net archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
-#HOOKS="base memdisk systemd archiso_shutdown archiso modconf ssh archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
+#HOOKS="base udev memdisk archiso_shutdown archiso modconf net archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
+#HOOKS="base udev autodetect memdisk systemd archiso_shutdown archiso modconf net ssh archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms block pcmcia filesystems keyboard livecd"
+HOOKS="base udev memdisk autodetect archiso_shutdown archiso archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_kms modconf net ssh block pcmcia filesystems keyboard livecd"
 
 # COMPRESSION
 # Use this to compress the initramfs image. By default, gzip compression

extra/pre-build.d/root/pre-build.sh

@@ -1,9 +1,14 @@
 #!/bin/bash
 
-source /etc/bash.bashrc
 # needed so we override whatever's set in python
 # alternatively, we can just mkdir -p $GNUPGHOME
-export GNUPGHOME=/root/.gnupg
+#export GNUPGHOME=/root/.gnupg
+unset GNUPGHOME
+mkdir -p /var/empty/.gnupg
+
+
+# set up a shell env
+source /etc/profile
 
 # Import settings.
 source /root/VARS.txt
@@ -67,6 +72,7 @@ cleanPacorigs
 pacman --noconfirm -U /root/apacman*.tar.xz &&\
 	 mkdir /var/tmp/apacman && chmod 0750 /var/tmp/apacman &&\
 	 chown root:aurbuild /var/tmp/apacman
+chown aurbuild:aurbuild /var/empty/.gnupg
 cleanPacorigs
 apacman -Syy
 apacman -S --noconfirm --noedit --skipinteg --needed -S apacman apacman-deps apacman-utils expac