diff --git a/README.rst b/README.rst index f71abc1..76aab9e 100644 --- a/README.rst +++ b/README.rst @@ -1,127 +1,133 @@ +==================================== Zettabyte File System for Arch Linux ==================================== :Modified: Sun Sep 11 18:01 2016 -This project contains the build scripts for archzfs. Two types of packages are -supported, Git packages and LTS packages. - -The Git packages include zfs-git, zfs-utils-git, spl-git, and spl-utils-git. -These track the mainline kernel releases in arch. Since Arch is bleeding edge, -so too are these packages. They usually pull from zfsonlinux master branch up -to the latest commit that add supports for latest kernel version. - -The LTS packages include zfs-lts, zfs-utils-lts, spl-lts, and spl-utils-lts. -These track the linux-lts packages in arch and are only built using stable -zfsonlinux releases. These packages are the best bet for those concerned with -stability. - -These packages must be re-built for each kernel release. +Welcome to the archzfs project. This repo contains everything used to deploy ZFS to Arch Linux. -------- -Licenses +Overview -------- -The license of the Arch Linux package sources is MIT. +WIP -The license of ZFS is CDDL. +.. The Git packages include zfs-git, zfs-utils-git, spl-git, and spl-utils-git. These track the mainline kernel releases in +.. arch. Since Arch is bleeding edge, so too are these packages. They usually pull from zfsonlinux master branch up to the +.. latest commit that add supports for latest kernel version. -The license of SPL is LGPL. +.. The LTS packages include zfs-lts, zfs-utils-lts, spl-lts, and spl-utils-lts. These track the linux-lts packages in arch and +.. are only built using stable zfsonlinux releases. These packages are the best bet for those concerned with stability. + +.. These packages must be re-built for each kernel release. ------------ Dependencies ------------ -``build.sh`` uses clean-chroot-manager_ to build packages in a systemd -namespace container. +Building +++++++++ + +* `clean-chroot-manager`_ to build packages in a systemd namespace container + +Testing ++++++++ + +Dependencies used by the `test.sh` command. + +* QEMU +* Packer +* NFS -------------------------- How to use this Repository -------------------------- -Submitting changes -++++++++++++++++++ +*WIP* + +.. Clone +.. +++++ + +.. .. code:: console -The Arch User Repository v4 requires package updates to be submitted through -Git. Unfortunately, this makes working with the archzfs build scripts on Github -slightly more complicated. The actual PKGBUILDS in the archzfs repo are set as -git submodules which reference the aur4 git repo and a commit hash. Github does -not support external submodules, and thus PR requests are no longer possible on -PKGBUILDS and mkinitcpio hook scripts. + .. git clone https://github.com/archzfs/archzfs.git --recursive -Contributions are still very much appreciated! To submit a contribution, please -open an issue in Github and within the description post a git generated patch -of the changes. This is only temporary until AUR supports some kind of "pull -request" type system. +.. Submitting changes +.. ++++++++++++++++++ -Clone -+++++ +.. T -.. code:: console +.. Update +.. ++++++ - git clone https://github.com/archzfs/archzfs.git --recursive +.. 1. Set the appropriate variables in conf.sh -Update -++++++ + .. * Repository base path -1. Set the appropriate variables in conf.sh + .. The parent directory of the ``demz-repo-core`` and ``demz-repo-archiso`` + .. repos. - * Repository base path + .. * GPG signing key - The parent directory of the ``demz-repo-core`` and ``demz-repo-archiso`` - repos. + .. Used to sign the packages and repo database. - * GPG signing key + .. * Your email address - Used to sign the packages and repo database. + .. Used for reporting changes changes in ``scraper.sh``. Mutt and msmtp are + .. used for sending email. Required only if ``scraper.sh`` or ``verifier.sh`` + .. are going to be used. - * Your email address + .. * SSH remote login - Used for reporting changes changes in ``scraper.sh``. Mutt and msmtp are - used for sending email. Required only if ``scraper.sh`` or ``verifier.sh`` - are going to be used. + .. Used in ``verifier.sh`` for making sure the local ``demz-repo-*`` are in + .. sync with the remote repos. - * SSH remote login +.. #. Set the appropriate kernel versions in conf.sh. - Used in ``verifier.sh`` for making sure the local ``demz-repo-*`` are in - sync with the remote repos. +.. #. Update the PKGBUILDs -#. Set the appropriate kernel versions in conf.sh. + .. Use ``./build git update`` to update the archzfs-git PKGBUILDS using the + .. ``conf.sh`` variables. -#. Update the PKGBUILDs + .. Use ``./build lts update`` to update the archzfs-lts PKGBUILDS using the + .. ``conf.sh`` variables. - Use ``./build git update`` to update the archzfs-git PKGBUILDS using the - ``conf.sh`` variables. + .. ``./build.sh (git|lts) update-test`` uses the ``AZB_LINUX_TEST_*`` + .. variables. Using the test values are useful for test building the zfs + .. packages against the Linux kernel version in the official testing repo. + .. Mostly used on minor Linux kernel updates (3.12 -> 3.13). - Use ``./build lts update`` to update the archzfs-lts PKGBUILDS using the - ``conf.sh`` variables. +.. #. Build the packages - ``./build.sh (git|lts) update-test`` uses the ``AZB_LINUX_TEST_*`` - variables. Using the test values are useful for test building the zfs - packages against the Linux kernel version in the official testing repo. - Mostly used on minor Linux kernel updates (3.12 -> 3.13). + .. Use ``./build.sh (git|lts) make -u`` to build the packages, update the clean + .. chroot in the process. -#. Build the packages + .. It is possible to use ``./build.sh (git|lts) update make -u`` in one shot. - Use ``./build.sh (git|lts) make -u`` to build the packages, update the clean - chroot in the process. + .. If you want to see command output only, use ``./build.sh (git|lts) make + .. -n``. Add the ``-d`` to see debugging output. - It is possible to use ``./build.sh (git|lts) update make -u`` in one shot. +.. #. Add packages to the repo - If you want to see command output only, use ``./build.sh (git|lts) make - -n``. Add the ``-d`` to see debugging output. + .. Use ``./repo.sh (git|lts) core -n`` to what changes will occur without + .. actually making them. -#. Add packages to the repo + .. ``./repo.sh (git|lts) core`` will add the package versions defined by + .. ``AZB_LINUX_VERSION`` to the ``demz-repo-core`` repository. - Use ``./repo.sh (git|lts) core -n`` to what changes will occur without - actually making them. +.. #. Push the package sources to AUR. - ``./repo.sh (git|lts) core`` will add the package versions defined by - ``AZB_LINUX_VERSION`` to the ``demz-repo-core`` repository. + .. Push to AUR4 using ``push.sh (git|lts) aur4``. This commits the latest + .. changes to each individual package repo (for AUR) and uses ``git push`` to + .. push to AUR. -#. Push the package sources to AUR. +-------- +Licenses +-------- - Push to AUR4 using ``push.sh (git|lts) aur4``. This commits the latest - changes to each individual package repo (for AUR) and uses ``git push`` to - push to AUR. +The license of the Arch Linux package sources is MIT. + +The license of ZFS is CDDL. + +The license of SPL is LGPL. .. _clean-chroot-manager: https://aur.archlinux.org/packages/clean-chroot-manager diff --git a/lib.sh b/lib.sh index 2a04ef0..3342281 100644 --- a/lib.sh +++ b/lib.sh @@ -206,6 +206,19 @@ run_cmd() { } +run_cmd_check() { + # $1 Exit code + # $2 Error string if defined with print an error message + if [[ ${run_cmd_return} -eq 0 ]]; then + return + fi + if [[ -n $2 ]]; then + error "$2" + fi + exit $1 +} + + # Runs a command. Ouput is not captured. dry_run=1 is ignored. # To use this function, define the following in your calling script: # run_cmd_return="" diff --git a/testing/README.rst b/testing/README.rst index fddc76c..77d9a8b 100644 --- a/testing/README.rst +++ b/testing/README.rst @@ -1,7 +1,19 @@ ===================== archzfs testing guide ===================== -:Modified: Sun Sep 04 10:04 2016 +:Modified: Mon Sep 12 00:32 2016 + +----- +Notes +----- + +- Sun Apr 19 19:45 2015: Found more tests at https://github.com/behlendorf/xfstests + + Requires additional pools + +- Sun Apr 19 19:51 2015: ztest slides http://blog.delphix.com/csiden/files/2012/01/ZFS_Backward_Compatability_Testing.pdf + +- Sun Apr 19 20:05 2015: What I am trying to do is described here: https://github.com/zfsonlinux/zfs/issues/1534 -------- Overview @@ -69,7 +81,7 @@ Build and test process ---------------------- Stage 1 -+++++++ +======= 1. Build the packages using the normal build process, but without signing. @@ -82,7 +94,7 @@ Stage 1 #. Perform regression tests Stage 2 -+++++++ +======= 1. Use packer to build zfs root filesystem test instances @@ -139,7 +151,7 @@ Qemu Setup # pacman -Sy qemu nfs -+++ +=== :: @@ -148,15 +160,171 @@ nfs qemu sends packets from 127.0.0.1:44730 to 127.0.0.1:2049 for mounting. The insecure option allows packets from ports > 1024 ------ -Notes ------ +How +=== -- Sun Apr 19 19:45 2015: Found more tests at https://github.com/behlendorf/xfstests +1. The archzfs-linux packages are built for the linux kernel and added to a package repository named "archzfs-testing". - Requires additional pools +#. The archzfs-testing repo is shared over NFS. -- Sun Apr 19 19:51 2015: ztest slides http://blog.delphix.com/csiden/files/2012/01/ZFS_Backward_Compatability_Testing.pdf +#. A custom archiso is built that boots into the linux-lts kernel. -- Sun Apr 19 20:05 2015: What I am trying to do is described here: https://github.com/zfsonlinux/zfs/issues/1534 +#. The test files are compressed into a tar archive. + +#. Packer is used to build a qemu base image using the custom archiso. + +#. `setup.sh` is ran in the archiso to install arch on ZFS. + +#. After installation of Arch on ZFS, the VM is rebooted and packer finalizes the base image. + +#. The qemu base image created by packer is booted, if the boot is successful, the test is considered passed. + +Adding a new test +================= + +While adding a new automated test is not as easy, it is extremely beneficial to the project. So if one is so inclined, please +consider helping us all! + +Copy one of the existing tests to a new directory. The name of the test is important, so follow this naming scheme:: + + archzfs-qemu--test-- + +If defining a brand new test for a brand new kernel, please use `test-00-default-` for the standard archzfs use case of +booting from ext4 and mounting a zfs data device. Use `test-00-boot-` to define a test that boots from archzfs. + +Once this is done edit the various files to sorta get what you want. There are a few files that will probably be edited +most:: + + conf.sh + fs.sh + hooks.sh + + +This is the most difficult part because it is necessary to define an installation for Arch Linux through a series of commands +that will run through packer/qemu. `testing/archzfs-qemu-base/setup.sh` is the script run in the archiso to install arch. It +loads the files defined in the test and calls each of the "hooks" in turn. Similar to how PKGBUILDs work with makepkg. + +Finally, run your test from the root project directory:: + + # testing/test.sh -d std-test-00-default + +You should see packer start archiso in qemu and begin previsioning the device. Once packer is done, the device will reboot +and the acceptance criteria will determine if the test succeeded. + +If you think any of these steps can be done simpler and/or more efficiently, please open an issue! + +---------------------------------- +Setting up Testing for ZOL 0.6.5.8 +---------------------------------- +:Date: Sun Sep 11 17:08 2016 + +This is an example session where I setup tests for ZOL 0.6.5.8. I will be setting up two tests, a simple test where zfs is +used for a data volume and a complex test where archzfs is used as a boot filesystem. + +The upstream ZOL maintainers released 0.6.5.8 that supports linux 4.7 and 4.8. Now that archzfs has a working example for an +automated test for archzfs-linux-lts, let's use the same test setup for the standard archzfs packages and this new ZOL +version. Since it is nontrivial to setup a new automated test, I want to record these steps for anyone that may take up the +noble task in the future. + +Test #1: ZFS data volume in Arch Linux +====================================== + +There are a few test criteria for this first test: + +1. Boot into a regular arch system with the archzfs packages already installed. +#. Create a new zpool and mount it. +#. Write some data to it and reboot. +#. After reboot make sure the pool is automatically mounted data is still there. + +If all of these criteria are met, the test is a success. + +Another key area I am going to look at is the changes made to the systemd scripts reported by bronek via +https://github.com/archzfs/archzfs/issues/72. How will these changes impact + +Deployng the archzfs-linux packages to the archzfs-testing repo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +This is the easiest part, all that needed to be done was to increment the version number in `src/kernels/linux.sh` and build +using `./build.sh std update make -u -U` once this was done the packages were added to the `archzfs-testing` repo using +`./repo std test`. This repo is mounted in the archiso and arch-chroot via NFS using the test setup scripts. + +Copying the files ++++++++++++++++++ + +Most of the work to use packer to build a base image has been done previously for an archzfs-linux-lts test, so we'll reuse +that configuration and modify it to boot into a regular arch linux installation on ext4. + +The files that were modified were:: + + fs.sh + conf.sh + config.sh + syslinux.cfg + boot.sh + chroot.sh + +Booting the base image +++++++++++++++++++++++ + +Booting the qemu image: + +.. code:: console + + sudo /usr/bin/qemu-system-x86_64 -device virtio-net,netdev=user.0 \ + -drive file=testing/files/packer_work/output-qemu/archzfs-qemu-std-test-00-default-archiso-2016.09.10,if=virtio,cache=writeback,discard=ignore \ + -vnc 0.0.0.0:32 -netdev user,id=user.0,hostfwd=tcp::3333-:22 \ + -name archzfs-qemu-std-test-00-default-archiso-2016.09.10 -machine type=pc,accel=kvm -display sdl -boot once=d -m 512M + +Connection via ssh: + +.. code:: console + + ssh root@10.0.2.15 -p 3333 + +Password is `azfstest` + +Attempting to run `zpool status` results in:: + + [root@test ~]# zpool status + The ZFS modules are not loaded. + Try running '/sbin/modprobe zfs' as root to load them. + +Let's make zfs start automatically on boot via the base image setup scripts. + +But first, we must understand the updated systemd configuration for ZFSonLinux. + +Running `pacman -Ql zfs-linux` show the systemd files:: + + zfs-utils-linux /usr/lib/systemd/ + zfs-utils-linux /usr/lib/systemd/system-preset/ + zfs-utils-linux /usr/lib/systemd/system-preset/50-zfs.preset + zfs-utils-linux /usr/lib/systemd/system/ + zfs-utils-linux /usr/lib/systemd/system/zfs-import-cache.service + zfs-utils-linux /usr/lib/systemd/system/zfs-import-scan.service + zfs-utils-linux /usr/lib/systemd/system/zfs-mount.service + zfs-utils-linux /usr/lib/systemd/system/zfs-share.service + zfs-utils-linux /usr/lib/systemd/system/zfs-zed.service + zfs-utils-linux /usr/lib/systemd/system/zfs.target + +Particularly, let's look at `50-zfs.preset`. This is the file used by the upstream maintainers to configure systemd to +autostart ZFS system at boot. The Arch ethos forbids starting a process unless not initiated by the user and this is +hardcoded into the systemd arch installation by default:: + + [root@test ~]# cat /usr/lib/systemd/system-preset/99-default.preset + disable * + +This file will disable all autoloaded systemd units. + +* `Enable installed units by default `_ + +In our case, we should at least enable the kernel module at boot so the user can at least issue zfs commands. + +So this was added to the zfs-utils packages in `src/zfs-utils/PKGBUILD.sh`. We autoload the zfs kernel module by placing a +config file in `/etc/modules-load.d/zfs.conf`. Now zfs can be used after installation and first reboot. + +.. Creating an disk for zfs: + +.. .. code:: console + + .. sudo qemu-img create -f qcow2 output-qemu/archzfs-qemu-std-test-00-default-archiso-2016.09.10 122880M diff --git a/testing/test.sh b/testing/test.sh index a09613f..08e1f82 100755 --- a/testing/test.sh +++ b/testing/test.sh @@ -180,13 +180,12 @@ if [[ "${test_mode}" != "" ]]; then msg2 "Using packer to build the base image ..." # Base files + run_cmd "check_symlink '${script_dir}/tests/archzfs-qemu-base/packages' '${packer_work_dir}/packages'" run_cmd "check_symlink '${script_dir}/tests/archzfs-qemu-base/packer.json' '${packer_work_dir}/packer.json'" - run_cmd "check_symlink '${script_dir}/tests/archzfs-qemu-base/base.sh' '${packer_work_dir}/base.sh'" run_cmd "check_symlink '${script_dir}/tests/archzfs-qemu-base/setup.sh' '${packer_work_dir}/setup.sh'" run_cmd "check_symlink '${script_dir}/../lib.sh' '${packer_work_dir}/lib.sh'" run_cmd "check_symlink '${script_dir}/../conf.sh' '${packer_work_dir}/archzfs-conf.sh'" run_cmd "check_symlink '${script_dir}/files/poweroff.timer' '${packer_work_dir}/poweroff.timer'" - # run_cmd "check_symlink '${repo_basepath}/${repo_name_test}' '${packer_work_dir}/${repo_name_test}'" # Test files run_cmd "check_symlink '${test_mode}/archiso.sh' '${packer_work_dir}/test-archiso.sh'" @@ -196,6 +195,7 @@ if [[ "${test_mode}" != "" ]]; then run_cmd "check_symlink '${test_mode}/fs.sh' '${packer_work_dir}/test-fs.sh'" run_cmd "check_symlink '${test_mode}/hooks.sh' '${packer_work_dir}/test-hooks.sh'" run_cmd "check_symlink '${test_mode}/pacman.sh' '${packer_work_dir}/test-pacman.sh'" + run_cmd "check_symlink '${test_mode}/config.sh' '${packer_work_dir}/test-config.sh'" run_cmd "check_symlink '${test_mode}/syslinux.cfg' '${packer_work_dir}/syslinux.cfg'" # Make it easy to get the files into the archiso environment diff --git a/testing/tests/archzfs-qemu-base/packages b/testing/tests/archzfs-qemu-base/packages new file mode 100644 index 0000000..44d5dba --- /dev/null +++ b/testing/tests/archzfs-qemu-base/packages @@ -0,0 +1 @@ +gptfdisk openssh syslinux htop tree vim diff --git a/testing/tests/archzfs-qemu-base/setup.sh b/testing/tests/archzfs-qemu-base/setup.sh index 4e51973..8f0610b 100644 --- a/testing/tests/archzfs-qemu-base/setup.sh +++ b/testing/tests/archzfs-qemu-base/setup.sh @@ -11,10 +11,10 @@ fi # source_safe "${script_dir}/archzfs-conf.sh" +source_safe "${script_dir}/test-conf.sh" source_safe "${script_dir}/test-archiso.sh" source_safe "${script_dir}/test-boot.sh" source_safe "${script_dir}/test-chroot.sh" -source_safe "${script_dir}/test-conf.sh" source_safe "${script_dir}/test-fs.sh" source_safe "${script_dir}/test-hooks.sh" source_safe "${script_dir}/test-pacman.sh" @@ -60,6 +60,9 @@ test_bootloader_install test_fs_config_root_postinstall +exit 0 + + # Reboot! test_setup_exit diff --git a/testing/tests/archzfs-qemu-iso-test-00-default/pacman.sh b/testing/tests/archzfs-qemu-iso-test-00-default/pacman.sh deleted file mode 100644 index f933752..0000000 --- a/testing/tests/archzfs-qemu-iso-test-00-default/pacman.sh +++ /dev/null @@ -1,19 +0,0 @@ -test_pacman_config() { - msg "Setting archiso pacman mirror" - # /usr/bin/cp mirrorlist /etc/pacman.d/mirrorlist - - # setup pacman repositories in the archiso - # msg "Installing local pacman package repositories" - # test_pacman_config /etc/pacman.conf - - # dirmngr < /dev/null - # pacman-key -r 0EE7A126 - # if [[ $? -ne 0 ]]; then - # exit 1 - # fi - # pacman-key --lsign-key 0EE7A126 - # pacman -Sy archzfs-archiso-linux - # modprobe zfs -} - - diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/README.rst b/testing/tests/archzfs-qemu-lts-test-00-bootfs/README.rst similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/README.rst rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/README.rst diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/archiso.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/archiso.sh similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/archiso.sh rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/archiso.sh diff --git a/testing/tests/archzfs-qemu-lts-test-00-bootfs/boot.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/boot.sh new file mode 100644 index 0000000..307f2df --- /dev/null +++ b/testing/tests/archzfs-qemu-lts-test-00-bootfs/boot.sh @@ -0,0 +1,8 @@ +test_bootloader_install() { + # Setup the boot loader + run_cmd "mkdir -p ${arch_target_dir}/boot/syslinux; cp -f /root/syslinux.cfg '${arch_target_dir}/boot/syslinux/syslinux.cfg'" + run_cmd "arch-chroot ${arch_target_dir} /usr/bin/syslinux-install_update -i -a -m" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi +} diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/chroot.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/chroot.sh similarity index 69% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/chroot.sh rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/chroot.sh index a72a985..029fc5f 100644 --- a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/chroot.sh +++ b/testing/tests/archzfs-qemu-lts-test-00-bootfs/chroot.sh @@ -1,10 +1,3 @@ -export fqdn='test.archzfs.test' -export keymap='us' -export language='en_US.UTF-8' -export password=$(/usr/bin/openssl passwd -crypt 'azfstest') -export timezone='UTC' - - test_chroot_setup() { # $1 arch-chroot target dir msg "Setting up arch install..." @@ -18,19 +11,19 @@ test_chroot_setup() { msg2 "generating the filesystem table" run_cmd "/usr/bin/genfstab -p ${arch_target_dir} >> '${arch_target_dir}/etc/fstab'" - msg2 "Create base.sh" - run_cmd "/usr/bin/install --mode=0755 /dev/null '${arch_target_dir}/usr/bin/base.sh'" - # http://comments.gmane.org/gmane.linux.arch.general/48739 msg2 "Adding workaround for shutdown race condition" run_cmd "/usr/bin/install --mode=0644 poweroff.timer '${arch_target_dir}/etc/systemd/system/poweroff.timer'" + msg2 "Create config.sh" + run_cmd "/usr/bin/install --mode=0755 /dev/null '${arch_target_dir}/usr/bin/config.sh'" + # Special filesystem configure script - source_safe /root/base.sh + source_safe /root/test-config.sh msg2 "Entering chroot and configuring system" - run_cmd "/usr/bin/arch-chroot ${arch_target_dir} base.sh" + run_cmd "/usr/bin/arch-chroot ${arch_target_dir} /usr/bin/config.sh" - msg2 "Deleting base.sh" - rm ${arch_target_dir}/usr/bin/base.sh + msg2 "Deleting config.sh" + rm ${arch_target_dir}/usr/bin/config.sh } diff --git a/testing/tests/archzfs-qemu-lts-test-00-bootfs/conf.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/conf.sh new file mode 100644 index 0000000..35d0e06 --- /dev/null +++ b/testing/tests/archzfs-qemu-lts-test-00-bootfs/conf.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +export test_root_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +debug "test_root_dir='${test_root_dir}'" + +export test_target_dir='/mnt' + +export test_archzfs_repo_name="archzfs-testing" + +# Additional packages to install in the archiso +export test_archiso_packages="archzfs-linux-lts" + +# Additional packages to install in the archiso +export test_chroot_packages="$( "${arch_target_dir}/usr/bin/base.sh" +cat <<-EOF > "${arch_target_dir}/usr/bin/config.sh" echo '${fqdn}' > /etc/hostname /usr/bin/ln -s /usr/share/zoneinfo/${timezone} /etc/localtime echo 'KEYMAP=${keymap}' > /etc/vconsole.conf /usr/bin/sed -i 's/#${language}/${language}/' /etc/locale.gen /usr/bin/locale-gen - /usr/bin/sed -i 's/filesystems/zfs filesystems/' '/etc/mkinitcpio.conf' + /usr/bin/sed -i 's/filesystems/zfs filesystems/' /etc/mkinitcpio.conf /usr/bin/mkinitcpio -p linux-lts /usr/bin/usermod --password ${password} root diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/fs.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/fs.sh similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/fs.sh rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/fs.sh diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/hooks.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/hooks.sh similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/hooks.sh rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/hooks.sh diff --git a/testing/tests/archzfs-qemu-lts-test-00-bootfs/pacman.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/pacman.sh new file mode 100644 index 0000000..54a5b0a --- /dev/null +++ b/testing/tests/archzfs-qemu-lts-test-00-bootfs/pacman.sh @@ -0,0 +1,43 @@ +# Requires the pacman cache and pacman package repos be mounted via NFS +test_pacman_config() { + # $1 arch-chroot target directory + arch_target_dir="" + arch_packages="${test_archiso_packages}" + if [[ -n $1 ]]; then + arch_target_dir="${1}" + arch_chroot="/usr/bin/arch-chroot ${1}" + fi + + msg "Installing archzfs repo into chroot" + printf "\n%s\n%s\n" "[${test_archzfs_repo_name}]" "Server = file:///repo/\$repo/\$arch" >> ${arch_target_dir}/etc/pacman.conf + + msg2 "Setting up gnupg" + run_cmd "${arch_chroot} dirmngr < /dev/null" + + msg2 "Installing the signer key" + run_cmd "${arch_chroot} pacman-key -r 0EE7A126" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + run_cmd "${arch_chroot} pacman-key --lsign-key 0EE7A126" + + if [[ ! -n $1 ]]; then + msg2 "Installing test packages" + # Install the required packages in the image + run_cmd "${arch_chroot} pacman -Sy --noconfirm ${arch_packages}" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + msg2 "Loading zfs modules" + run_cmd "modprobe zfs" + fi +} + + +test_pacman_pacstrap() { + msg "bootstrapping the base installation" + run_cmd "/usr/bin/pacstrap -c '${test_target_dir}/ROOT' base base-devel ${test_chroot_packages}" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi +} diff --git a/testing/tests/archzfs-qemu-lts-test-00-zol-tests/runner.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/runner.sh similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-00-zol-tests/runner.sh rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/runner.sh diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/syslinux.cfg b/testing/tests/archzfs-qemu-lts-test-00-bootfs/syslinux.cfg similarity index 100% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/syslinux.cfg rename to testing/tests/archzfs-qemu-lts-test-00-bootfs/syslinux.cfg diff --git a/testing/tests/archzfs-qemu-lts-test-00-bootfs/vm.sh b/testing/tests/archzfs-qemu-lts-test-00-bootfs/vm.sh new file mode 100644 index 0000000..e69de29 diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/README.rst b/testing/tests/archzfs-qemu-std-test-00-bootfs/README.rst new file mode 100644 index 0000000..b9a8159 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/README.rst @@ -0,0 +1,52 @@ +========================================= +Test archzfs-qemu-lts-test-01-root-bootfs +========================================= + +Tests all the steps required for archzfs-linux-lts to be used as a boot filesystem. + +-------- +Overview +-------- + +Builds a custom archiso with the linux-lts kernel used by packer to create a Qemu base image. Syslinux is used as the boot +loader. + +--- +How +--- + +1. The archzfs-linux-lts packages are built for the linux-lts kernel and added to a package repository named "archzfs-testing". + +#. The archzfs-testing repo is shared over NFS. + +#. A custom archiso is built that boots into the linux-lts kernel. See `Archiso customization`_ + +#. The test files are compressed into a tar archive. + +#. Packer is used to build a qemu base image using the custom archiso. + +#. `setup.sh` is ran in the archiso to install arch on ZFS. + +#. After installation of Arch on ZFS, the VM is rebooted and packer finalizes the base image. + +#. The qemu base image created by packer is booted, if the boot is successful, the test is considered passed. + +--------------------- +Archiso customization +--------------------- + +At the time of putting this test together (2016.09.03), there was no stable ZFSonLinux release that supported kernel 4.7 and +the archiso release at the time shipped with kernel 4.7. In order to install Arch on ZFS for test, I needed an archiso with +the linux-lts kernel. Thus, the archzfs-archiso was born! + +The archiso is built by `test.sh` and used by packer to create a Qemu base image with ZFS as the root filesystem for testing. + +The archiso comes with a bunch of features that are not needed in test, so they have been stripped out or modified. This +includes: + +* boot straight into the linux-lts kernel to speed up the testing cycle. +* ZFS does not support arch-i686, so it was stripped. +* iPXE was not needed. + +The archiso sources are copied from `/usr/share/archiso/configs/releng` after installation of the "archiso" package. The +modifed code is contained in the `testing/archiso-linux-lts` directory of this project. diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/archiso.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/archiso.sh new file mode 100644 index 0000000..cba15ae --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/archiso.sh @@ -0,0 +1,12 @@ +# We need an archiso with the lts kernel used by default +test_build_archiso() { + msg "Building archiso" + cd ${test_root_dir}/../../../archiso/ &> /dev/null + if [[ -d ${packer_work_dir}/out ]] && [[ $(ls -1 | wc -l) -gt 0 ]]; then + run_cmd "rm -rf ${test_root_dir}/archiso/out/archlinux*" + fi + run_cmd "./build.sh -v" + msg2 "Coping archiso to packer_work_dir" + run_cmd "cp ${test_root_dir}/../../../archiso/out/archlinux* ${packer_work_dir} && rm -rf ${test_root_dir}/archiso/work" + cd - &> /dev/null +} diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/boot.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/boot.sh similarity index 93% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/boot.sh rename to testing/tests/archzfs-qemu-std-test-00-bootfs/boot.sh index 35039ce..b2fa575 100644 --- a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/boot.sh +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/boot.sh @@ -2,4 +2,5 @@ test_bootloader_install() { # Setup the boot loader run_cmd "mkdir -p ${arch_target_dir}/boot/syslinux; cp -f /root/syslinux.cfg '${arch_target_dir}/boot/syslinux/syslinux.cfg'" run_cmd "arch-chroot ${arch_target_dir} /usr/bin/syslinux-install_update -i -a -m" + run_cmd_check 1 } diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/chroot.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/chroot.sh new file mode 100644 index 0000000..029fc5f --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/chroot.sh @@ -0,0 +1,29 @@ +test_chroot_setup() { + # $1 arch-chroot target dir + msg "Setting up arch install..." + export arch_target_dir="${test_target_dir}" + if [[ -n $1 ]]; then + arch_target_dir="${1}" + fi + msg2 "Setting base image pacman mirror" + run_cmd "/usr/bin/cp /etc/pacman.d/mirrorlist ${arch_target_dir}/etc/pacman.d/mirrorlist" + + msg2 "generating the filesystem table" + run_cmd "/usr/bin/genfstab -p ${arch_target_dir} >> '${arch_target_dir}/etc/fstab'" + + # http://comments.gmane.org/gmane.linux.arch.general/48739 + msg2 "Adding workaround for shutdown race condition" + run_cmd "/usr/bin/install --mode=0644 poweroff.timer '${arch_target_dir}/etc/systemd/system/poweroff.timer'" + + msg2 "Create config.sh" + run_cmd "/usr/bin/install --mode=0755 /dev/null '${arch_target_dir}/usr/bin/config.sh'" + + # Special filesystem configure script + source_safe /root/test-config.sh + + msg2 "Entering chroot and configuring system" + run_cmd "/usr/bin/arch-chroot ${arch_target_dir} /usr/bin/config.sh" + + msg2 "Deleting config.sh" + rm ${arch_target_dir}/usr/bin/config.sh +} diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/conf.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/conf.sh new file mode 100644 index 0000000..5fdcaf7 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/conf.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +export test_root_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +debug "test_root_dir='${test_root_dir}'" + +export test_target_dir='/mnt' + +export test_archzfs_repo_name="archzfs-testing" + +# Additional packages to install in the archiso +export test_archiso_packages="archzfs-linux-lts" + +# Additional packages to install in the archiso +export test_chroot_packages="$( "${arch_target_dir}/usr/bin/config.sh" + echo '${fqdn}' > /etc/hostname + /usr/bin/ln -s /usr/share/zoneinfo/${timezone} /etc/localtime + echo 'KEYMAP=${keymap}' > /etc/vconsole.conf + /usr/bin/sed -i 's/#${language}/${language}/' /etc/locale.gen + /usr/bin/locale-gen + + /usr/bin/sed -i 's/filesystems/zfs filesystems/' /etc/mkinitcpio.conf + + /usr/bin/mkinitcpio -p linux + + /usr/bin/usermod --password ${password} root + + # https://wiki.archlinux.org/index.php/Network_Configuration#Device_names + /usr/bin/ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules + /usr/bin/ln -s '/usr/lib/systemd/system/dhcpcd@.service' '/etc/systemd/system/multi-user.target.wants/dhcpcd@eth0.service' + + # Configure ssh + sed -e '/^#PermitRootLogin prohibit-password$/c PermitRootLogin yes' \ + -e '/^#UseDNS no$/c UseDNS no' \ + -i /etc/ssh/sshd_config + + /usr/bin/systemctl enable sshd.service +EOF diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/fs.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/fs.sh new file mode 100644 index 0000000..1bafe92 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/fs.sh @@ -0,0 +1,85 @@ +test_fs_config_nfs() { + # $1 arch-chroot directory + # prefix="${test_target_dir}" + if [[ -n $1 ]]; then + prefix="${1}" + fi + + msg "Create NFS mount points" + run_cmd "/usr/bin/mkdir -p ${prefix}/repo" + + msg "Setting the package cache (nfs mount)" + run_cmd "mount -t nfs4 -o rsize=32768,wsize=32768,timeo=3 10.0.2.2:/var/cache/pacman/pkg ${prefix}/var/cache/pacman/pkg" + + msg "Mounting the AUR package repo" + run_cmd "mount -t nfs4 -o rsize=32768,wsize=32768,timeo=3 10.0.2.2:/mnt/data/pacman/repo ${prefix}/repo" +} + + +test_fs_config_root_preinstall() { + msg "Configuring root filesystem!" + + export disk='/dev/vda' + export root_partition="${disk}1" + + msg2 "Clearing partition table on ${disk}" + run_cmd "sgdisk --zap ${disk}" + + msg2 "Destroying magic strings and signatures on ${disk}" + run_cmd "dd if=/dev/zero of=${disk} bs=512 count=2048" + run_cmd "wipefs --all ${disk}" + + # See http://www.rodsbooks.com/gdisk/sgdisk-walkthrough.html + # http://www.rodsbooks.com/gdisk/sgdisk.htm + msg2 "Creating boot partition on ${disk}" + run_cmd "sgdisk --new=1:0:512M --typecode=1:8300 ${disk}" + + msg2 "Creating root partition on ${disk}" + run_cmd "sgdisk --new=2:0:0 --typecode=2:bf00 ${disk}" + + msg2 "The disk" + run_cmd "sgdisk -p ${disk}" + + msg2 "Creating root filesystem" + run_cmd "zpool create -m ${test_target_dir} -f zroot /dev/vda2" + run_cmd "zfs create -o mountpoint=none zroot/ROOT" + run_cmd "zfs create -o compression=lz4 -o mountpoint=${test_target_dir}/ROOT zroot/ROOT/default" + run_cmd "zfs create -o mountpoint=none zroot/data" + run_cmd "zfs create -o compression=lz4 -o mountpoint=${test_target_dir}/ROOT/home zroot/data/home" + run_cmd "zfs set mountpoint=legacy zroot/data/home" + + msg2 "Mounting /home" + run_cmd "mount -t zfs -o default,noatime zroot/data/home ${test_target_dir}/ROOT/home" + + msg2 "Create boot directory" + run_cmd "mkdir -p ${test_target_dir}/ROOT/boot" + + msg2 "Creating /boot filesystem (ext4)" + run_cmd "mkfs.ext4 -F -m 0 -q -L boot /dev/vda1" + + msg2 "Mounting boot filesystem" + run_cmd "mount -o noatime,errors=remount-ro /dev/vda1 ${test_target_dir}/ROOT/boot" + +} + +test_fs_config_root_postinstall() { + msg "Performing final filesystem operations" + + msg2 "Unmounting boot partition" + run_cmd "umount ${test_target_dir}/ROOT/boot" + + msg2 "Unmounting nfs partitions" + run_cmd "umount -a -t nfs4" + + msg2 "Unmounting home partition" + run_cmd "umount ${test_target_dir}/ROOT/home" + + msg2 "Setting flags and exporting ZFS root" + run_cmd "zfs umount -a" + run_cmd "zpool set bootfs=zroot/ROOT/default zroot" + run_cmd "zfs set mountpoint=none zroot" + run_cmd "zfs set mountpoint=/ zroot/ROOT/default" + run_cmd "zfs set mountpoint=/home zroot/data/home" + run_cmd "zfs set mountpoint=legacy zroot/data/home" + run_cmd "zpool export zroot" +} diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/hooks.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/hooks.sh new file mode 100644 index 0000000..8a85176 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/hooks.sh @@ -0,0 +1,12 @@ +#!/bin/bash + + +test_setup_exit() { + msg "Installation complete!" + systemctl reboot +} + + +test_met_acceptance_criteria() { + return 1 +} diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/pacman.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/pacman.sh similarity index 84% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/pacman.sh rename to testing/tests/archzfs-qemu-std-test-00-bootfs/pacman.sh index bb61577..610f4b6 100644 --- a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/pacman.sh +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/pacman.sh @@ -16,19 +16,16 @@ test_pacman_config() { msg2 "Installing the signer key" run_cmd "${arch_chroot} pacman-key -r 0EE7A126" - if [[ $? != 0 ]]; then - exit 1 - fi - run_cmd "${arch_chroot} pacman-key --lsign-key 0EE7A126" + run_cmd_check 1 + run_cmd "${arch_chroot} pacman-key --lsign-key 0EE7A126" + run_cmd_check 1 if [[ ! -n $1 ]]; then msg2 "Installing test packages" # Install the required packages in the image run_cmd "${arch_chroot} pacman -Sy --noconfirm ${arch_packages}" - if [[ $? != 0 ]]; then - exit 1 - fi + run_cmd_check 1 msg2 "Loading zfs modules" run_cmd "modprobe zfs" fi @@ -37,5 +34,6 @@ test_pacman_config() { test_pacman_pacstrap() { msg "bootstrapping the base installation" - /usr/bin/pacstrap -c ${test_target_dir}/ROOT base base-devel ${test_chroot_packages} + run_cmd "/usr/bin/pacstrap -c '${test_target_dir}/ROOT' base base-devel ${test_chroot_packages}" + run_cmd_check 1 } diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/syslinux.cfg b/testing/tests/archzfs-qemu-std-test-00-bootfs/syslinux.cfg new file mode 100644 index 0000000..a5f3edc --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-bootfs/syslinux.cfg @@ -0,0 +1,8 @@ +DEFAULT arch +TIMEOUT 0.1 + +LABEL arch + MENU LABEL Arch Linux + LINUX ../vmlinuz-linux + APPEND zfs=zroot/ROOT/default rw + INITRD ../initramfs-linux.img diff --git a/testing/tests/archzfs-qemu-std-test-00-bootfs/vm.sh b/testing/tests/archzfs-qemu-std-test-00-bootfs/vm.sh new file mode 100644 index 0000000..e69de29 diff --git a/testing/tests/archzfs-qemu-std-test-00-default/README.rst b/testing/tests/archzfs-qemu-std-test-00-default/README.rst new file mode 100644 index 0000000..9121988 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/README.rst @@ -0,0 +1,13 @@ +===================================== +Test archzfs-qemu-std-test-00-default +===================================== + +Installs archzfs onto an ext4 bootable vm and verifies a zpool mounts correctly at boot. + +-------- +Overview +-------- + +Builds a custom archiso with the linux-lts kernel used by packer to create a Qemu base image. Syslinux is used as the boot +loader. + diff --git a/testing/tests/archzfs-qemu-std-test-00-default/archiso.sh b/testing/tests/archzfs-qemu-std-test-00-default/archiso.sh new file mode 100644 index 0000000..02bb4dd --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/archiso.sh @@ -0,0 +1,16 @@ +# +# Used to configure custom built archiso +# + +# We need an archiso with the lts kernel used by default +test_build_archiso() { + msg "Building archiso" + cd ${test_root_dir}/../../../archiso/ &> /dev/null + if [[ -d ${packer_work_dir}/out ]] && [[ $(ls -1 | wc -l) -gt 0 ]]; then + run_cmd "rm -rf ${test_root_dir}/archiso/out/archlinux*" + fi + run_cmd "./build.sh -v" + msg2 "Coping archiso to packer_work_dir" + run_cmd "cp ${test_root_dir}/../../../archiso/out/archlinux* ${packer_work_dir} && rm -rf ${test_root_dir}/archiso/work" + cd - &> /dev/null +} diff --git a/testing/tests/archzfs-qemu-std-test-00-default/boot.sh b/testing/tests/archzfs-qemu-std-test-00-default/boot.sh new file mode 100644 index 0000000..307f2df --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/boot.sh @@ -0,0 +1,8 @@ +test_bootloader_install() { + # Setup the boot loader + run_cmd "mkdir -p ${arch_target_dir}/boot/syslinux; cp -f /root/syslinux.cfg '${arch_target_dir}/boot/syslinux/syslinux.cfg'" + run_cmd "arch-chroot ${arch_target_dir} /usr/bin/syslinux-install_update -i -a -m" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi +} diff --git a/testing/tests/archzfs-qemu-std-test-00-default/chroot.sh b/testing/tests/archzfs-qemu-std-test-00-default/chroot.sh new file mode 100644 index 0000000..029fc5f --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/chroot.sh @@ -0,0 +1,29 @@ +test_chroot_setup() { + # $1 arch-chroot target dir + msg "Setting up arch install..." + export arch_target_dir="${test_target_dir}" + if [[ -n $1 ]]; then + arch_target_dir="${1}" + fi + msg2 "Setting base image pacman mirror" + run_cmd "/usr/bin/cp /etc/pacman.d/mirrorlist ${arch_target_dir}/etc/pacman.d/mirrorlist" + + msg2 "generating the filesystem table" + run_cmd "/usr/bin/genfstab -p ${arch_target_dir} >> '${arch_target_dir}/etc/fstab'" + + # http://comments.gmane.org/gmane.linux.arch.general/48739 + msg2 "Adding workaround for shutdown race condition" + run_cmd "/usr/bin/install --mode=0644 poweroff.timer '${arch_target_dir}/etc/systemd/system/poweroff.timer'" + + msg2 "Create config.sh" + run_cmd "/usr/bin/install --mode=0755 /dev/null '${arch_target_dir}/usr/bin/config.sh'" + + # Special filesystem configure script + source_safe /root/test-config.sh + + msg2 "Entering chroot and configuring system" + run_cmd "/usr/bin/arch-chroot ${arch_target_dir} /usr/bin/config.sh" + + msg2 "Deleting config.sh" + rm ${arch_target_dir}/usr/bin/config.sh +} diff --git a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/conf.sh b/testing/tests/archzfs-qemu-std-test-00-default/conf.sh similarity index 61% rename from testing/tests/archzfs-qemu-lts-test-01-root-bootfs/conf.sh rename to testing/tests/archzfs-qemu-std-test-00-default/conf.sh index b827df0..3095da7 100644 --- a/testing/tests/archzfs-qemu-lts-test-01-root-bootfs/conf.sh +++ b/testing/tests/archzfs-qemu-std-test-00-default/conf.sh @@ -11,4 +11,10 @@ export test_archzfs_repo_name="archzfs-testing" export test_archiso_packages="archzfs-linux-lts" # Additional packages to install after base and base-devel -export test_chroot_packages="gptfdisk openssh syslinux archzfs-linux-lts" +export test_chroot_packages="$( "${arch_target_dir}/usr/bin/config.sh" + echo '${fqdn}' > /etc/hostname + /usr/bin/ln -s /usr/share/zoneinfo/${timezone} /etc/localtime + echo 'KEYMAP=${keymap}' > /etc/vconsole.conf + /usr/bin/sed -i 's/#${language}/${language}/' /etc/locale.gen + /usr/bin/locale-gen + + /usr/bin/mkinitcpio -p linux + /usr/bin/usermod --password ${password} root + + # https://wiki.archlinux.org/index.php/Network_Configuration#Device_names + /usr/bin/ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules + /usr/bin/ln -s '/usr/lib/systemd/system/dhcpcd@.service' '/etc/systemd/system/multi-user.target.wants/dhcpcd@eth0.service' + + # Configure ssh + sed -e '/^#PermitRootLogin prohibit-password$/c PermitRootLogin yes' \ + -e '/^#UseDNS no$/c UseDNS no' \ + -i /etc/ssh/sshd_config + + /usr/bin/systemctl enable sshd.service +EOF diff --git a/testing/tests/archzfs-qemu-std-test-00-default/fs.sh b/testing/tests/archzfs-qemu-std-test-00-default/fs.sh new file mode 100644 index 0000000..b2752dd --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/fs.sh @@ -0,0 +1,59 @@ +test_fs_config_nfs() { + # $1 arch-chroot directory + if [[ -n $1 ]]; then + prefix="${1}" + fi + + msg "Create NFS mount points" + run_cmd "/usr/bin/mkdir -p ${prefix}/repo" + + msg "Setting the package cache (nfs mount)" + run_cmd "mount -t nfs4 -o rsize=32768,wsize=32768,timeo=3 10.0.2.2:/var/cache/pacman/pkg ${prefix}/var/cache/pacman/pkg" + + msg "Mounting the AUR package repo" + run_cmd "mount -t nfs4 -o rsize=32768,wsize=32768,timeo=3 10.0.2.2:/mnt/data/pacman/repo ${prefix}/repo" +} + + +test_fs_config_root_preinstall() { + msg "Configuring root filesystem!" + + export disk='/dev/vda' + export root_partition="${disk}1" + + msg2 "Clearing partition table on ${disk}" + run_cmd "sgdisk --zap ${disk}" + + msg2 "Destroying magic strings and signatures on ${disk}" + run_cmd "dd if=/dev/zero of=${disk} bs=512 count=2048" + run_cmd "wipefs --all ${disk}" + + # See http://www.rodsbooks.com/gdisk/sgdisk-walkthrough.html + # http://www.rodsbooks.com/gdisk/sgdisk.htm + msg2 "Creating boot partition on ${disk}" + run_cmd "sgdisk --new=1:0:0 --typecode=1:8300 ${disk}" + + msg2 "Making root partition bootable" + run_cmd "sgdisk --attributes=1:set:2 ${disk}" + + msg2 "The disk" + run_cmd "sgdisk -p ${disk}" + + msg2 "Creating / filesystem (ext4)" + run_cmd "mkfs.ext4 -O ^64bit -F -m 0 -q -L root /dev/vda1" + + msg2 "Mounting root filesystem" + run_cmd "/usr/bin/mkdir -p ${test_target_dir}/ROOT" + run_cmd "mount -o noatime,errors=remount-ro /dev/vda1 ${test_target_dir}/ROOT/" +} + + +test_fs_config_root_postinstall() { + msg "Performing final filesystem operations" + + msg2 "Unmounting nfs partitions" + run_cmd "umount -a -t nfs4" + + msg2 "Unmounting root partition" + run_cmd "umount ${test_target_dir}/ROOT" +} diff --git a/testing/tests/archzfs-qemu-std-test-00-default/hooks.sh b/testing/tests/archzfs-qemu-std-test-00-default/hooks.sh new file mode 100644 index 0000000..8a85176 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/hooks.sh @@ -0,0 +1,12 @@ +#!/bin/bash + + +test_setup_exit() { + msg "Installation complete!" + systemctl reboot +} + + +test_met_acceptance_criteria() { + return 1 +} diff --git a/testing/tests/archzfs-qemu-std-test-00-default/pacman.sh b/testing/tests/archzfs-qemu-std-test-00-default/pacman.sh new file mode 100644 index 0000000..568144a --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/pacman.sh @@ -0,0 +1,50 @@ +# Requires the pacman cache and pacman package repos be mounted via NFS +test_pacman_config() { + # $1 arch-chroot target directory + arch_target_dir="" + arch_packages="${test_archiso_packages}" + if [[ -n $1 ]]; then + arch_target_dir="${1}" + arch_chroot="/usr/bin/arch-chroot ${1}" + fi + + msg "Installing archzfs repo into chroot" + printf "\n%s\n%s\n" "[${test_archzfs_repo_name}]" "Server = file:///repo/\$repo/\$arch" >> ${arch_target_dir}/etc/pacman.conf + + msg2 "Setting up gnupg" + run_cmd "${arch_chroot} dirmngr < /dev/null" + + msg2 "Installing the signer key" + run_cmd "${arch_chroot} pacman-key -r 0EE7A126" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + run_cmd "${arch_chroot} pacman-key --lsign-key 0EE7A126" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + + + if [[ ! -n $1 ]]; then + # Install the required packages in the image + msg2 "Installing test packages" + run_cmd "${arch_chroot} pacman -Sy --noconfirm ${arch_packages}" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + msg2 "Loading zfs modules" + run_cmd "modprobe zfs" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi + fi +} + + +test_pacman_pacstrap() { + msg "bootstrapping the base installation" + run_cmd "/usr/bin/pacstrap -c '${test_target_dir}/ROOT' base base-devel ${test_chroot_packages}" + if [[ ${run_cmd_return} -ne 0 ]]; then + exit 1 + fi +} diff --git a/testing/tests/archzfs-qemu-std-test-00-default/syslinux.cfg b/testing/tests/archzfs-qemu-std-test-00-default/syslinux.cfg new file mode 100644 index 0000000..c31e892 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/syslinux.cfg @@ -0,0 +1,8 @@ +DEFAULT arch +TIMEOUT 0.1 + +LABEL arch + MENU LABEL Arch Linux + APPEND root=/dev/vda1 rw + LINUX /boot/vmlinuz-linux + INITRD /boot/initramfs-linux.img diff --git a/testing/tests/archzfs-qemu-std-test-00-default/vm.sh b/testing/tests/archzfs-qemu-std-test-00-default/vm.sh new file mode 100644 index 0000000..ff59e06 --- /dev/null +++ b/testing/tests/archzfs-qemu-std-test-00-default/vm.sh @@ -0,0 +1,46 @@ +# +test_vm_boot() { + # $1 the image to boot + # /usr/bin/qemu-system-x86_64 -device virtio-net,netdev=user.0 \ + # -drive file=testing/files/packer_work/output-qemu/archzfs-qemu-std-test-00-default-archiso-2016.09.10,if=virtio,cache=writeback,discard=ignore \ + # -vnc 0.0.0.0:32 -netdev user,id=user.0,hostfwd=tcp::3333-:22 \ + # -name archzfs-qemu-std-test-00-default-archiso-2016.09.10 -machine type=pc,accel=kvm -display sdl -boot once=d -m 512M +} + +# Configure +test_vm_config_zfs_zpool() { + + # ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@localhost -p 3543 + + # TODO NEED TO CREATE NEW QEMU VOL + + # zpool create -m /bigdata -f zroot /dev/vdb1 + + + # run_cmd "zfs create -o mountpoint=none zroot/ROOT" + # run_cmd "zfs create -o compression=lz4 -o mountpoint=${test_target_dir}/ROOT zroot/ROOT/default" + # run_cmd "zfs create -o mountpoint=none zroot/data" + # run_cmd "zfs create -o compression=lz4 -o mountpoint=${test_target_dir}/ROOT/home zroot/data/home" + # run_cmd "zfs set mountpoint=legacy zroot/data/home" + + # msg2 "Mounting /home" + # run_cmd "mount -t zfs -o default,noatime zroot/data/home ${test_target_dir}/ROOT/home" + + # msg2 "Create boot directory" + + # run_cmd "mkdir -p ${test_target_dir}/ROOT/boot" + + # msg2 "Unmounting home partition" + # run_cmd "umount ${test_target_dir}/ROOT/home" + + # msg2 "Setting flags and exporting ZFS root" + # run_cmd "zfs umount -a" + # run_cmd "zpool set bootfs=zroot/ROOT/default zroot" + # run_cmd "zfs set mountpoint=none zroot" + # run_cmd "zfs set mountpoint=/ zroot/ROOT/default" + # run_cmd "zfs set mountpoint=/home zroot/data/home" + # run_cmd "zfs set mountpoint=legacy zroot/data/home" + # run_cmd "zpool export zroot" +} + +