mirror
/
archzfs
mirror of https://github.com/archzfs/archzfs
2
0
Fork 0
Code Issues Releases Wiki Activity

web_archzfs: Rewrite

Browse Source
pull/7/head
Jesus Alvarez 12 years ago
parent 0f66de6fc6
commit b544ef442b
1 changed files with 516 additions and 183 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 699
      web_archzfs.rst

699
web_archzfs.rst
Unescape View File

@ -1,78 +1,60 @@
======================================
Arch ZFS - ZFS On Linux Kernel Modules
======================================
:Modified: Thu Dec 13 20:24:33 PST 2012
:status: hidden
:slug: archzfs
This is the official web page of the Arch ZFS kernel modules for ZFS on Linux.
Here you can find makepkg sources for building Arch ZFS yourself, or pre-built
x86_64 packages. You can also add the unofficial repository to your
makepkg.conf and receive automatic updates. For the source to the build script
used to generate these packages, see archzfs-github_.
This is the official web page of the Arch ZFS kernel module packages for native
ZFS on Linux. Here you can find pacman package sources and pre-built x86_64
packages. For effortless package installation and updates, it is possible to
add the unofficial repository to your pacman.conf. There is also a special
repository for using ZFS with the archiso install media for installing arch
onto a ZFS root filesystem, or doing emergency maintenance. To see the package
sources and repository development history, see archzfs-github_.
**Please note**, if you install these binaries, you will not be able to update
to the next kernel versions without first having available the latest compiled
ZFS package for that update. The current version of Arch ZFS, 0.6.0-rc12, can
only be used with kernel version 3.6.6. Once 3.6.7 is released, ZFS on Linux
will have to be recompiled to target 3.6.7.
.. note:: The ZFS and SPL packages are depend on a specific kernel
version. You will not be able to perform kernel updates until updated
ZFS packages are pushed to the archzfs repository for the new kernel
version. If you installing ZFS manually using the AUR packages, you
would be required to first un-install ZFS, perform the kernel update,
restart the host, and then build and install the updated AUR ZFS
packages.
For more information about the packager, Jesus Alvarez, see demizerone.com_. My
packages and database are signed by my PGP key, but my key is not signed by any
of the Arch Linux master keys, so you will have to locally sign my key and add
it to your trust, see below for more info.
32bit support for ZFS on Linux is unstable due to inconsistencies in memory
management between the Solaris kernel and the Linux kernel. For this reason,
**the ZFS packages for Arch Linux do not yet support i686**. However, 32bit
support will be added in the future for those brave enough to face the
consequences. See `ZFS on Linux FAQ - 64bit`_
My arch linux contributions
---------------------------
* `AUR - Packages`_
* `ArchWiki - Edits`_
* `bbs.archlinux.org - Posts`_ (Search for demizer)
* github_
My Unofficial Arch Linux Repository
-----------------------------------
If you are adding my repository to pacman, you will first need to un-install all
versions of zfs.
Stop any services accessing your drives, like nfs,
.. code-block:: console
The archzfs repository and packages are signed, but the key is not trusted by
any of the Arch Linux master keys. You will have to locally sign the key and
add it to your trust. See below for more info.
# systemctl stop nfsd.service
For more information about the packager, Jesus Alvarez, see demizerone.com_.
Then umount your ZFS filesystem,
------------------------------------
The archzfs un-official repositories
------------------------------------
.. code-block:: console
# umount /mnt/data
Remove the old version of zfs,
.. code-block:: console
# pacman -Rs zfs
This repository is updated on every kernel release. This allows for effortless
installation and updates.
Next, add the following to `/etc/pacman.conf`:
To start, add the server information to `/etc/pacman.conf`,
.. code-block:: bash
[archzfs]
SigLevel = Required DatabaseOptional TrustedOnly
Server = http://demizerone.com/$repo/$arch
Both the database and the packages are signed, so you will have to add my key
to pacman's trusted key list.
Both the database and the packages are signed, so you will have to add the
signing key to pacman's trusted key list:
.. code-block:: console
# pacman-key -r 0EE7A126
verify it using the info below and then sign it with the local master key:
verify it using the info below and then sign it with the local master key
.. code-block:: console
@ -84,27 +66,36 @@ next, update your pacman database
# pacman -Syy
and install the package group,
and finally, install the package group
.. code-block:: console
# pacman -S zfs
# pacman -S archzfs
Notes
-----
.. note:: To read about key management in Arch, see pacman-key_ and
pacman.conf_
* My key is not signed by any of the master keys, so you will have to self sign
it with your local master key. This page is hopefully an attempt to persuade
you that it is legit and I mean no harm. Look below on how to verify my key.
ZFS support for archiso
=======================
* To read about key management in Arch, see
`pacman-key <https://wiki.archlinux.org/index.php/Pacman-key>`_ and
`pacman.conf <https://www.archlinux.org/pacman/pacman.conf.5.html#_package_and_database_signature_checking>`_.
If emergency maintenance is ever required on a ZFS file system from an archiso
live environment, you will need to use the archzfs repository that tracks the
current archiso release. To use it, follow the steps above for accessing the
archzfs repository, but instead use the special server line below when adding
the server information to pacman.conf:
My PGP Key
----------
.. code-block:: bash
[archzfs]
Server = http://demizerone.com/$repo/archiso/$arch
All of my packages and package databases are signed with the following key,
-----------------------
Signing key for archzfs
-----------------------
The ZFS packages and database are signed with the package maintainer's key. The
current maintainer is Jesus Alvarez and his key can be verified at
demizerone.com_ This key is not trusted by any of the Arch Linux Master Keys.
0EE7A126_
~~~~~~~~~
@ -146,227 +137,569 @@ and the long version::
=TUqo
-----END PGP PUBLIC KEY BLOCK-----
ZFS update procedure
====================
---------------------
Insalling ZFS on ROOT
---------------------
https://github.com/dajhorn/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem
ZFS Cheatsheet: http://lildude.co.uk/zfs-cheatsheet
1. Create live usb for UEFI: https://wiki.archlinux.org/index.php/UEFI#Create_UEFI_bootable_USB_from_ISO
#. Boot from live usb.
#. Use cgdisk and create a GPT partition table
=============================
Part Size Type
==== ===== =============
1 512M EFI (ef00)
2 512M Ext4 (8200)
2 117G Solaris Root (bf00)
=============================
Note the EFI partion will contain the kernel images
#. Format the EFI partion fat32
mkfs.vfat -F 32 /dev/sda1 -n EFIBOOT
#. Format the Ext4 boot partition
mkfs.ext4 /dev/sda2 -L BOOT
#. Check /etc/pacman.d/mirrorlist and make sure the mirrors are agreeable.
#. Add the archzfs repo to pacman.conf
[archzfs]
SigLevel = Required DatabaseOptional TrustedOnly
Server = http://demizerone.com/$repo/$arch
#. Connect to the internet
wifi-menu
#. Install archzfs key
pacman-key -r 0EE7A126
pacman-key --lsign-key 0EE7A126
#. Update pacman
pacman -Syy
#. Install zfs
pacman -S archzfs
#. Load the modules
modprobe zfs
#. Create zfs pool
# zpool create rpool /dev/disk/by-id/<id>
Always use id names when working with zfs, otherwise import errors will
occur.
#. Create zfs file systems
Create the root filesystem
# zfs create rpool/ROOT
create the decendent file system that will hold the installation:
# zfs create rpool/ROOT/arch
We will set the mountpoints after we have created the filesystems so that
they are not mounted automatically to occupied directories causing errors.
Note: If you like you can create sub-filesystem mount points here such as
/home and /root by doing the following:
# zfs create rpool/HOME
# zfs create rpool/HOME/root
#. Umount all zfs filesystems
# zfs umount -a
#. Set the mount point for the decendent root filesystem
# zfs set mountpoint=/ rpool/ROOT/arch
optionally,
# zfs set mountpoint=/home rpool/HOME
# zfs set mountpoint=/root rpool/HOME/root
#. Set the bootfs property on the decendent root filesystem so the bootloader
knows where to find the operating system.
# zpool set bootfs=rpool/ROOT/arch rpool
#. Export the pool
# zpool export rpool
Don't skip this, otherwise you will be required to use -f when importing
your pools. This unloads the imported pool.
Note: Ubuntu help says if this command isn't used, the system will be in an
incossistant state. The docs say that this allows the pools to be shared
accross systems. Is this why I had to use -f when creating the pools the
last time?
#. Re-import the pool
# zpool import -d /dev/disk/by-id -R /mnt rpool
Note: -d is not the actual device id, but the by-id directory containing
the symlinks.
If there is an error in this step, you can export the pool to redo the
command:
# zpool export rpool
#. Mount the EFI and boot partition
mkdir /mnt/boot
mount /dev/sda2 /mnt/boot
mkdir /mnt/boot/efi
mount /dev/sda1 /mnt/boot/efi
#. Install base packages
pacstrap -i /mnt base base-devel archzfs
#. Generate the fstab
# genfstab -U -p /mnt >> /mnt/etc/fstab
#. Open fstab to edit contents
# nano /mnt/etc/fstab
This is the procedure used to update the zfs package versions. This could be
caused by a new zfs release version or a kernel update.
Delete all the lines except for the boot partion. ZFS auto mounts it's own
partitions.
1. Unmount all zfs pools
#. Load the efivars module
.. code-block:: console
modprobe efivars
# systemctl stop zfs
#. Chroot into the installation
If there is a problem unmounting the drive, such as "target is busy", you
can see what process is using the mount by using fuser.
arch-chroot /mnt /bin/bash
.. code-block:: console
# Install a real text editor
# fuser /mnt/data
# sudo fuser -v /mnt/data
# pacman -S vim
USER PID ACCESS COMMAND
/mnt/data: root kernel mount /mnt/data
#. Follow https://wiki.archlinux.org/index.php/Beginners%27_Guide from the
Locale section to the Configure Pacman Section
This directory is exported by nfs, so we'll have to stop the nfs server
before unmounting.
#. Edit pacman.conf and add the archzfs repository. If on arch64, uncomment the
multilib repo.
.. code-block:: console
#. Update the pacman database
# systemctl stop nfsd
pacman -Syy
#. Remove the old zfs version:
#. Create the initramfs, edit mkinitcpio.conf and add zfs before filesystems.
Remove fsck and then regen the initramfs:
.. code-block:: console
mkinitcpio -p linux
# pacman -R archzfs
#. Set root passwd and add a regular user.
#. Update all packages and restart:
#. Install UEFI boot loader
.. code-block:: console
Continuing from the EFISTUB section at
https://wiki.archlinux.org/index.php/Beginners'_Guide#Chroot_and_configure_the_base_system
# pacman -Syu
# systemctl restart
# mkdir /boot/efi
# modprobe efivars
# arch-chroot /mnt /bin/bash
# mkdir -p /boot/efi/EFI/arch
# cp /boot/vmlinuz-linux /boot/efi/EFI/arch/vmlinuz-arch.efi
# cp /boot/initramfs-linux.img /boot/efi/EFI/arch/initramfs-arch.img
# cp /boot/initramfs-linux-fallback.img /boot/efi/EFI/arch/initramfs-arch-fallback.img
#. Update PKGBUILD.py
The images will need to be recopied everytime there is an update, see
https://wiki.archlinux.org/index.php/Beginners'_Guide#EFISTUB for more
information.
* Change ``PACKAGE_REL``
#. Install rEFInd
* Change ``SOURCE_VER``
# pacman -S refind-efi efibootmgr
# mkdir -p /boot/efi/EFI/refind
# cp /usr/lib/refind/refindx64.efi /boot/efi/EFI/refind/refindx64.efi
# cp /usr/lib/refind/config/refind.conf /boot/efi/EFI/refind/refind.conf
# cp -r /usr/share/refind/icons /boot/efi/EFI/refind/icons
* Update ``MD5SUMS`` with ``./build.py -m``
# nano /boot/efi/EFI/arch/refind_linux.conf
"Boot to X" "root=PARTUUID=<id> zfs=bootfs ro rootfstype=ext4 systemd.unit=graphical.target"
"Boot to Console" "root=PARTUUID=<id> zfs=bootfs ro rootfstype=ext4 systemd.unit=multi-user.target"
#. Build the new packages and repo:
#. Add rEFInd to the UEFI boot menu
.. code-block:: console
# efibootmgr -c -g -d /dev/sdX -p Y -w -L "rEFInd" -l '\EFI\refind\refindx64.efi'
./build.py -bxs split
Note: In the above command, X and Y denote the drive and partition of the
UEFISYS partition. For example, in /dev/sdc5, X is "c" and Y is "5".
#. Add PKGBUILD.py and archzfs/ to the index
To delete an existing boot menu item,
#. Commit the changes with
# efibootmgr
.. code-block:: console
Lists the menu items and
gc -m "Update to 0.6.0-rc12\_6-linux-3.6.9"
# efibootmgr -b D -B
Note: "\_6" is the pkgrel version.
deletes.
#. Tag the commit on the master branch
#. Unmount and restart
.. code-block:: console
# exit
# umount /mnt/boot
# zfs umount -a
# zpool export rpool
# reboot
git tag 0.6.0-rc12\_6-linux-3.6.9 -as -m "Support for zfs-0.6.0-rc12\_6 and Kernel 3.6.9"
Emergency chroot repair with archzfs
====================================
#. Go to the demizerone.com repo and "make publish"
Here is how to use the archiso to get into your ZFS filesystem.
#. Push the changes:
1. Boot the latest archiso.
.. code-block:: console
#. Bring up your network
git push web
wifi-menu
#. Testing
or
* Uninstall the built packages:
ip link set eth0 up
.. code-block:: console
#. Test network
# pacman -Rs zfs
ping google.com
* Install the repo packages to test the repo:
#. Sync pacman package database
.. code-block:: console
pacman -Syy
# pacman -Syy
# pacman -S zfs
#. (optional) Install a better text editor:
#. The entire process went very smoothly.
pacman -S vim
#. AUR COMMENT:
#. Add archzfs archiso repository to pacman.conf
The packages have been updated for kernel 3.6.7.
[archzfs]
SigLevel = Required DatabaseOptional TrustedOnly
Server = http://demizerone.com/$repo/archiso/$arch/
If you installed the packages from AUR, you will need to first remove the zfs
and spl packages:
#. Sync the pacman package database
# pacman -Rsc spl-utils
pacman -Syy
and then update the kernel:
#. Install archzfs
# pacman -S linux linux-headers
pacman -S archzfs
You will now have to restart your system.
#. Load the kernel modules
Once your system is back up, you can proceed with building and installing zfs
and spl, in the following order: spl-utils, spl, zfs-utils, and zfs. Then
restart, or:
modprobe zfs
# modprobe zfs spl
#. Import your pool
You could also use the prebuilt signed repository available at
http://demizerone.com/archzfs and you will not have to remove the packages,
update the kernel, and restart before performing the update.
zpool import -a -R /mnt
Also, these new packages now have a group, 'arch-zfs'. So next time you could
remove the packages with just:
#. Mount your boot partitions (if you have them)
# pacman -R arch-zfs
mount /dev/sda2 /mnt/boot
mount /dev/sda1 /mnt/boot/efi
If usig the signed repository, you can now install all the packages with:
#. Chroot into your zfs filesystem
# pacman -S arch-zfs
arch-chroot /mnt /bin/bash
Git merge Arch ZFS repo_support to master and tag
=================================================
:Added: Thu Nov 15 12:52:21 PST 2012
:Tags: git, merge, tag
#. Check your kernel version
1. git co master
pacman -Qi linux
uname -r
#. git tag -s 0.6.0-rc11
uname will show the kernel version of the archiso. If they are different,
you will need to run depmod (in the chroot) with the correct kernel version
of your chroot installation:
#. git merge --no-ff repo_support
depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux)
#. I messed up and want to undo the merge
This will load the correct kernel modules for the kernel version installed
in your chroot installation.
git reset --hard 6f764ef
#. Regenerate your ramdisk
#. git co repo_support
mkinitcpio -p linux
#. Undo merge
There should be no errors.
git reset --hard 3115cb5
--------------------
ZFS update procedure
--------------------
This is the procedure the ZFS package maintainer should use to update the ZFS
package versions on the development host. This could be provoked by a new ZFS
release version or a kernel update.
Unmount all zfs pools
=====================
#. git merge --no-ff 0.6.0-rc12
.. code-block:: console
# systemctl stop zfs
#. git br -d 0.6.0-rc12
If there is a problem unmounting the drive, such as "target is busy", you can
see what process is using the mount by using fuser.
#. git br -m develop repo_support
.. code-block:: console
# fuser /mnt/data
# sudo fuser -v /mnt/data
USER PID ACCESS COMMAND
/mnt/data: root kernel mount /mnt/data
This directory is exported by nfs, so we'll have to stop the nfs server before
unmounting.
.. code-block:: console
#. git co master
# systemctl stop nfsd
#. git merge --no-ff develop
Remove the old ZFS version
==========================
#. I need to rename the old 0.6.0-rc11 tag
.. code-block:: console
git co 0.6.0-rc11
git tag 0.6.0-rc11 0.6.0-rc11-linux-3.5.6
git tag -d 0.6.0-rc11
# pacman -R archzfs
#. Switch back to previous development
Perform pacman update and restart
=================================
git co logging_support
git rebase develop
git stash pop
git reset HEAD
.. code-block:: console
# pacman -Syu
# systemctl restart
Create a new branch in git
==========================
Building arch-zfs
=================
The new git branch should be name for the current version of the ZFS on Linux
project and the Linux Kernel version it will target.
.. code-block:: console
$ ./build.py -bnxs split
$ git checkout -b zfs-0.6.0-rc12_linux-3.7
Update the ZFS PKGBUILDs
========================
1. Change ``pkgrel``.
#. Change the kernel versions to the targeted kernel version.
#. Update ``md5sums`` with ``makepkg -g``.
1. Use namcap.
This step is only necessary if the upstream ZFS version has changed. If this
is the case, the ``pkgrel`` should also be changed to ``1``.
#. Inspect package files
Building archzfs
================
Go into each package directory in order: spl-utils, spl, zfs-utils, zfs and use
makepkg to build the packages:
.. code-block:: console
$ makepkg -sfi
.. note:: If either SPL or ZFS do not build due to kernel incompatibilities,
patches will be needed to allow building to continue. See `Patching
ZFS`_.
Load the ZFS kernel modules
---------------------------
.. code-block:: console
# modprobe zfs
Import and mount your ZFS pools
-------------------------------
.. code-block:: console
# zpool import -a
# zfs mount -a
Inspect package files
---------------------
.. code-block:: console
$ pacman -Qnp <package>
#. Add packages to repository
Add packages to repository
--------------------------
This is done using the ``repo_add.py`` python script for efficiency. It can be
found `here <https://github.com/demizer/binfiles>`_.
.. code-block:: console
$ ./build.py -Rx
$ repo_add.py -r archzfs -d
#. Remove install packages, reboot, reinstall from repository, and reboot to
make sure they are excepted.
Testing
-------
#. Make sure the kernel modules are loaded
Reboot to make sure the ZFS packages are used after a system boot and the
systemd file is in working order. Also sync the updates to other local systems
to make sure the updated packages are picked up by pacman and install properly.
Commit changes to git
---------------------
Add PKGBUILD.py and archzfs/ to the index and commit the changes with
.. code-block:: console
$ lsmod | grep spl
$ lsmod | grep zfs
git commit -m "Update to ZFS version 0.6.0-rc12-8 and linux-3.7"
Updating the repository
=======================
.. note:: "-8" at the end of the ZFS version is the pkgrel.
Now tag the commit on the master branch
.. code-block:: console
git tag 0.6.0-rc12\_6-linux-3.6.9 -as -m "Support for zfs-0.6.0-rc12\_6 and Kernel 3.6.9"
Update the webpage
==================
Open the command terminal and cd to the webpage repository powered by Pelican.
Use make to generate the updated website:
.. code-block:: console
make publish
then push the changes with rsync,
.. code-block:: console
./push_archzfs.sh -n
'-n' is used to verify the files being pushed are correct. Once that is done,
re-use the command without the dry-run argument.
Anoucement template
===================
AUR
---
The packages have been updated for kernel 3.6.7.
If you installed the packages from AUR, you will need to first remove the zfs
and spl packages:
# pacman -Rsc spl-utils
and then update the kernel:
# pacman -S linux linux-headers
You will now have to restart your system.
Once your system is back up, you can proceed with building and installing zfs
and spl, in the following order: spl-utils, spl, zfs-utils, and zfs. Then
restart, or:
# modprobe zfs spl
You could also use the prebuilt signed repository available at
http://demizerone.com/archzfs and you will not have to remove the packages,
update the kernel, and restart before performing the update.
Also, these new packages now have a group, 'arch-zfs'. So next time you could
remove the packages with just:
# pacman -R arch-zfs
If usig the signed repository, you can now install all the packages with:
# pacman -S arch-zfs
.. _Patching ZFS:
Creating a patch for ZFS
========================
On some occasions, a new kernel version is pushed to the [core] repository
that the latest ZFS on Linux release does not build against. The biggest
problem with this is that the master branch of the ZFS on Linux repository
already contains the required build fixes, but the next release could be weeks
away, causing the packages in AUR to be flagged out of date for that period of
time.
The goal of this section is to document the procedure for creating a patch to
bring the release version up-to-date with the latest kernel so that the AUR
packages do not remain out of date. Otherwise, the user would have to
un-install the current AUR packages and install special 'zfs-git' packages
until the next ZFS on Linux release is made and then switch back to the
standard ZFS AUR packages.
.. note:: The ZFS and SPL projects track each other. If either package requires
a patch, then both projects should be patched. Each project is split
into two packages for Arch Linux so the patch must be applied to both
packages for each project.
.. code-block:: console
$ git clone https://github.com/zfsonlinux/zfs.git
Once the repository is cloned, create a branch.
.. code-block:: console
$ git checkout -b archzfs_patch
Revert the head to the last release.
.. code-block:: console
$ git reset --hard <commit>
Merge the master branch into the archzfs_patch branch.
.. code-block:: console
$ git merge --squash master
Finally, generate the new patch.
.. code-block:: console
$ cd ~/projects/html/demizerone.com
$ make publish
$ git diff --cached > ../linux-3.7.patch
.. _archzfs-github: https://github.com/demizer/archzfs
.. _demizerone.com: http://demizerone.com
.. _0EE7A126: http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x5E1ABF240EE7A126
.. _ArchWiki - Edits: https://wiki.archlinux.org/index.php/Special:Contributions/Demizer
.. _bbs.archlinux.org - Posts: https://bbs.archlinux.org/search.php
.. _AUR - Packages: https://aur.archlinux.org/packages/?O=0&C=0&SeB=m&K=demizer&outdated=&SB=n&SO=a&PP=50&do_Search=Go
.. _github: http://www.github.com/demizer
.. _pacman-key: https://wiki.archlinux.org/index.php/Pacman-key
.. _pacman.conf: https://www.archlinux.org/pacman/pacman.conf.5.html#_package_and_database_signature_checking
.. _ZFS on Linux FAQ - 64bit: http://zfsonlinux.org/faq.html#WhyShouldIUseA64BitSystem

Write Preview
Loading…
Cancel
Save