These notes should be read in conjunction with the README notes.

=== Generic Setup Steps for a PCP QA VM ===

(a) get the smallest possible boot media, install and boot
(b) check hostname, timezone and date-time are correct
(c) setup DNS: my server is bozo (192.168.1.100) and needs the VM to
    have localdomain in the domain search list
(d) change from DHCP (if setup during install) to fixed IP addresses,
    as per the DNS configured on bozo (other VMs need to be able to find
    you, and my local DHCP is not hooked into the DNS)
(e) if the install was done with a user different to the one you're going
    to run QA with, you need to set up the QA user
(f) make sure the user you're going to run QA with can sudo without
    a password
    For the QA Farm
      $ su - root
      # which sudo || apt install sudo
      # EDITOR=vi visudo
      add NOPASSWD: tag before last ALL in %sudo (or similar line), e.g.
      %sudo   ALL=(ALL:ALL) NOPASSWD:ALL
      # vi /etc/group and make sure kenj and pcpqa are in the sudo:x:... line
      if /etc/group needed changing, logout and login again, then
      $ sudo id
      should not prompt for a password and return uid=0 and gid=0
(g) git clone the PCP tree
    For the QA Farm
      $ cd
      $ sudo apt install git
      $ [ -d src ] || mkdir src
      $ cd src
      $ git clone bozo:git-mirror/pcp.git
      $ cd pcp
(h) make sure there is no firewall, or if there is one the ports mentioned
    in qa/README (pmcd, pmlogger control, pmproxy, etc) are open
    Use the scripts/smack-firewall script, or ...
    In the simplest case with ufw(1) in play, the following should suffice:
      # see if it is Active
      $ sudo ufw status
      # and if so, ...
      $ sudo ufw allow 44321/tcp comment pmcd
      $ sudo ufw allow 44322/tcp comment pmproxy
      $ sudo ufw allow 44323/tcp comment pmwebapi
      $ sudo ufw allow 5353/udp comment mDNS
      $ for port in `seq 4320 4350`
          do sudo ufw allow $port/tcp comment PCPQA
	done
    If firewalld(1) is in charge, this should work:
      $ sudo firewall-cmd --get-default-zone
      # save this one for use below
      zone=....
      $ sudo firewall-cmd --zone=$zone --add-service=pmcd --permanent
      $ sudo firewall-cmd --zone=$zone --add-service=pmproxy --permanent
      $ sudo firewall-cmd --zone=$zone --add-service=pmwebapi --permanent
      # and for https://...
      $ sudo firewall-cmd --zone=$zone --add-service=pmwebapis --permanent
      $ sudo firewall-cmd --zone=$zone --add-service=mdns --permanent
      $ sudo firewall-cmd --zone=$zone --add-port=4320-4350/tcp --permanent
      $ sudo systemctl restart firewalld
      # check
      $ sudo firewall-cmd --zone=$zone --list-services
      $ sudo firewall-cmd --zone=$zone --list-ports
(i) cd qa/admin and run list-packages -m until you're tired ...
    - the files in qa/admin/package-lists are the inventories for each
      distribution, version and architecture we know about ... see the
      README in that directory if you need to create a new package-list
      file ... and send it back to us for inclusion, please

If you're NOT going to run any of the -g remote tests, then you don't need
steps (c) or (d).

The specific details for (b), (e), (f) and (g) are platform and distro
version specific I'm afraid ... good luck with that!

=== Distro Releases and Packaging Hints ===

dpkg
    # purge removed packages
    $ dpkg --list |grep "^rc" | cut -d " " -f 3 | xargs sudo dpkg --purge
    # largest installed packages on disk
    $ dpkg-query -Wf '${db:Status-Status} ${Installed-Size}\t${Package}\n' | sed -ne 's/^installed //p' | sort -n

rpm
    # largest installed packages on disk
    $ rpm -qa --queryformat '%{SIZE} %{NAME} \n' | sort -n

Ubuntu
    https://wiki.ubuntu.com/Releases
    [just the LTS releases to end of standard support]
    11 Sep 2025: 24.04 (Noble), 22.04 (Jammy)
    # which packages are installed
    $ dpkg-query -W -f '${package}\n'
    # package search recipe (name and info or description)
    $ dpkg-query -l <pattern>
    # which package installs a file/directory (only installed packages)
    $ dpkg-query -S file-or-dir-path
    # to find package for an uninstalled file/directory
    https://packages.debian.org/search
    # which files are in an installed package
    $ dpkg -l <package>
    # package install recipe
    $ sudo apt install <package>
    # update all packages recipe
    $ sudo apt update; sudo apt dist-upgrade
    # release upgrade recipe ... update, then
    $ sudo do-release-upgrade

Debian
    https://wiki.debian.org/DebianReleases
    08 Sep 2025: 12 (Bookworm) and 13 (Trixie)
    # see Ubuntu for most notes
    #
    # release upgrade recipe
    # make sure everything is up to date
    $ sudo apt-get update; sudo apt-get -y upgrade
    # In a text editor, replace the codename of your release with that
    # of the next release in APT's package sources
    $ sudo vi /etc/apt/sources.list /etc/apt/sources.list.d/*
    # For instance, the line
    #    deb https://deb.debian.org/debian/ buster main
    # should be replaced with
    #    deb https://deb.debian.org/debian/ bullseye main
    # Debian 12 == bookworm
    # Debian 13 == trixie
    # Debian 14 == forky
    $ sudo apt-get clean; sudo apt-get update
    $ sudo DEBIAN_FRONTEND=noninteractive apt-get --show-progress -y -o DPkg::Options::=--force-confdef --without-new-pkgs upgrade
    $ sudo DEBIAN_FRONTEND=noninteractive apt-get --show-progress -y -o DPkg::Options::=--force-confdef full-upgrade
    $ sudo apt-get -y autoremove
    $ sudo reboot -r now
    #
    # Perform a minor release upgrade, removing packages if required
    $ sudo apt-get dist-upgrade

Fedora
    https://fedoraproject.org/wiki/Releases
    11 Sep 2025: 42 & 41
    # which packages are installed
    $ rpm -qa --qf '%{NAME}\n'
    # package search recipe (name and info or description)
    $ dnf search <pattern>
    # which package installs a file/directory
    $ rpm -qf <path>
    # which files are in an installed package
    $ rpm -ql <package>
    # update all packages recipe
    $ sudo dnf update
    # upgrade recipe from Fedora 21 onwards
    $ sudo dnf --refresh update; sudo dnf install dnf-plugin-system-upgrade
    $ sudo dnf install fedora-gpg-keys

    # only upgrade from N to N+1 is supported
    # for Fedora 33, else "33" replaced by the desired version number
    $ ver=33
    $ sudo dnf -y --refresh upgrade
    $ sudo dnf -y system-upgrade download --allowerasing --releasever=$ver
    $ sudo dnf system-upgrade reboot
    # and then watch the console window ...

    # recipe below works (at least) for Fedora 34
    # setting up a static IP addr using nmcli (assume interface enp1s0)
    $ if=enp1s0
    $ sudo nmcli connection modify $if IPv4.address 192.168.20.226/24
			       200 + vm ordinal, e.g vm26 here ^^^
    $ sudo nmcli connection modify $if IPv4.gateway 192.168.20.1
    $ sudo nmcli connection modify $if IPv4.dns 192.168.20.100
    $ sudo nmcli connection modify $if IPv4.dns-search localdomain
    $ sudo nmcli connection modify $if IPv4.method manual
    $ sudo nmcli connection down $if
    $ sudo nmcli connection up $if

RedHat
    https://access.redhat.com/articles/3078
    Full Support versions (5 years from release)
    11 Sep 2025: 8.10, 9.6 & 10.1
    # which packages are installed
    $ rpm -qa --qf '%{NAME}\n'
    # package search recipe (name and info or description)
    $ yum search <pattern>
    # which package installs a file/directory
    $ rpm -qF <path>
    # which files are in an installed package
    $ rpm -ql <package>
    # package install recipe
    $ sudo yum install <package>
    # update all packages recipe
    $ sudo yum update
    # upgrade recipe
    $ ???
    # sort packages on installed size, smallest to largest
    $ rpm -qa --qf '%{SIZE} %{NAME}\n' | sort -k1 -n

    Update.  As of RHEL 8, dnf is preferred over yum.

    Note on dnf modules.
    To get libuv-devel installed on RHEL8 needed this recipe from Andreas.
    [update 7 Jan 2024 on Oracle Linux 8.8 - this is not needed]
    # dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
    # dnf install -y libuv
    # dnf module install -y libuv/devel
    # rpm -q libuv-devel
    libuv-devel-1.23.1-2.module_el8+8698+518af944.x86_64

CentOS Stream
    https://en.wikipedia.org/wiki/CentOS_Stream
    11 Sep 2025: 9, 10
    # everything the same as RedHat
    # only sane upgrade plan is fresh install!, then
    # sudo dnf install epel-release -y

    # special notes for CentOS Stream X
    # - need to register with entitlement server
    $ sudo subscription-manager register
    ... details from Enpass
    # - need CRB repo enabled (it is not enabled by default)
    $ sudo dnf config-manager --set-enabled crb
    # - need EPEL repo enabled
    $ sudo dnf install epel-release

openSUSE
    https://en.opensuse.org/Lifetime
    11 Sep 2025: 15.6 and 16.0
    # which packages are installed
    $ rpm -qa --qf '%{NAME}\n'
    # package search recipe (name and info or description)
    $ zypper search -s <pattern>
    # which package installs a file/directory
    $ rpm -qF <path>
    # which files are in an installed package
    $ rpm -ql <package>
    # package install recipe
    $ sudo zypper install <package>
    # update all packages recipe
    $ sudo zypper refresh; sudo zypper update
    #
    # new upgrade recipe (from 15.6 onwards)
    $ sudo zypper install opensuse-migration-tool
    $ sudo opensuse-migration-tool
    $ sudo reboot # After a successful migration
    #
    # old upgrade recipe ... e.g from 15.4 to 15.5
    # only upgrade from 15.N to 15.N+1 is supported
    # 15.4 change in RSA algorithm
    $ sudo rpm --import /usr/lib/rpm/gnupg/keys/gpg-pubkey-29b700a4-62b07e22.asc
    $ sudo rpm --import /usr/lib/rpm/gnupg/keys/gpg-pubkey-25db7ae0-645bae34.asc
    $ sudo zypper refresh; sudo zypper update
    $ sudo sed -i 's/15\.4/15.5/g' /etc/zypp/repos.d/*
    $ sudo zypper --gpg-auto-import-keys refresh
    $ sudo zypper --releasever=15.5 refresh
    # and then in a console window
    $ sudo zypper --releasever=15.5 dist-upgrade

SUSE (SLES)
    https://www.suse.com/lifecycle/
    01 Jan 2026: 15 SP7 and 16
    # which packages are installed
    $ rpm -qa --qf '%{NAME}\n'
    # package search recipe (name and info or description)
    $ zypper search -s <pattern>
    # which package installs a file/directory
    $ rpm -qF <path>
    # which files are in an installed package
    $ rpm -ql <package>
    # package install recipe
    $ sudo zypper install <package>
    # update all packages recipe
    $ sudo zypper refresh; sudo zypper update
    # release upgrade recipe
    # once only ...
    $ sudo zypper install zypper-migration-plugin
    # then
    $ sudo zypper update; sudo zypper patch
    $ sudo zypper migration

FreeBSD
    https://www.freebsd.org/releases/
    01 Jan 2026: 14.3 and 15.0
    # which packages are installed
    $ pkg info -a | sed -e 's/-[0-9].*//'
    # package search recipe (name and info or description)
    $ pkg search <pattern>
    # which package installs a file/directory
    $ pkg which file-or-dir-path
    # which files are in an installed package
    $ pkg list pkgname
    # package install recipe
    $ sudo pkg install -y <package>
    # package removal recipe
    $ sudo pkg delete <package> or <glob pattern>
    # update all packages recipe
    $ sudo pkg update; sudo pkg upgrade -y
    # clean pkg cache
    $ sudo pkg clean -ay
    #
    # release upgrade recipe
    $ sudo freebsd-update fetch; sudo freebsd-update install
    $ rel=13.2; export rel
    # has to be a N.M-RELEASE version for -r
    $ sudo -E freebsd-update upgrade -r $rel-RELEASE; sudo freebsd-update install
    $ sudo reboot
    ...
    $ sudo freebsd-update install
    $ sudo pkg update; sudo pkg upgrade -y
    $ sudo reboot

    on vm32:
    Message from gcc6-6.4.0_7:

    To ensure binaries built with this toolchain find appropriate versions
    of the necessary run-time libraries, you may want to link using

      -Wl,-rpath=/usr/local/lib/gcc6

    For ports leveraging USE_GCC, USES=compiler, or USES=fortran this happens
    transparently.

NetBSD
    http://www.netbsd.org/releases/formal.html
    11 Sep 2025: 9.4 and 10.1
    # --- pkgin for pre-build binaries ---
    # which packages are installed
    $ pkg_info -a | sed -e 's/-[0-9].*//'
    # package search recipe (name and info or description)
    $ pkgin search <pattern>
    # which package installs a file/directory
    $ pkg_info -F <path>
    # which files are in an installed package
    $ pkgin pkg-content <package>
    # available packages
    $ pkgin avail | grep <pattern> (matches name or summary)
    # installed packages
    $ pkgin list
    # package install recipe
    $ sudo pkgin install <package>
    # update all packages recipe
    $ sudo pkgin update; sudo pkgin full-upgrade
    # release upgrade recipe
    $ ???
    # --- pkgsrc for source --
    $ sudo pkgin install cvs
    # run everything as root, after cd /usr
    # to seed with the 2018Q3 stable release
    # cvs -q -z2 -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -r pkgsrc-2018Q3 -P pkgsrc
    # to update CVS
    # cd /usr/pkgsrc && cvs update -dP

Gentoo
    https://wiki.gentoo.org/wiki/Main_Page
    No concept of versions, just install latest.
    # which packages are installed
    $ equery list '*' | sed -e 's/.*].*] //' -e 's/-[0-9].*//'
    # package search recipe - names
    $ emerge --search |-s somestring
    $ emerge --search |-s "%someregexp"
    # package search recipe - descriptions
    $ emerge --searchdesc|-S somestring
    # which package installs a file/directory
    $ equery belongs <full-pathname>
    # which files are in an installed package
    $ equery files <package>
    # package install recipe
    $ sudo emerge --ask <package> [...]
    # update all packages recipe
    No real concept of releases ... just weekly updates
    $ sudo emerge --sync
    # simple
    $ sudo emerge --update --ask @world
    # after some time may need
    $ sudo emerge --update --backtrack=100 --deep --with-bdeps=y --newuse --ask @world
    # ...
    $ sudo emerge --depclean
    # may need to rm -rf /var/tmp/portage if lots of updates
    # release upgrade recipe
    N/A

Linux Mint
    http://www.linuxmint.com/download.php
    11 Sep 2025: 22.2
    # same as Ubuntu
    
OpenMandriva
    https://wiki.openmandriva.org/en/distribution/releases/download
    11 Sep 2025: Rock (OMLx 6.0) and Rome (rolling)
    # same as Fedora now
    ...
    # for 4.1 -> 4.2 this looks promising
    https://forum.openmandriva.org/t/how-to-upgrade-from-rock-lx-4-1-to-lx-4-2/3791
    - update repository source to Release
    - dnf clean-all
    - sudo dnf clean-all
    - sudo dnf upgrade
    - revert repository source to Rock
    - dnf clean all
    - sudo dnf clean all ; sudo dnf --allowerasing distro-sync

Slackware
    https://www.slackware.com
    11 Sep 2025: 15.0
    # which packages are installed
    $ ls /var/log/packages | sed -e 's/-[0-9].*//'
    # package search recipe (name only)
    $ slackpkg search <info-pattern>
    # which package installs a file/directory
    $ slackpkg file-search <filename-pattern>
    # which files are in an installed package
    $ sed -e '1,/^FILE LIST:/d' </var/log/packages/<package>
    # package install recipe
    $ sudo slackpkg install <package>
    # update all packages recipe
    $ sudo slackpkg -batch=on update; sudo slackpkg -batch=on -default_answer=y install-new
    # release upgrade recipe: update all packages as above, then ...
    $ sudo slackpkg -batch=on -default_answer=y upgrade-all; sudo slackpkg clean-system
    # then reconcile any .new files in /etc

Arch Linux
    https://www.archlinux.org/releng/releases/
    11 Sep 2025: 2025-07-01, 2025-08-01 and 2025-09-01
    # after virgin install, need at least this to get going
    $ sudo pacman -S inetutils	# for hostname(1)
    $ sudo pacman -S iputils	# for ping(1)
    $ sudo pacman -S which	# for which(1)
    # which packages are installed
    $ pacman -Q -i | sed -n -e '/^Name /s/.* : //p'
    # package search recipe (name and info or description)
    $ pacman -S -l | grep <package-name-pattern>
    $ pacman -S -s <regexp>
    # which package installs a file/directory
    $ pacman -Q -o <path> | <filename>
    # which files are in an installed package
    $ sudo pacman -F -y (refresh files database) then
    $         pacman -F -l <package> (files in a package)
    $         pacman -F -o <fullpath> (package(s) owning a file)
    $         pacman -F -x <regexp> (package(s) owning matching files)
    # package install recipe, always do 
    $ sudo pacman -Syu
    # then install the package ... never try to upgrade a package
    $ sudo pacman -S <package>
    # update all packages recipe
    $ sudo pacman -Sy archlinux-keyring
    $ sudo pacman -Syyu
    # clean pacman cache
    $ sudo pacman -Sc

OpenIndiana
    https://www.openindiana.org/downloads/
    11 Sep 2025: 2025.06
    # initial setup to get rid of restricitions and to get up to date ...
    $ sudo pkg uninstall userland-incorporation entire
    $ sudo pkg update
    # then need a reboot if pkg reports
    # 	Create boot environment: Yes
    # because the boot environment will have changed
    #
    # visudo ... need /usr/gnu/bin: at the start of secure_path so that
    # sudo make ... picks (gnu) make ahead of illumos make in /usr/bin
    # (for example)
    #
    # which packages are installed
    $ pkg list -H | sed -e 's/ .*//'
    # package search recipe (name and info or description)
    $ pkg search -pr <pattern>
    # which package installs a file/directory
    $ pkg contents -H -o pkg.name -a path=<path without leading slash>
    # which files are in an installed package
    $ pkg contents <package>
    # which files are in a package (installed or not installed)
    $ pkg contents -r <package>
    # package install recipe
    $ sudo pkg install <package>
    # update all packages recipe
    $ sudo pkg update
    # release upgrade recipe
    $ ???
    # to show zfs snapshots
    $ zfs list -t snapshot
    # to remove zfs snapshots
    $ sudo zfs destroy [-R] <name@date>
    # initial zoneinfo setup ... there is no /etc/localtime for
    # Solaris|OpenIndiana
    #
    $ eval $(grep '^TZ=' /etc/TIMEZONE)
    $ sudo ln -s /usr/share/lib/zoneinfo/$TZ /etc/localtime

OpenBSD
    https://cdn.openbsd.org/pub/OpenBSD/
    22 Oct 2025: 7.7, 7.8
    # package search recipe (name only, not info or description)
    $ pkg_info -Q <pattern>
    # which package installs a file/directory
    # -- pkg_add pkglocatedb first to get the "all" packages DB,
    #    rather than searching just the installed packages
    $ pkglocate <filename> or <pathname>
    # which files are in an installed package
    $ pkg_info -L <package>
    # package install recipe
    $ sudo pkg_add <package>
    # update all packages recipe
    $ sudo pkg_add -u
    # remove a package
    $ sudo pkg_delete <package>
    # unattended release upgrade recipe ... only supported from
    # one version to the next version
    $ sudo sysupgrade
    ... switch to console window after Fetching and Verifying sets ...
    - will reboot and install fetched sets
    - will reboot again
    $ sudo sysmerge
    $ sudo pkg_add -u
    - this will take some time and may involve answering some questions
      to resolve issues

Template
    URL
    DATE: VERSION(S)
    # which packages are installed
    # package search recipe (name and info or description)
    # which package installs a file/directory
    # which files are in an installed package
    # package install recipe
    # update all packages recipe
    # release upgrade recipe

=== Package Hit List of Removal Candidates ===

If you're short of space, or have long-standing VMs that you'd like to
minimize update times for, the following list of packages are ones that
are commonly installed and are not needed for PCP builds nor QA.

Debian packages removed but not purged
    $ dpkg-query -Wf '${Package}\t${Status}\n' | grep 'deinstall ok config-files' | awk '{print $1}'

Debian installed (and not purged) packages sorted by size:
    $ dpkg-query -Wf '${Installed-Size}\t${Package}\t${Status}\n' | grep -v deinstall | awk '{print $1,$2}' | sort -nr | head -20

    qt4-doc qt5-doc nodejs-docs libx11-doc
    imagemagick libreoffice* libfreerdp freerdp*
    cups foomatic-filters foomatic-db-filesystem hplip*
    sane xsane libsane libksane sane-backends*
    brasero evince totem evolution thunderbird firefox
    gimp ghostscript* busybox mono* libmono-* *games*
    espeak espeak-data aspell ispell hunspell hyphen
    atril analog coinor shotwell synaptic wamerican samba* libsamba*
    lohit-*-fonts adobe-source-han-sans-* cjkuni-uming-fonts
    wqy-zenhei-fonts julietaula-montserrat-fonts naver-nanum-fonts-common
    abattis-cantarell-fonts
    naver-nanum-gothic-fonts gnome-orca ndiswrapper* libhangul* libpinyin*
    libzhuyin*

=== Miscellaneous Hints ===

Extending a root LVM with device-mapper by adding a partition
    From
    https://serverfault.com/questions/810221/how-to-correctly-extend-a-linux-device-mapper-partition
    or
    https://rbgeek.wordpress.com/2013/01/27/how-to-extend-the-root-partition-in-lvm/
    Assume device-mapper vgname fedora-server
    Assume partiton to add is partiton 1 of /dev/vdb
    $ disk=/dev/vdb
    $ part=1
    $ sudo cfdisk $disk		<- make $part partition, all of the disk
    				<- gpt, new, write, quit
    $ sudo pvcreate $disk$part	<- initialize partition for use with LVM
    $ sudo vgdisplay		<- get volume group name
    $ vg=<name>			<- from above will be "fedora-server" or
    				   "centos" or similar
    $ sudo vgextend /dev/$vg $disk$part	<- extend the volume group
    # get pathname for base volume
    $ ls /dev/$vg
    $ root=...
    $ sudo lvextend -l +100%FREE /dev/$vg/$root	<- extend LVM
    # for ext2
    $ sudo resize2fs /dev/$vg/$root		<- extend filesystem
    # for xfs
    $ sudo xfs_growfs /				<- extend filesytem

Just remember that if you're using XFS, you need xfs_growfs instead of
resize2fs.

Extending a root btrfs filesystem by adding a partition

    Assume partiton to add is partiton 1 of /dev/vdb
    $ disk=/dev/vdb
    $ part=1
    $ sudo cfdisk $disk		<- make $part partition, all of the disk
    $ sudo btrfs -f device add $disk$part /

Recreate the mirror disk on vm03

    $ sudo mkdir /silver
    # make sure kernel auto-detected RAID1 md volume /dev/md127 with
    # parts /dev/sda1 and /dev/sdb1
    $ sudo mdadm --query --detail /dev/md127
    $ sudo lsblk -o +UUID /dev/md127
    # append this line to /etc/fstab
UUID=8f865731-443c-4b84-ade7-31e6ff94ee7c /silver ext2 defaults 0 0
    $ sudo systemctl daemon-reload
    $ sudo mount -a

Adding a new disk

    $ disk=/dev/vdb
    $ sudo cfdisk $disk         <- make partition 1, all of the disk
    $ mkfs.xfs ${disk}1
    $ lsblk -fs ${disk}1 	-> UUID=<uuid>
    $ [ -d /data ] || sudo mkdir /data
    $ echo "UUID=<uuid> /data xfs default 0 2" | sudo tee -a /etc/fstab