MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928,

r281783, r281809, r281832:

 r280299 (cperciva):
  When creating VM images, copy the contents of the created filesystem
  into a new filesystem before packaging it into a disk image.  This
  prevents "remnants" of deleted files from showing up in the VM images,
  and reduces their compressed size (by about 10% for the cloudware
  images) as a result.

 r280840 (cperciva):
  Clean up filesystem unmounting in vmimage builds:
  - Remove vm_umount_base function which is currently unused.
  - Add umount_loop function which loops attempting to unmount one
    filesystem.
  - Replace calls to umount with calls to umount_loop.
  - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted.

  The looping is necessary because sometimes umount fails due to
  filesystems being busy.  The most common cause of such busyness is
  periodic(8) jobs running `find / ...`.

 r280846 (cperciva):
  Improve check for whether ${DESTDIR}/dev is mounted.

 r280879 (cperciva):
  Add bits for building EC2 disk images.

 r280881 (cperciva):
  Partially revert r278118 now that the required logic for deciding
  whether freebsd-update can be useful has moved into the
  firstboot_freebsd_update script.

 r280884 (cperciva):
  Fix pkg(8) command line in vm_extra_pre_umount().

 r280928 (cperciva):
  Add code for creating an EC2 AMI.

 r281783:
  When building VM disk images, vm_copy_base() uses tar(1) to copy
  the userland from one md(4)-mounted filesystem to a clean filesystem
  to prevent remnants of files that were added and removed from
  resulting in an unclean filesystem.  When newfs(8) creates the first
  filesystem with journaled soft-updates enabled, the /.sujournal file
  in the new filesystem cannot be overwritten by the /.sujournal in the
  original filesystem.

  To avoid this particular error case, do not enable journaled
  soft-updates when creating the md(4)-backed filesystems, and instead
  use tunefs(8) to enable journaled soft-updates after the new
  filesystem is populated in vm_copy_base().

  While here, fix a long standing bug where the build environment /boot
  files were used by mkimg(1) when creating the VM disk images by using
  the files in .OBJDIR.

 r281809:
  Simplify variable expansion in attempt to fix the vm-image build.

 r281832:
  Fix locating the /boot files.

Sponsored by:	The FreeBSD Foundation

1 files changed, 83 insertions, 0 deletions

diff --git a/release/tools/ec2.conf b/release/tools/ec2.conf
new file mode 100644
index 0000000..4969ad5
--- /dev/null
+++ b/release/tools/ec2.conf
@@ -0,0 +1,83 @@
+#!/bin/sh
+#
+# $FreeBSD$
+#
+
+# Packages to install into the image we're creating.  This is a deliberately
+# minimalist set, providing only the packages necessary to bootstrap further
+# package installation as specified via EC2 user-data.
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs"
+
+# Set to a list of third-party software to enable in rc.conf(5).
+export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap ec2_loghostkey firstboot_freebsd_update firstboot_pkgs"
+
+# Build with a 1.5 GB UFS partition; the growfs rc.d script will expand
+# the partition to fill the root disk after the EC2 instance is launched.
+# Note that if this is set to <N>G, we will end up with an <N+1> GB disk
+# image since VMSIZE is the size of the UFS partition, not the disk which
+# it resides within.
+export VMSIZE=1536M
+
+# No swap space; the ec2_ephemeralswap rc.d script will allocate swap
+# space on EC2 ephemeral disks.  (If they exist -- the T2 low-cost instances
+# and the C4 compute-optimized instances don't have ephemeral disks.  But
+# it would be silly to bloat the image and increase costs for every instance
+# just for those two families, especially since instances ranging in size
+# from 1 GB of RAM to 60 GB of RAM would need different sizes of swap space
+# anyway.)
+export NOSWAP=YES
+
+vm_extra_pre_umount() {
+	# The firstboot_pkgs rc.d script will download the repository
+	# catalogue and install or update pkg when the instance first
+	# launches, so these files would just be replaced anyway; removing
+	# them from the image allows it to boot faster.
+	pkg -c ${DESTDIR} delete -f -y pkg
+	rm ${DESTDIR}/var/db/pkg/repo-*.sqlite
+
+	# The size of the EC2 root disk can be configured at instance launch
+	# time; expand our filesystem to fill the disk.
+	echo 'growfs_enable="YES"' >> ${DESTDIR}/etc/rc.conf
+
+	# EC2 instances use DHCP to get their network configuration.
+	echo 'ifconfig_DEFAULT="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf
+
+	# Unless the system has been configured via EC2 user-data, the user
+	# will need to SSH in to do anything.
+	echo 'sshd_enable="YES"' >> ${DESTDIR}/etc/rc.conf
+
+	# The AWS CLI tools are generally useful, and small enough that they
+	# will download quickly; but users will often override this setting
+	# via EC2 user-data.
+	echo 'firstboot_pkgs_list="awscli"' >> ${DESTDIR}/etc/rc.conf
+
+	# The EC2 console is output-only, so while printing a backtrace can
+	# be useful, there's no point dropping into a debugger or waiting
+	# for a keypress.
+	echo 'debug.trace_on_panic=1' >> ${DESTDIR}/etc/sysctl.conf
+	echo 'debug.debugger_on_panic=0' >> ${DESTDIR}/etc/sysctl.conf
+	echo 'kern.panic_reboot_wait_time=0' >> ${DESTDIR}/etc/sysctl.conf
+
+	# The console is not interactive, so we might as well boot quickly.
+	echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf
+	echo 'beastie_disable="YES"' >> ${DESTDIR}/boot/loader.conf
+
+	# The EC2 console is an emulated serial port.
+	echo 'console="comconsole"' >> ${DESTDIR}/boot/loader.conf
+
+	# Some older EC2 hardware used a version of Xen with a bug in its
+	# emulated serial port.  It is not clear if EC2 still has any such
+	# nodes, but apply the workaround just in case.
+	echo 'hw.broken_txfifo="1"' >> ${DESTDIR}/boot/loader.conf
+
+	# The first time the AMI boots, the installed "first boot" scripts
+	# should be allowed to run:
+	# * ec2_configinit (download and process EC2 user-data)
+	# * ec2_fetchkey (arrange for SSH using the EC2-provided public key)
+	# * growfs (expand the filesystem to fill the provided disk)
+	# * firstboot_freebsd_update (install critical updates)
+	# * firstboot_pkgs (install packages)
+	touch ${DESTDIR}/firstboot
+
+	return 0
+}