Improve the RC framework for the clean booting/shutdown of Jails:

1. Feature: for flexibility reasons and as a prerequisite to clean
   shutdowns, allow the configuration of a stop/shutdown command
   via rc.conf variable "jail_<name>_exec_stop" in addition to the
   start/boot command (rc.conf variable "jail_<name>_exec_start"). For
   backward compatibility reasons, rc.conf variable "jail_<name>_exec"
   is still supported, too.

2. Debug: Add the used boot/shutdown commands to the debug output of
   the /etc/rc.d/jail script, too.

3. Security: Run the Jail start/boot command in a cleaned environment
   to not leak information from the host to the Jail during startup.

4. Feature: Run the Jail stop/shutdown command "jail_<name>_exec_stop" on
   "/etc/rc.d/jail stop <name>" to allow a graceful shutdown of the Jail
   before its processes are just killed.

5. Bugfix: When killing the remaining Jail processes give the processes
   time to actually perform their termination sequence. Without this the
   subsequent umount(8) operations usually fail because the resources
   are still in use. Additionally, if after trying to TERM-inate the
   processes there are still processes hanging around, finally just KILL
   them.

6. Bugfix: In rc.shutdown, if running inside a Jail, skip the /etc/rc.d/*
   scripts which are flagged with the KEYWORD "nojail" to allow the
   correct operation of rc.shutdown under jail_<name>_exec_stop="/bin/sh
   /etc/rc.shutdown". This is analogous to what /etc/rc does inside a Jail.

Now the following typical host-configuration for two Jails works as
expected and correctly boots and shutdowns the Jails:

-----------------------------------------------------------
#  /etc/rc.conf:
jail_enable="YES"
jail_list="foo bar"
jail_foo_rootdir="/j/foo"
jail_foo_hostname="foo.example.com"
jail_foo_ip="192.168.0.1"
jail_foo_devfs_enable="YES"
jail_foo_mount_enable="YES"
jail_foo_exec_start="/bin/sh /etc/rc"
jail_foo_exec_stop="/bin/sh /etc/rc.shutdown"
jail_bar_rootdir="/j/bar"
jail_bar_hostname="bar.example.com"
jail_bar_ip="192.168.0.2"
jail_bar_devfs_enable="YES"
jail_bar_mount_enable="YES"
jail_bar_exec_start="/path/to/kjailer -v"
jail_bar_exec_stop="/bin/sh -c 'killall kjailer && sleep 60'"
-----------------------------------------------------------
#  /etc/fstab.foo
/v/foo /j/foo/v/foo nullfs rw 0 0
-----------------------------------------------------------
#  /etc/fstab.bar
/v/bar /j/bar/v/bar nullfs rw 0 0
-----------------------------------------------------------

Reviewed by:	freebsd-hackers
MFC after:	2 weeks

1 files changed, 3 insertions, 1 deletions

diff --git a/etc/rc.shutdown b/etc/rc.shutdown
index 1d15154..075881f 100644
--- a/etc/rc.shutdown
+++ b/etc/rc.shutdown
@@ -80,7 +80,9 @@ fi
 # Determine the shutdown order of the /etc/rc.d scripts,
 # and perform the operation
 #
-files=`rcorder -k shutdown /etc/rc.d/* 2>/dev/null`
+rcorder_opts="-k shutdown"
+[ `/sbin/sysctl -n security.jail.jailed` -eq 1 ] && rcorder_opts="$rcorder_opts -s nojail"
+files=`rcorder ${rcorder_opts} /etc/rc.d/* 2>/dev/null`
 
 for _rc_elem in `reverse_list $files`; do
 	debug "run_rc_script $_rc_elem faststop"