Merge from project branch via main. Uninteresting commits are trimmed.

Refactor of /dev/random device. Main points include:

* Userland seeding is no longer used. This auto-seeds at boot time
on PC/Desktop setups; this may need some tweeking and intelligence
from those folks setting up embedded boxes, but the work is believed
to be minimal.

* An entropy cache is written to /entropy (even during installation)
and the kernel uses this at next boot.

* An entropy file written to /boot/entropy can be loaded by loader(8)

* Hardware sources such as rdrand are fed into Yarrow, and are no
longer available raw.

------------------------------------------------------------------------
r256240 | des | 2013-10-09 21:14:16 +0100 (Wed, 09 Oct 2013) | 4 lines

Add a RANDOM_RWFILE option and hide the entropy cache code behind it.
Rename YARROW_RNG and FORTUNA_RNG to RANDOM_YARROW and RANDOM_FORTUNA.
Add the RANDOM_* options to LINT.

------------------------------------------------------------------------
r256239 | des | 2013-10-09 21:12:59 +0100 (Wed, 09 Oct 2013) | 2 lines

Define RANDOM_PURE_RNDTEST for rndtest(4).

------------------------------------------------------------------------
r256204 | des | 2013-10-09 18:51:38 +0100 (Wed, 09 Oct 2013) | 2 lines

staticize struct random_hardware_source

------------------------------------------------------------------------
r256203 | markm | 2013-10-09 18:50:36 +0100 (Wed, 09 Oct 2013) | 2 lines

Wrap some policy-rich code in 'if NOTYET' until we can thresh out
what it really needs to do.

------------------------------------------------------------------------
r256184 | des | 2013-10-09 10:13:12 +0100 (Wed, 09 Oct 2013) | 2 lines

Re-add /dev/urandom for compatibility purposes.

------------------------------------------------------------------------
r256182 | des | 2013-10-09 10:11:14 +0100 (Wed, 09 Oct 2013) | 3 lines

Add missing include guards and move the existing ones out of the
implementation namespace.

------------------------------------------------------------------------
r256168 | markm | 2013-10-08 23:14:07 +0100 (Tue, 08 Oct 2013) | 10 lines

Fix some just-noticed problems:

o Allow this to work with "nodevice random" by fixing where the
MALLOC pool is defined.

o Fix the explicit reseed code. This was correct as submitted, but
in the project branch doesn't need to set the "seeded" bit as this
is done correctly in the "unblock" function.

o Remove some debug ifdeffing.

o Adjust comments.

------------------------------------------------------------------------
r256159 | markm | 2013-10-08 19:48:11 +0100 (Tue, 08 Oct 2013) | 6 lines

Time to eat crow for me.

I replaced the sx_* locks that Arthur used with regular mutexes;
this turned out the be the wrong thing to do as the locks need to
be sleepable. Revert this folly.

# Submitted by:	Arthur Mesh <arthurmesh@gmail.com> (In original diff)

------------------------------------------------------------------------
r256138 | des | 2013-10-08 12:05:26 +0100 (Tue, 08 Oct 2013) | 10 lines

Add YARROW_RNG and FORTUNA_RNG to sys/conf/options.

Add a SYSINIT that forces a reseed during proc0 setup, which happens
fairly late in the boot process.

Add a RANDOM_DEBUG option which enables some debugging printf()s.

Add a new RANDOM_ATTACH entropy source which harvests entropy from the
get_cyclecount() delta across each call to a device attach method.

------------------------------------------------------------------------
r256135 | markm | 2013-10-08 07:54:52 +0100 (Tue, 08 Oct 2013) | 8 lines

Debugging. My attempt at EVENTHANDLER(multiuser) was a failure; use
EVENTHANDLER(mountroot) instead.

This means we can't count on /var being present, so something will
need to be done about harvesting /var/db/entropy/... .

Some policy now needs to be sorted out, and a pre-sync cache needs
to be written, but apart from that we are now ready to go.

Over to review.

------------------------------------------------------------------------
r256094 | markm | 2013-10-06 23:45:02 +0100 (Sun, 06 Oct 2013) | 8 lines

Snapshot.

Looking pretty good; this mostly works now. New code includes:

* Read cached entropy at startup, both from files and from loader(8)
preloaded entropy. Failures are soft, but announced. Untested.

* Use EVENTHANDLER to do above just before we go multiuser. Untested.

------------------------------------------------------------------------
r256088 | markm | 2013-10-06 14:01:42 +0100 (Sun, 06 Oct 2013) | 2 lines

Fix up the man page for random(4). This mainly removes no-longer-relevant
details about HW RNGs, reseeding explicitly and user-supplied
entropy.

------------------------------------------------------------------------
r256087 | markm | 2013-10-06 13:43:42 +0100 (Sun, 06 Oct 2013) | 6 lines

As userland writing to /dev/random is no more, remove the "better
than nothing" bootstrap mode.

Add SWI harvesting to the mix.

My box seeds Yarrow by itself in a few seconds! YMMV; more to follow.

------------------------------------------------------------------------
r256086 | markm | 2013-10-06 13:40:32 +0100 (Sun, 06 Oct 2013) | 11 lines

Debug run. This now works, except that the "live" sources haven't
been tested. With all sources turned on, this unlocks itself in
a couple of seconds! That is no my box, and there is no guarantee
that this will be the case everywhere.

* Cut debug prints.

* Use the same locks/mutexes all the way through.

* Be a tad more conservative about entropy estimates.

------------------------------------------------------------------------
r256084 | markm | 2013-10-06 13:35:29 +0100 (Sun, 06 Oct 2013) | 5 lines

Don't use the "real" assembler mnemonics; older compilers may not
understand them (like when building CURRENT on 9.x).

# Submitted by:	Konstantin Belousov <kostikbel@gmail.com>

------------------------------------------------------------------------
r256081 | markm | 2013-10-06 10:55:28 +0100 (Sun, 06 Oct 2013) | 12 lines

SNAPSHOT.

Simplify the malloc pools; We only need one for this device.

Simplify the harvest queue.

Marginally improve the entropy pool hashing, making it a bit faster
in the process.

Connect up the hardware "live" source harvesting. This is simplistic
for now, and will need to be made rate-adaptive.

All of the above passes a compile test but needs to be debugged.

------------------------------------------------------------------------
r256042 | markm | 2013-10-04 07:55:06 +0100 (Fri, 04 Oct 2013) | 25 lines

Snapshot. This passes the build test, but has not yet been finished or debugged.

Contains:

* Refactor the hardware RNG CPU instruction sources to feed into
the software mixer. This is unfinished. The actual harvesting needs
to be sorted out. Modified by me (see below).

* Remove 'frac' parameter from random_harvest(). This was never
used and adds extra code for no good reason.

* Remove device write entropy harvesting. This provided a weak
attack vector, was not very good at bootstrapping the device. To
follow will be a replacement explicit reseed knob.

* Separate out all the RANDOM_PURE sources into separate harvest
entities. This adds some secuity in the case where more than one
is present.

* Review all the code and fix anything obviously messy or inconsistent.
Address som review concerns while I'm here, like rename the pseudo-rng
to 'dummy'.

# Submitted by:	Arthur Mesh <arthurmesh@gmail.com> (the first item)

------------------------------------------------------------------------
r255319 | markm | 2013-09-06 18:51:52 +0100 (Fri, 06 Sep 2013) | 4 lines

Yarrow wants entropy estimations to be conservative; the usual idea
is that if you are certain you have N bits of entropy, you declare
N/2.

------------------------------------------------------------------------
r255075 | markm | 2013-08-30 18:47:53 +0100 (Fri, 30 Aug 2013) | 4 lines

Remove short-lived idea; thread to harvest (eg) RDRAND enropy into the
usual harvest queues. It was a nifty idea, but too heavyweight.

# Submitted by:	Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r255071 | markm | 2013-08-30 12:42:57 +0100 (Fri, 30 Aug 2013) | 4 lines

Separate out the Software RNG entropy harvesting queue and thread
into its own files.

# Submitted by:	 Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r254934 | markm | 2013-08-26 20:07:03 +0100 (Mon, 26 Aug 2013) | 2 lines

Remove the short-lived namei experiment.

------------------------------------------------------------------------
r254928 | markm | 2013-08-26 19:35:21 +0100 (Mon, 26 Aug 2013) | 2 lines

Snapshot; Do some running repairs on entropy harvesting. More needs
to follow.

------------------------------------------------------------------------
r254927 | markm | 2013-08-26 19:29:51 +0100 (Mon, 26 Aug 2013) | 15 lines

Snapshot of current work;

1) Clean up namespace; only use "Yarrow" where it is Yarrow-specific
or close enough to the Yarrow algorithm. For the rest use a neutral
name.

2) Tidy up headers; put private stuff in private places. More could
be done here.

3) Streamline the hashing/encryption; no need for a 256-bit counter;
128 bits will last for long enough.

There are bits of debug code lying around; these will be removed
at a later stage.

------------------------------------------------------------------------
r254784 | markm | 2013-08-24 14:54:56 +0100 (Sat, 24 Aug 2013) | 39 lines

1) example (partially humorous random_adaptor, that I call "EXAMPLE")
 * It's not meant to be used in a real system, it's there to show how
   the basics of how to create interfaces for random_adaptors. Perhaps
   it should belong in a manual page

2) Move probe.c's functionality in to random_adaptors.c
 * rename random_ident_hardware() to random_adaptor_choose()

3) Introduce a new way to choose (or select) random_adaptors via tunable
"rngs_want" It's a list of comma separated names of adaptors, ordered
by preferences. I.e.:
rngs_want="yarrow,rdrand"

Such setting would cause yarrow to be preferred to rdrand. If neither of
them are available (or registered), then system will default to
something reasonable (currently yarrow). If yarrow is not present, then
we fall back to the adaptor that's first on the list of registered
adaptors.

4) Introduce a way where RNGs can play a role of entropy source. This is
mostly useful for HW rngs.

The way I envision this is that every HW RNG will use this
functionality by default. Functionality to disable this is also present.
I have an example of how to use this in random_adaptor_example.c (see
modload event, and init function)

5) fix kern.random.adaptors from
kern.random.adaptors: yarrowpanicblock
to
kern.random.adaptors: yarrow,panic,block

6) add kern.random.active_adaptor to indicate currently selected
adaptor:
root@freebsd04:~ # sysctl kern.random.active_adaptor
kern.random.active_adaptor: yarrow

# Submitted by:	Arthur Mesh <arthurmesh@gmail.com>

Submitted by:	Dag-Erling Smørgrav <des@FreeBSD.org>, Arthur Mesh <arthurmesh@gmail.com>
Reviewed by:	des@FreeBSD.org
Approved by:	re (delphij)
Approved by:	secteam (des,delphij)

2 files changed, 79 insertions, 83 deletions

diff --git a/share/examples/kld/random_adaptor/random_adaptor_example.c b/share/examples/kld/random_adaptor/random_adaptor_example.c
index c0ab10a..da588a8 100644
--- a/share/examples/kld/random_adaptor/random_adaptor_example.c
+++ b/share/examples/kld/random_adaptor/random_adaptor_example.c
@@ -30,32 +30,29 @@ __FBSDID("$FreeBSD$");
 
 #include <sys/param.h>
 #include <sys/kernel.h>
+#include <sys/lock.h>
 #include <sys/module.h>
-#include <sys/selinfo.h>
+#include <sys/random.h>
 #include <sys/systm.h>
 
+#include <dev/random/live_entropy_sources.h>
 #include <dev/random/random_adaptors.h>
 #include <dev/random/randomdev.h>
 
-#define RNG_NAME "example"
-
 static int random_example_read(void *, int);
 
 struct random_adaptor random_example = {
 	.ident = "Example RNG",
-	.init = (random_init_func_t *)random_null_func,
-	.deinit = (random_deinit_func_t *)random_null_func,
+	.source = RANDOM_PURE_BOGUS,	/* Make sure this is in
+					 * sys/random.h and is unique */
 	.read = random_example_read,
-	.write = (random_write_func_t *)random_null_func,
-	.reseed = (random_reseed_func_t *)random_null_func,
-	.seeded = 1,
 };
 
 /*
  * Used under the license provided @ http://xkcd.com/221/
  * http://creativecommons.org/licenses/by-nc/2.5/
  */
-static u_char
+static uint8_t
 getRandomNumber(void)
 {
 	return 4;   /* chosen by fair dice roll, guaranteed to be random */
@@ -64,14 +61,13 @@ getRandomNumber(void)
 static int
 random_example_read(void *buf, int c)
 {
-	u_char *b;
+	uint8_t *b;
 	int count;
 
 	b = buf;
 
-	for (count = 0; count < c; count++) {
+	for (count = 0; count < c; count++)
 		b[count] = getRandomNumber();
-	}
 
 	printf("returning %d bytes of pure randomness\n", c);
 	return (c);
@@ -80,15 +76,26 @@ random_example_read(void *buf, int c)
 static int
 random_example_modevent(module_t mod, int type, void *unused)
 {
+	int error = 0;
 
 	switch (type) {
 	case MOD_LOAD:
-		random_adaptor_register(RNG_NAME, &random_example);
-		EVENTHANDLER_INVOKE(random_adaptor_attach, &random_example);
-		return (0);
+		live_entropy_source_register(&random_example);
+		break;
+
+	case MOD_UNLOAD:
+		live_entropy_source_deregister(&random_example);
+		break;
+
+	case MOD_SHUTDOWN:
+		break;
+
+	default:
+		error = EOPNOTSUPP;
+		break;
 	}
 
-	return (EINVAL);
+	return (error);
 }
 
-RANDOM_ADAPTOR_MODULE(random_example, random_example_modevent, 1);
+LIVE_ENTROPY_SRC_MODULE(live_entropy_source_example, random_example_modevent, 1);
diff --git a/share/man/man4/random.4 b/share/man/man4/random.4
index 5e73bc0..a87a2ed7 100644
--- a/share/man/man4/random.4
+++ b/share/man/man4/random.4
@@ -1,4 +1,4 @@
-.\" Copyright (c) 2001	Mark R V Murray.  All rights reserved.
+.\" Copyright (c) 2001-2013	Mark R V Murray.  All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
 .\" modification, are permitted provided that the following conditions
@@ -23,7 +23,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd August 7, 2013
+.Dd October 12, 2013
 .Dt RANDOM 4
 .Os
 .Sh NAME
@@ -43,35 +43,48 @@ The device will probe for
 certain hardware entropy sources,
 and use these in preference to the fallback,
 which is a generator implemented in software.
-If the kernel environment MIB's
-.Va hw.nehemiah_rng_enable
-or
-.Va hw.ivy_rng_enable
-are set to
-.Dq Li 0 ,
-the associated hardware entropy source will be ignored.
 .Pp
-If the device is using
-the software generator,
-writing data to
-.Nm
-would perturb the internal state.
-This perturbation of the internal state
-is the only userland method of introducing
-extra entropy into the device.
-If the writer has superuser privilege,
-then closing the device after writing
-will make the software generator reseed itself.
-This can be used for extra security,
-as it immediately introduces any/all new entropy
-into the PRNG.
-The hardware generators will generate
-sufficient quantities of entropy,
-and will therefore ignore user-supplied input.
-The software
+The software generator will start in an
+.Em unseeded
+state, and will block reads until
+it is (re)seeded.
+This may cause trouble at system boot
+when keys and the like
+are generated from
+/dev/random
+so steps should be taken to ensure a
+reseed as soon as possible.
+The
+.Xr sysctl 8
+controlling the
+.Em seeded
+status (see below) may be used
+if security is not an issue
+or for convenience
+during setup or development.
+.Pp
+This initial seeding
+of random number generators
+is a bootstrapping problem
+that needs very careful attention.
+In some cases,
+it may be difficult
+to find enough randomness
+to seed a random number generator
+until a system is fully operational,
+but the system requires random numbers
+to become fully operational.
+It is (or more accurately should be)
+critically important that the
 .Nm
-device may be controlled with
-.Xr sysctl 8 .
+device is seeded
+before the first time it is used.
+In the case where a dummy or "blocking-only"
+device is used,
+it is the responsibility
+of the system architect
+to ensure that no blocking reads
+hold up critical processes.
 .Pp
 To see the current settings of the software
 .Nm
@@ -81,22 +94,20 @@ device, use the command line:
 .Pp
 which results in something like:
 .Bd -literal -offset indent
-kern.random.adaptors: yarrow
+kern.random.adaptors: yarrow,dummy
+kern.random.active_adaptor: yarrow
+kern.random.yarrow.gengateinterval: 10
+kern.random.yarrow.bins: 10
+kern.random.yarrow.fastthresh: 96
+kern.random.yarrow.slowthresh: 128
+kern.random.yarrow.slowoverthresh: 2
 kern.random.sys.seeded: 1
 kern.random.sys.harvest.ethernet: 1
 kern.random.sys.harvest.point_to_point: 1
 kern.random.sys.harvest.interrupt: 1
-kern.random.sys.harvest.swi: 0
-kern.random.yarrow.gengateinterval: 10
-kern.random.yarrow.bins: 10
-kern.random.yarrow.fastthresh: 192
-kern.random.yarrow.slowthresh: 256
-kern.random.yarrow.slowoverthresh: 2
+kern.random.sys.harvest.swi: 1
 .Ed
 .Pp
-(These would not be seen if a
-hardware generator is present.)
-.Pp
 Other than
 .Dl kern.random.adaptors
 all settings are read/write.
@@ -107,9 +118,10 @@ variable indicates whether or not the
 .Nm
 device is in an acceptably secure state
 as a result of reseeding.
-If set to 0, the device will block (on read) until the next reseed
-(which can be from an explicit write,
-or as a result of entropy harvesting).
+If set to 0,
+the device will block (on read)
+until the next reseed
+as a result of entropy harvesting.
 A reseed will set the value to 1 (non-blocking).
 .Pp
 The
@@ -276,19 +288,6 @@ the generator produce independent sequences.
 However, the guessability or reproducibility of the sequence is unimportant,
 unlike the previous cases.
 .Pp
-One final consideration for the seeding of random number generators
-is a bootstrapping problem.
-In some cases, it may be difficult to find enough randomness to
-seed a random number generator until a system is fully operational,
-but the system requires random numbers to become fully operational.
-There is no substitute for careful thought here,
-but the
-.Fx
-.Nm
-device,
-which is based on the Yarrow system,
-should be of some help in this area.
-.Pp
 .Fx
 does also provide the traditional
 .Xr rand 3
@@ -325,17 +324,7 @@ and is an implementation of the
 .Em Yarrow
 algorithm by Bruce Schneier,
 .Em et al .
-The only hardware implementations
-currently are for the
-.Tn VIA C3 Nehemiah
-(stepping 3 or greater)
-CPU
-and the
-.Tn Intel
-.Dq Bull Mountain
-.Em RdRand
-instruction and underlying random number generator (RNG).
-More will be added in the future.
+Significant infrastructure work was done by Arthur Mesh.
 .Pp
 The author gratefully acknowledges
 significant assistance from VIA Technologies, Inc.