This main goals of this project are:

1. separating L2 tables (ARP, NDP) from the L3 routing tables
2. removing as much locking dependencies among these layers as
   possible to allow for some parallelism in the search operations
3. simplify the logic in the routing code,

The most notable end result is the obsolescent of the route
cloning (RTF_CLONING) concept, which translated into code reduction
in both IPv4 ARP and IPv6 NDP related modules, and size reduction in
struct rtentry{}. The change in design obsoletes the semantics of
RTF_CLONING, RTF_WASCLONE and RTF_LLINFO routing flags. The userland
applications such as "arp" and "ndp" have been modified to reflect
those changes. The output from "netstat -r" shows only the routing
entries.

Quite a few developers have contributed to this project in the
past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and
Andre Oppermann. And most recently:

- Kip Macy revised the locking code completely, thus completing
  the last piece of the puzzle, Kip has also been conducting
  active functional testing
- Sam Leffler has helped me improving/refactoring the code, and
  provided valuable reviews
- Julian Elischer setup the perforce tree for me and has helped
  me maintaining that branch before the svn conversion

3 files changed, 17 insertions, 95 deletions

diff --git a/share/man/man4/route.4 b/share/man/man4/route.4
index 0e10be5..aaf59e4 100644
--- a/share/man/man4/route.4
+++ b/share/man/man4/route.4
@@ -196,7 +196,7 @@ Messages include:
 #define	RTM_REDIRECT	0x6    /* Told to use different route */
 #define	RTM_MISS	0x7    /* Lookup failed on this address */
 #define	RTM_LOCK	0x8    /* fix specified metrics */
-#define	RTM_RESOLVE	0xb    /* request to resolve dst to LL addr */
+#define	RTM_RESOLVE	0xb    /* request to resolve dst to LL addr - unused */
 #define	RTM_NEWADDR	0xc    /* address being added to iface */
 #define	RTM_DELADDR	0xd    /* address being removed from iface */
 #define	RTM_IFINFO	0xe    /* iface going up/down etc. */
@@ -308,7 +308,7 @@ Specifiers for which addresses are present in the messages are:
 #define RTA_DST       0x1    /* destination sockaddr present */
 #define RTA_GATEWAY   0x2    /* gateway sockaddr present */
 #define RTA_NETMASK   0x4    /* netmask sockaddr present */
-#define RTA_GENMASK   0x8    /* cloning mask sockaddr present */
+#define RTA_GENMASK   0x8    /* cloning mask sockaddr present - unused */
 #define RTA_IFP       0x10   /* interface name sockaddr present */
 #define RTA_IFA       0x20   /* interface addr sockaddr present */
 #define RTA_AUTHOR    0x40   /* sockaddr for author of redirect */
diff --git a/share/man/man9/rtalloc.9 b/share/man/man9/rtalloc.9
index 4e7f818..cbd2c5e 100644
--- a/share/man/man9/rtalloc.9
+++ b/share/man/man9/rtalloc.9
@@ -27,7 +27,8 @@
 .\" SUCH DAMAGE.
 .\"
 .\" $FreeBSD$
-.Dd October 11, 2004
+.\"
+.Dd December 11, 2008
 .Os
 .Dt RTALLOC 9
 .Sh NAME
@@ -64,21 +65,6 @@ certain protocol\- and interface-specific actions to take place.
 .\" XXX - -mdoc should contain a standard request for getting em and
 .\" en dashes.
 .Pp
-When a route with the flag
-.Dv RTF_CLONING
-is retrieved, and the action of this flag is not masked, the
-.Nm
-facility automatically generates a new route using information in the
-old route as a template, and
-sends an
-.Dv RTM_RESOLVE
-message to the appropriate interface-address route-management routine
-.Pq Fn ifa->ifa_rtrequest .
-This generated route is called
-.Em cloned ,
-and has
-.Dv RTF_WASCLONED
-flag set.
 .Dv RTF_PRCLONING
 flag is obsolete and thus ignored by facility.
 If the
@@ -123,22 +109,19 @@ field.
 .Pp
 The
 .Fn rtalloc_ign
-interface can be used when the default actions of
-.Fn rtalloc
-in the presence of the
-.Dv RTF_CLONING
-flag is undesired.
+interface can be used when the caller does not want to receive
+the returned
+.Fa rtentry
+locked.
 The
 .Fa ro
 argument is the same as
 .Fn rtalloc ,
 but there is additionally a
 .Fa flags
-argument, which lists the flags in the route which are to be
-.Em ignored
-(in most cases this is
-.Dv RTF_CLONING
-flag).
+argument, which is now only used to pass
+.Dv RTF_RNH_LOCKED
+indicating that the radix tree lock is already held.
 Both
 .Fn rtalloc
 and
@@ -163,16 +146,7 @@ directly as the
 argument.
 The second argument,
 .Fa report ,
-controls whether
-.Dv RTM_RESOLVE
-requests are sent to the lower layers when an
-.Dv RTF_CLONING
-or
-.Dv RTF_PRCLONING
-route is cloned.
-Ordinarily a value of one should be passed, except
-in the processing of those lower layers which use the cloning
-facility.
+controls whether the lower layers are notified when a lookup fails.
 The third argument,
 .Fa flags ,
 is a set of flags to ignore, as in
diff --git a/share/man/man9/rtentry.9 b/share/man/man9/rtentry.9
index a1c48a6..0be14f1 100644
--- a/share/man/man9/rtentry.9
+++ b/share/man/man9/rtentry.9
@@ -28,7 +28,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd October 7, 2004
+.Dd December 11, 2008
 .Os
 .Dt RTENTRY 9
 .Sh NAME
@@ -76,8 +76,11 @@ right (some protocols will put a link-layer address here), or some
 intermediate stop on the way to that destination (if the
 .Dv RTF_GATEWAY
 flag is set).
-.It Vt "u_long rt_flags" ;
+.It Vt "int rt_flags" ;
 See below.
+.It Vt "int rt_refcnt" ;
+Route entries are reference-counted; this field indicates the number
+of external (to the radix tree) references.
 .It Vt "struct ifnet *rt_ifp" ;
 .It Vt "struct ifaddr *rt_ifa" ;
 These two fields represent the
@@ -88,48 +91,17 @@ packet to the destination or set of destinations which this route
 represents.
 .It Vt "struct rt_metrics_lite rt_rmx" ;
 See below.
-.It Vt "long rt_refcnt" ;
-Route entries are reference-counted; this field indicates the number
-of external (to the radix tree) references.
-If the
 .Dv RTF_UP
 flag is not present, the
 .Fn rtfree
 function will delete the route from the radix tree when the last
 reference drops.
-.It Vt "struct sockaddr *rt_genmask" ;
-When the
-.Fn rtalloc
-family of functions performs a cloning operation as requested by the
-.Dv RTF_CLONING
-flag, this field is used as the mask for the new route which is
-inserted into the table.
-If this field is a null pointer, then a host
-route is generated.
-.It Vt "caddr_t rt_llinfo" ;
-When the
-.Dv RTF_LLINFO
-flag is set, this field contains information specific to the link
-layer represented by the named interface address.
-(It is normally managed by the
-.Va rt_ifa->ifa_rtrequest
-routine.)
-Protocols such as
-.Xr arp 4
-use this field to reference per-destination state internal to that
-protocol.
 .It Vt "struct rtentry *rt_gwroute" ;
 This member is a reference to a route whose destination is
 .Va rt_gateway .
 It is only used for
 .Dv RTF_GATEWAY
 routes.
-.It Vt "struct rtentry *rt_parent" ;
-A reference to the route from which this route was cloned, or a null
-pointer if this route was not generated by cloning.
-See also the
-.Dv RTF_WASCLONED
-flag.
 .It Vt "struct mtx rt_mtx" ;
 Mutex to lock this routing entry.
 .El
@@ -162,23 +134,12 @@ This route was modified by
 Used only in the
 .Xr route 4
 protocol, indicating that the request was executed.
-.It Dv RTF_CLONING
-When this route is returned as a result of a lookup, automatically
-create a new route using this one as a template and
-.Va rt_genmask
-(if present) as a mask.
 .It Dv RTF_XRESOLVE
 When this route is returned as a result of a lookup, send a report on
 the
 .Xr route 4
 interface requesting that an external process perform resolution for
 this route.
-(Used in conjunction with
-.Dv RTF_CLONING . )
-.It Dv RTF_LLINFO
-Indicates that this route represents information being managed by a
-link layer's adaptation layer (e.g.,
-.Tn ARP ) .
 .It Dv RTF_STATIC
 Indicates that this route was manually added by means of the
 .Xr route 8
@@ -191,14 +152,6 @@ Requests that output sent via this route be discarded.
 Protocol-specific.
 .It Dv RTF_PRCLONING
 This flag is obsolete and simply ignored by facility.
-.It Dv RTF_WASCLONED
-Indicates that this route was generated as a result of cloning
-requested by the
-.Dv RTF_CLONING
-flag.
-When set, the
-.Va rt_parent
-field indicates the route from which this one was generated.
 .It Dv RTF_PINNED
 (Reserved for future use to indicate routes which are not to be
 modified by a routing protocol.)
@@ -296,8 +249,3 @@ The
 and
 .Va rmx_filler
 fields could be named better.
-.Pp
-There is some disagreement over whether it is legitimate for
-.Dv RTF_LLINFO
-to be set by any process other than
-.Va rt_ifa->ifa_rtrequest .