Add a queue to hold packets while we await an ARP reply.

When a fast machine first brings up some non TCP networking program
it is quite possible that we will drop packets due to the fact that
only one packet can be held per ARP entry.  This leads to packets
being missed when a program starts or restarts if the ARP data is
not currently in the ARP cache.

This code adds a new sysctl, net.link.ether.inet.maxhold, which defines
a system wide maximum number of packets to be held in each ARP entry.
Up to maxhold packets are queued until an ARP reply is received or
the ARP times out.  The default setting is the old value of 1
which has been part of the BSD networking code since time
immemorial.

Expose the time we hold an incomplete ARP entry by adding
the sysctl net.link.ether.inet.wait, which defaults to 20
seconds, the value used when the new ARP code was added..

Reviewed by:	bz, rpaulo
MFC after: 3 weeks

1 files changed, 20 insertions, 3 deletions

diff --git a/sys/net/if_llatbl.c b/sys/net/if_llatbl.c
index a0fb071..3444c97 100644
--- a/sys/net/if_llatbl.c
+++ b/sys/net/if_llatbl.c
@@ -100,18 +100,34 @@ done:
  * This function is called by the timer functions
  * such as arptimer() and nd6_llinfo_timer(), and
  * the caller does the locking.
+ *
+ * Returns the number of held packets, if any, that were dropped.
  */
-void
+size_t
 llentry_free(struct llentry *lle)
 {
-	
+	size_t pkts_dropped;
+	struct mbuf *next;
+
+	pkts_dropped = 0;
 	LLE_WLOCK_ASSERT(lle);
 	LIST_REMOVE(lle, lle_next);
 
-	if (lle->la_hold != NULL)
+	while ((lle->la_numheld > 0) && (lle->la_hold != NULL)) {
+		next = lle->la_hold->m_nextpkt;
 		m_freem(lle->la_hold);
+		lle->la_hold = next;
+		lle->la_numheld--;
+		pkts_dropped++;
+	}
+
+	KASSERT(lle->la_numheld == 0, 
+		("%s: la_numheld %d > 0, pkts_droped %ld", __func__, 
+		 lle->la_numheld, pkts_dropped));
 
 	LLE_FREE_LOCKED(lle);
+
+	return (pkts_dropped);
 }
 
 /*
@@ -412,6 +428,7 @@ llatbl_lle_show(struct llentry_sa *la)
 	db_printf(" lle_tbl=%p\n", lle->lle_tbl);
 	db_printf(" lle_head=%p\n", lle->lle_head);
 	db_printf(" la_hold=%p\n", lle->la_hold);
+	db_printf(" la_numheld=%d\n", lle->la_numheld);
 	db_printf(" la_expire=%ju\n", (uintmax_t)lle->la_expire);
 	db_printf(" la_flags=0x%04x\n", lle->la_flags);
 	db_printf(" la_asked=%u\n", lle->la_asked);