MFH: r309295

bhyve: stability and performance improvement for dbgport

The TCP server implementation in dbgport does not track clients, so it
may try to write to a disconected socket resulting in SIGPIPE.
Avoid that by setting SO_NOSIGPIPE socket option.

Because dbgport emulates an I/O port to guest, the communication is done
byte by byte.  Reduce latency of the TCP/IP transfers by using
TCP_NODELAY option.  In my tests that change improves performance of
kgdb commands with lots of output (e.g. info threads) by two orders of
magnitude.

A general note.  Since we have a uart emulation in bhyve, that can be
used for the console and gdb access to guests.  So, bvmconsole and bvmdebug
could be de-orbited now.  But there are many existing deployments that
still dependend on those.

Discussed with:	julian, jhb
Sponsored by:	Panzura

1 files changed, 12 insertions, 2 deletions

diff --git a/usr.sbin/bhyve/dbgport.c b/usr.sbin/bhyve/dbgport.c
index 1421402..68af025 100644
--- a/usr.sbin/bhyve/dbgport.c
+++ b/usr.sbin/bhyve/dbgport.c
@@ -32,6 +32,7 @@ __FBSDID("$FreeBSD$");
 #include <sys/types.h>
 #include <sys/socket.h>
 #include <netinet/in.h>
+#include <netinet/tcp.h>
 #include <sys/uio.h>
 
 #include <stdio.h>
@@ -55,8 +56,9 @@ static int
 dbg_handler(struct vmctx *ctx, int vcpu, int in, int port, int bytes,
 	    uint32_t *eax, void *arg)
 {
-	char ch;
 	int nwritten, nread, printonce;
+	int on = 1;
+	char ch;
 
 	if (bytes == 2 && in) {
 		*eax = BVM_DBG_SIG;
@@ -74,8 +76,16 @@ again:
 			printonce = 1;
 		}
 		conn_fd = accept4(listen_fd, NULL, NULL, SOCK_NONBLOCK);
-		if (conn_fd < 0 && errno != EINTR)
+		if (conn_fd >= 0) {
+			/* Avoid EPIPE after the client drops off. */
+			(void)setsockopt(conn_fd, SOL_SOCKET, SO_NOSIGPIPE,
+			    &on, sizeof(on));
+			/* Improve latency for one byte at a time tranfers. */
+			(void)setsockopt(conn_fd, IPPROTO_TCP, TCP_NODELAY,
+			    &on, sizeof(on));
+		} else if (errno != EINTR) {
 			perror("accept");
+		}
 	}
 
 	if (in) {