MFC r257155, r257582, r259191, r259192, r259193, r259194, r259195, r259196:

r257155:

Make hastctl list command output current queue sizes.

Reviewed by:	pjd

r257582 (pjd):

Correct alignment.

r259191:

For memsync replication, hio_countdown is used not only as an
indication when a request can be moved to done queue, but also for
detecting the current state of memsync request.

This approach has problems, e.g. leaking a request if memsynk ack from
the secondary failed, or racy usage of write_complete, which should be
called only once per write request, but for memsync can be entered by
local_send_thread and ggate_send_thread simultaneously.

So the following approach is implemented instead:

1) Use hio_countdown only for counting components we waiting to
   complete, i.e. initially it is always 2 for any replication mode.

2) To distinguish between "memsync ack" and "memsync fin" responses
   from the secondary, add and use hio_memsyncacked field.

3) write_complete() in component threads is called only before
   releasing hio_countdown (i.e. before the hio may be returned to the
   done queue).

4) Add and use hio_writecount refcounter to detect when
   write_complete() can be called in memsync case.

Reported by:	Pete French petefrench ingresso.co.uk
Tested by:	Pete French petefrench ingresso.co.uk

r259192:

Add some macros to make the code more readable (no functional chages).

r259193:

Fix compiler warnings.

r259194:

In remote_send_thread, if sending a request fails don't take the
request back from the receive queue -- it might already be processed
by remote_recv_thread, which lead to crashes like below:

  (primary) Unable to receive reply header: Connection reset by peer.
  (primary) Unable to send request (Connection reset by peer):
      WRITE(954662912, 131072).
  (primary) Disconnected from kopusha:7772.
  (primary) Increasing localcnt to 1.
  (primary) Assertion failed: (old > 0), function refcnt_release,
      file refcnt.h, line 62.

Taking the request back was not necessary (it would properly be
processed by the remote_recv_thread) and only complicated things.

r259195:

Send wakeup to threads waiting on empty queue before releasing the
lock to decrease spurious wakeups.

Submitted by:	davidxu

r259196:

Check remote protocol version only for the first connection (when it
is actually sent by the remote node).

Otherwise it generated confusing "Negotiated protocol version 1" debug
messages when processing the second connection.

1 files changed, 5 insertions, 0 deletions

diff --git a/sbin/hastd/hast.h b/sbin/hastd/hast.h
index 65c24f8..c529de5 100644
--- a/sbin/hastd/hast.h
+++ b/sbin/hastd/hast.h
@@ -137,6 +137,8 @@ struct hastd_config {
 #define	HAST_CHECKSUM_CRC32	1
 #define	HAST_CHECKSUM_SHA256	2
 
+struct nv;
+
 /*
  * Structure that describes single resource.
  */
@@ -254,6 +256,9 @@ struct hast_resource {
 	/* Number of activemap flush errors. */
 	uint64_t	hr_stat_activemap_flush_error;
 
+	/* Function to output worker specific info on control status request. */
+	void	(*output_status_aux)(struct nv *);
+
 	/* Next resource. */
 	TAILQ_ENTRY(hast_resource) hr_next;
 };