Add two new generic control messages, NGM_ASCII2BINARY and

NGM_BINARY2ASCII, which convert control messages to ASCII and back.
This allows control messages to be sent and received in ASCII form
using ngctl(8), which makes ngctl a lot more useful.

This also allows all the type-specific debugging code in libnetgraph
to go away -- instead, we just ask the node itself to do the ASCII
translation for us.

Currently, all generic control messages are supported, as well as
messages associated with the following node types: async, cisco,
ksocket, and ppp.

See /usr/share/examples/netgraph/ngctl for an example of using this.

Also give ngctl(8) the ability to print out incoming data and
control messages at any time.  Eventually nghook(8) may be subsumed.

Several other misc. bug fixes.

Reviewed by:	julian

4 files changed, 280 insertions, 4 deletions

diff --git a/share/examples/netgraph/ngctl b/share/examples/netgraph/ngctl
new file mode 100644
index 0000000..6f9507b
--- /dev/null
+++ b/share/examples/netgraph/ngctl
@@ -0,0 +1,173 @@
+# $FreeBSD$
+
+#
+# This is an example that shows how to send ASCII formatted control
+# messages to a node using ngctl(8).
+#
+# What we will do here create a divert(4) tap.  This simply dumps
+# out all packets diverted by some ipfw(8) divert rule to the console.
+#
+# Lines that begin with ``$'' (shell prompt) or ``+'' (ngctl prompt)
+# indicate user input
+#
+
+# First, start up ngctl in interactive mode:
+
+    $ ngctl
+    Available commands:
+      connect    Connects hook <peerhook> of the node at <relpath> to <hook>
+      debug      Get/set debugging verbosity level
+      help       Show command summary or get more help on a specific command
+      list       Show information about all nodes
+      mkpeer     Create and connect a new node to the node at "path"
+      msg        Send a netgraph control message to the node at "path"
+      name       Assign name <name> to the node at <path>
+      read       Read and execute commands from a file
+      rmhook     Disconnect hook "hook" of the node at "path"
+      show       Show information about the node at <path>
+      shutdown   Shutdown the node at <path>
+      status     Get human readable status information from the node at <path>
+      types      Show information about all installed node types
+      quit       Exit program
+    +
+
+# Now let's create a ng_ksocket(8) node, in the family PF_INET,
+# of type SOCK_RAW, and protocol IPPROTO_DIVERT:
+
+    + mkpeer ksocket foo inet/raw/divert
+
+# Note that ``foo'' is the hook name on the socket node, which can be
+# anything.  The ``inet/raw/divert'' is the hook name on the ksocket
+# node, which tells it what kind of socket to create.
+
+# Lets give our ksocket node a global name.  How about ``fred'':
+
+    + name foo fred
+
+# Note that we used ngctl's ``name'' command to do this.  However,
+# the following manually constructed netgraph message would have
+# acomplished the exact same thing:
+
+    + msg foo name { name="fred" }
+
+# Here we are using the ASCII <-> binary control message conversion
+# routines.  ngctl does this for us automatically when we use the
+# ``msg'' command.
+
+# Now lets bind the socket associated with the ksocket node to a port
+# supplied by the system.  We do this by sending the ksocket node a
+# ``bind'' control message.  Again, ngctl does the conversion of the
+# control message from ASCII to binary behind the scenes.
+
+    + msg fred: bind inet/192.168.1.1
+
+# The ksocket accepts arbitrary sockaddr structures, but also has
+# special support for the PF_LOCAL and PF_INET protocol families.
+# That is why we can specify the struct sockaddr argument to the
+# ``bind'' command as ``inet/192.168.1.1'' (since we didn't specify
+# a port number, it's assumed to be zero).  We could have also
+# relied on the generic sockaddr syntax and instead said this:
+
+    + msg fred: bind { family=2 len=16 data=[ 2=192 168 1 1 ] }
+
+# This is what you would have to do for protocol families other
+# that PF_INET and PF_LOCAL, at least until special handling for
+# new ones is added.
+
+# The reason for the ``2=192'' is to skip the two byte IP port number,
+# which causes it to be set to zero, the default value for integral
+# types when parsing.  Now since we didn't ask for a specific port
+# number, we need to do a ``getname'' to see what port number we got:
+
+    + msg fred: getname
+    Rec'd response "getname" (5) from "fred:":
+    Args:   inet/192.168.1.1:1029
+
+# As soon as we sent the message, we got back a response.  Here
+# ngctl is telling us that it received a control message with the
+# NGF_RESP (response) flag set, the reponse was to a prior ``getname''
+# control message, that the originator was the node addressable
+# as ``fred:''.  The message arguments field is then displayed to
+# us in its ASCII form.  In this case, what we get back is a struct
+# sockaddr, and there we see that our port number is 1029.
+
+# So now let's add the ipfw divert rule for whatever packets we
+# want to see.  How about anything from 192.168.1.129.
+
+    + ^Z
+    Suspended
+    $ ipfw add 100 divert 1029 ip from 192.168.1.129 to any
+    00100 divert 1029 ip from 192.168.1.129 to any
+    $ fg
+
+# Now watch what happens when we try to ping from that machine:
+
+    + 
+    Rec'd data packet on hook "foo":
+    0000:  45 00 00 3c 57 00 00 00 20 01 bf ee c0 a8 01 81  E..<W... .......
+    0010:  c0 a8 01 01 08 00 49 5c 03 00 01 00 61 62 63 64  ......I\....abcd
+    0020:  65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74  efghijklmnopqrst
+    0030:  75 76 77 61 62 63 64 65 66 67 68 69              uvwabcdefghi
+    + 
+    Rec'd data packet on hook "foo":
+    0000:  45 00 00 3c 58 00 00 00 20 01 be ee c0 a8 01 81  E..<X... .......
+    0010:  c0 a8 01 01 08 00 48 5c 03 00 02 00 61 62 63 64  ......H\....abcd
+    0020:  65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74  efghijklmnopqrst
+    0030:  75 76 77 61 62 63 64 65 66 67 68 69              uvwabcdefghi
+    + 
+    Rec'd data packet on hook "foo":
+    0000:  45 00 00 3c 59 00 00 00 20 01 bd ee c0 a8 01 81  E..<Y... .......
+    0010:  c0 a8 01 01 08 00 47 5c 03 00 03 00 61 62 63 64  ......G\....abcd
+    0020:  65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74  efghijklmnopqrst
+    0030:  75 76 77 61 62 63 64 65 66 67 68 69              uvwabcdefghi
+    +
+
+# So we're seeing the output from the ksocket socket appear on the ``foo''
+# hook of ngctl's socket node.  Since the packets are getting diverted,
+# the 192.168.1.129 machine doesn't see any response from us.
+
+# Of course, any type of socket can be used, even TCP:
+
+    + mkpeer ksocket bar inet/stream/tcp
+    + msg bar connect inet/192.168.1.33:13
+    ngctl: send msg: Operation now in progress
+    + 
+    Rec'd data packet on hook "foo":
+    0000:  4d 6f 6e 20 4e 6f 76 20 32 39 20 31 37 3a 34 38  Mon Nov 29 17:48
+    0010:  3a 33 37 20 31 39 39 39 0d 0a                    :37 1999..
+    +
+
+# Or, UNIX domain:
+
+    + mkpeer ksocket bar local/stream/0
+    + msg bar bind local/"/tmp/bar.socket"
+    + 
+
+# Here's an example of a more complicated ASCII control message argument.
+# If you look in /sys/netgraph/ng_message.h, you will see that a node
+# responds to a NGM_LISTHOOKS with a struct hooklist, which contains
+# an array of struct linkinfo:
+#
+#     /* Structure used for NGM_LISTHOOKS */
+#     struct linkinfo {
+#             char            ourhook[NG_HOOKLEN + 1];        /* hook name */
+#             char            peerhook[NG_HOOKLEN + 1];       /* peer hook */
+#             struct nodeinfo nodeinfo;
+#     };
+#
+#     struct hooklist {
+#             struct nodeinfo nodeinfo;               /* node information */
+#             struct linkinfo link[0];                /* info about each hook */
+#     };
+#
+# By sending a node the ``listhooks'' command using ngctl, we can see
+# this structure in ASCII form (lines wrapped for readability):
+
+    + msg bar bind local/"/tmp/bar.socket"
+    + msg bar listhooks
+    Rec'd response "listhooks" (7) from "bar":
+    Args:   { nodeinfo={ type="ksocket" id=9 hooks=1 }
+	    linkinfo=[ { ourhook="local/stream/0" peerhook="bar"
+	    nodeinfo={ name="ngctl1327" type="socket" id=8 hooks=1 } } ] }
+
+
diff --git a/share/man/man4/netgraph.4 b/share/man/man4/netgraph.4
index e85e22e..f0d0e85 100644
--- a/share/man/man4/netgraph.4
+++ b/share/man/man4/netgraph.4
@@ -57,7 +57,7 @@ Nodes communicate along the edges to process data, implement protocols, etc.
 The aim of
 .Nm 
 is to supplement rather than replace the existing kernel networking
-infrastructure. It provides:
+infrastructure.  It provides:
 .Pp
 .Bl -bullet -compact -offset 2n
 .It
@@ -132,8 +132,19 @@ in the graph, one edge at a time. The first mbuf in a chain must have the
 .Dv M_PKTHDR
 flag set. Each node decides how to handle data coming in on its hooks.
 .Pp
-Control messages are type-specific structures sent from one node directly
-to an arbitrary other node. There are two ways to address such a message. If
+Control messages are type-specific C structures sent from one node
+directly to some arbitrary other node.  Control messages have a common
+header format, followed by type-specific data, and are binary structures
+for efficiency.  However, node types also may support conversion of the
+type specific data between binary and
+ASCII for debugging and human interface purposes (see the
+.Dv NGM_ASCII2BINARY
+and
+.Dv NGM_BINARY2ASCII
+generic control messages below).  Nodes are not required to support
+these conversions.
+.Pp
+There are two ways to address a control message. If
 there is a sequence of edges connecting the two nodes, the message
 may be ``source routed'' by specifying the corresponding sequence
 of hooks as the destination address for the message (relative
@@ -448,8 +459,17 @@ incrementing
 From a hook you can obtain the corresponding node, and from
 a node the list of all active hooks.
 .Pp
-Node types are described by this structure:
+Node types are described by these structures:
 .Bd -literal
+/** How to convert a control message from binary <-> ASCII */
+struct ng_cmdlist {
+  u_int32_t                  cookie;     /* typecookie */
+  int                        cmd;        /* command number */
+  const char                 *name;      /* command name */
+  const struct ng_parse_type *mesgType;  /* args if !NGF_RESP */
+  const struct ng_parse_type *respType;  /* args if NGF_RESP */
+};
+
 struct ng_type {
   u_int32_t version;                    /* Must equal NG_VERSION */
   const  char *name;                    /* Unique type name */
@@ -476,6 +496,9 @@ struct ng_type {
             struct mbuf *m,                     /* The data in an mbuf */
             meta_p meta);                       /* Meta-data, if any */
   int    (*disconnect)(hook_p hook);    /* Notify disconnection of hook */
+
+  /** How to convert control messages binary <-> ASCII */
+  const struct ng_cmdlist *cmdlist; 	/* Optional; may be NULL */
 };
 .Ed
 .Pp
@@ -554,6 +577,58 @@ purposes only).
 .Pp
 Some modules may choose to implement messages from more than one 
 of the header files and thus recognize more than one type cookie. 
+.Sh Control Message ASCII Form
+Control messages are in binary format for efficiency.  However, for
+debugging and human interface purposes, and if the node type supports
+it, control messages may be converted to and from an equivalent ASCII
+form.  The ASCII form is similar to the binary form, with two exceptions:
+.Pp
+.Bl -tag -compact -width xxx
+.It o
+The
+.Dv cmdstr
+header field must contain the ASCII name of the command, corresponding to the
+.Dv cmd
+header field.
+.It o
+The
+.Dv args
+field contains a NUL-terminated ASCII string version of the message arguments.
+.El
+.Pp
+In general, the arguments field of a control messgage can be any
+arbitrary C data type.  Netgraph includes parsing routines to support
+some pre-defined datatypes in ASCII with this simple syntax:
+.Pp
+.Bl -tag -compact -width xxx
+.It o
+Integer types are represented by base 8, 10, or 16 numbers.
+.It o
+Strings are enclosed in double quotes and respect the normal
+C language backslash escapes.
+.It o
+IP addresses have the obvious form.
+.It o
+Arrays are enclosed in square brackets, with the elements listed
+consecutively starting at index zero.  An element may have an optional
+index and equals sign preceeding it.  Whenever an element
+does not have an explicit index, the index is implicitly the previous
+element's index plus one.
+.It o
+Structures are enclosed in curly braces, and each field is specified
+in the form ``fieldname=value''.
+.It o
+Any array element or structure field whose value is equal to its
+``default value'' may be omitted. For integer types, the default value
+is usually zero; for string types, the empty string.
+.It o
+Array elements and structure fields may be specified in any order.
+.El
+.Pp
+Each node type may define its own arbitrary types by providing
+the necessary routines to parse and unparse.  ASCII forms defined
+for a specific node type are documented in the documentation for
+that node type.
 .Sh Generic Control Messages
 There are a number of standard predefined messages that will work
 for any node, as they are supported directly by the framework itself.
@@ -612,6 +687,32 @@ elect to not support this message. The text response must be less than
 .Dv NG_TEXTRESPONSE
 bytes in length (presently 1024). This can be used to return general
 status information in human readable form.
+.It Dv NGM_BINARY2ASCII
+This generic converts a binary control message to its ASCII form.
+The entire control message to be converted is contained within the
+arguments field of the
+.Dv Dv NGM_BINARY2ASCII
+message itself.  If successful, the reply will contain the same control
+message in ASCII form.
+A node will typically only know how to translate messages that it
+itself understands, so
+the target node of the
+.Dv NGM_BINARY2ASCII
+is often the same node that would actually receive that message.
+.It Dv NGM_ASCII2BINARY
+The opposite of
+.Dv NGM_BINARY2ASCII .
+The entire control message to be converted, in ASCII form, is contained
+in the arguments section of the
+.Dv NGM_ASCII2BINARY
+and need only have the
+.Dv flags ,
+.Dv cmdstr ,
+and
+.Dv arglen
+header fields filled in, plus the NUL-terminated string version of
+the arguments in the arguments field.  If successful, the reply
+contains the binary version of the control message.
 .El
 .Sh Metadata
 Data moving through the
diff --git a/share/man/man4/ng_cisco.4 b/share/man/man4/ng_cisco.4
index 452252f..26370cd 100644
--- a/share/man/man4/ng_cisco.4
+++ b/share/man/man4/ng_cisco.4
@@ -42,6 +42,7 @@
 .Nm ng_cisco
 .Nd Cisco HDLC protocol netgraph node type
 .Sh SYNOPSIS
+.Fd #include <netinet/in.h>
 .Fd #include <netgraph/ng_cisco.h>
 .Sh DESCRIPTION
 The
diff --git a/share/man/man4/ng_pppoe.4 b/share/man/man4/ng_pppoe.4
index c0b0209..69d0016 100644
--- a/share/man/man4/ng_pppoe.4
+++ b/share/man/man4/ng_pppoe.4
@@ -42,6 +42,7 @@
 .Nm ng_pppoe
 .Nd RFC 2516 PPPOE protocol netgraph node type
 .Sh SYNOPSIS
+.Fd #include <net/ethernet.h>
 .Fd #include <netgraph/ng_pppoe.h>
 .Sh DESCRIPTION
 The