modify protocol description to reflect new encoding

diff --git a/doc/protocol.html b/doc/protocol.html
index bfc64f4..eef749c 100644 (file)
--- a/doc/protocol.html
+++ b/doc/protocol.html
@@ -13,21 +13,238 @@
 
 <h2>Hand-Shake</h2>
 
+<p>The child process connects to the process that started it - i.e. agent connects to controller, runner connects to agent.</p>
+
 <!-- TODO: authentication, TLS -->
+
+<h2>Instance Identification</h2>
+
+<p>Each instance in a controller-runner connection network is identified by a UUID that is dynamically generated when that instance starts.</p>
+
+
+<!-- =========================================================== -->
+<h1>Message Encoding</h1>
+
+<h2>Data Encoding</h2>
+
+<p><b>Integers:</b> are encoded most significant byte first, with a length of 1, 2, 4, or 8 bytes.</p>
+
+<p><b>UUIDs:</b> are encoded in the binary format described in RFC 4122 (16 bytes wide, same order as the printed hex digits).</p>
+
+<p><b>Byte arrays:</b> are naively encoded as a list of bytes.</p>
+
+<p><b>Strings:</b> are encoded in UTF-8 format.</p>
+
+<p>The special <b>BinStrings</b> are encoded by prefixing them with a single byte containing their length in bytes (excluding the length byte itself). The maximum length is currently 127 bytes, bigger length values are reserved for future extensions and should currently be rejected as invalid.</p>
   
 <h2>Framing Format</h2>
 
-(Like HTTP streaming, but bi-directional)
-<pre>
-30\r\n
-&lt;HelloWorld>hi&lt;/HelloWorld>\r\n
-</pre>
+<p>Each frame consists of a length, a header, and optional message body. The length is just a 32bit integer in big endian byte order (most significant first) - it contains the length of the entire message, header plus body, but without the length bytes themselves.</p>
+
+<h2>Message Header</h2>
+
+<p>The message header contains the following fields:</p>
+
+<table border="1" frame="1">
+<tr>
+ <td><b>Field</b>
+ <td><b>Size in Bytes</b>
+ <td><b>Description</b>
+</tr>
+<tr>
+ <td>Message Type
+ <td>1
+ <td>defines the type of message; 0 = notification message, no response expected; 1 = request message, needs a response; 2 = response message
+</tr>
+<tr>
+ <td>Receiver
+ <td>16
+ <td>UUID of the receiver instance of this message, set to the null UUID it means the message is handled and broadcast in the same direction it is currently travelling in (up- or down-stream)
+</tr>
+<tr>
+ <td>Sender
+ <td>16
+ <td>UUID of the sender of the message
+</tr>
+<tr>
+ <td>Transaction ID
+ <td>16
+ <td>identifies the transaction of this message - a request and the corresponding response carry the same transaction ID
+</tr>
+<tr>
+ <td>Function Name
+ <td>1-128
+ <td>BinString describing the purpose of the message, Notifications and Requests must carry a Function Name, while successful Responses usually do not have one - unsuccessful Responses may carry a Function Name like "error" (see Function descriptions for details)
+</tr>
+</table>
+
+<h2>Body Encoding</h2>
+
+<p>If a body is present it is a data item or hierarchy of data items - usually it is encoded as a dictionary of key-value-pairs.</p>
+
+<p>Each data item starts with a type description byte:</p>
+
+<table frame="1" border="1">
+<tr><td>Bit
+ <td>&nbsp;1&nbsp;<br/>MSB
+ <td>&nbsp;2&nbsp;
+ <td>&nbsp;3&nbsp;
+ <td>&nbsp;4&nbsp;
+ <td>&nbsp;5&nbsp;
+ <td>&nbsp;6&nbsp;
+ <td>&nbsp;7&nbsp;
+ <td>&nbsp;8&nbsp;<br/>LSB</tr>
+<tr><td>Data
+ <td colspan="2">Length<br/>Bytes
+ <td colspan="3">Element<br/>Size
+ <td colspan="3">Type Code</tr>
+</table>
+
+<p>The bits describing <b>Length Bytes</b> can have the following values:</p>
+
+<ul>
+<li><tt>00</tt> - the type has no length bytes, it is limited to a single scalar element</li>
+<li><tt>01</tt> - the type has one length byte, meaning it can represent 0-255 elements</li>
+<li><tt>10</tt> - the type has two length bytes, meaning it can represent 0-65535 elements</li>
+<li><tt>11</tt> - the type has four length bytes, meaning it can represent 0-4G elements</li>
+</ul>
+
+<p>The bits describing <b>Element Size</b> can have the following values:</p>
+
+<ul>
+<li><tt>000</tt> - unspecific element size, used for complex types only</li>
+<li><tt>001</tt> - elements are 1 byte in size</li>
+<li><tt>010</tt> - elements are 2 byte in size</li>
+<li><tt>011</tt> - elements are 4 byte in size</li>
+<li><tt>100</tt> - elements are 8 byte in size</li>
+<li><tt>101</tt> - elements are 16 byte in size</li>
+<li><tt>110</tt> - elements are 32 byte in size, currently not used</li>
+<li><tt>111</tt> - elements are 64 byte in size, currently not used</li>
+</ul>
+
+<p>The bits describing the <b>Type Code</b> can have the following values:</p>
+
+<ul>
+<li><tt>000</tt> - Key-Value-Dictionary</li>
+<li><tt>001</tt> - List of data items</li>
+<li><tt>010</tt> - Byte Array</li>
+<li><tt>011</tt> - String data</li>
+<li><tt>100</tt> - Scalar Integer</li>
+<li><tt>101</tt> - Scalar UUID</li>
+<li><tt>110</tt> - Reserved</li>
+<li><tt>111</tt> - Reserved</li>
+</ul>
+
+<p>Taken together these are valid type codes:</p>
+
+<table frame="1" border="1">
+<tr>
+ <td><b>Binary Code</b>
+ <td><b>Hex Code</b>
+ <td><b>Description</b></tr>
+
+<tr>
+ <td>01 000 000 <td>0x40
+ <td>Key-Value Dictionary with one length bytes (max. 255 Key-Value pairs)</tr>
+<tr>
+ <td>10 000 000 <td>0x80
+ <td>Key-Value Dictionary with 2 length bytes (max. 64k Key-Value pairs)</tr>
+<tr>
+ <td>11 000 000 <td>0xC0
+ <td>Key-Value Dictionary with 4 length bytes (max. 4G Key-Value pairs)</tr>
+
+<tr>
+ <td>01 000 001 <td>0x41
+ <td>List with one length bytes (max. 255 items)</tr>
+<tr>
+ <td>10 000 001 <td>0x81
+ <td>List with 2 length bytes (max. 64k items)</tr>
+<tr>
+ <td>11 000 001 <td>0xC1
+ <td>List with 4 length bytes (max. 4G items)</tr>
+
+<tr>
+ <td>01 001 010 <td>0x4A
+ <td>Byte Array with one length bytes (max. 255 items)</tr>
+<tr>
+ <td>10 001 010 <td>0x8A
+ <td>Byte Array with one length bytes (max. 64k items)</tr>
+<tr>
+ <td>11 001 010 <td>0xCA
+ <td>Byte Array with one length bytes (max. 4G items)</tr>
+
+<tr>
+ <td>01 001 011 <td>0x4B
+ <td>String with one length bytes (max. 255 items)</tr>
+<tr>
+ <td>10 001 011 <td>0x8B
+ <td>String with one length bytes (max. 64k items)</tr>
+<tr>
+ <td>11 001 011 <td>0xCB
+ <td>String with one length bytes (max. 4G items)</tr>
+
+<tr>
+ <td>00 001 100 <td>0x0C
+ <td>One byte signed Integer</tr>
+<tr>
+ <td>00 010 100 <td>0x14
+ <td>Two byte signed Integer</tr>
+<tr>
+ <td>00 011 100 <td>0x1C
+ <td>Four byte signed Integer</tr>
+<tr>
+ <td>00 100 100 <td>0x24
+ <td>Eight byte signed Integer</tr>
+
+<tr>
+ <td>00 101 101 <td>0x2D
+ <td>UUID - 16 bytes, binary notation</tr>
+</table>
+
+<p>All other Type Codes are currently invalid.</p>
+
+<h3>Encoding of Scalar Values</h3>
+
+<p>For all scalar values (Integers, UUIDs) the type code is immediately followed by the binary encoded value.</p>
+
+<p>E.g. The byte sequence <tt>14&nbsp;07&nbsp;D0</tt> describes the two byte integer 2000 (0x7d0 in hex).</p>
+
+<h3>Encoding of Byte Arrays and Strings</h3>
+
+<p>The type code is followed by the length bytes, which contain the amount of bytes it takes to encode the value. The value bytes follow the length bytes.</p>
+
+<p>E.g. the byte sequence <tt>4B&nbsp;0D&nbsp;48&nbsp;65&nbsp;6c&nbsp;6c&nbsp;c3&nbsp;b6&nbsp;20&nbsp;57&nbsp;c3&nbsp;b6&nbsp;72&nbsp;6c&nbsp;64</tt> describes a UTF-8 encoded string with one length byte (<tt>4B</tt>), with a length of 13 bytes (<tt>0D</tt>) and the value "Hell&ouml; W&ouml;rld" ("&ouml;" is encoded as c3b6).</p>
+
+<h3>Encoding of Complex Types</h3>
+
+<p>Complex types are encoded as type code followed by length bytes, followed by values. The length bytes contain the number of sub-items encoded in this complex item.</p>
+
+<p>For lists each item is just encoded exactly as described above.</p>
+
+<p>For key-value dictionaries each item is encoded as a key in BinString format, followed by the item encoded as described above. Dictionaries may contain several values for the same key, by encoding several items with the same key.</p>
+
+<p>Lists and Dictionaries can be recursively nested.</p>
+
+<p>E.g. <tt>4102&nbsp;0c2f&nbsp;4b05&nbsp;68656c6c6f</tt> represents a list with one length byte and 2 items, specifically the one-byte integer 47 (0x2f) and the string "hello".</p>
+
+<p>E.g. <tt>40&nbsp;03&nbsp;0131&nbsp;0c2a&nbsp;0131&nbsp;0c2f&nbsp;023132&nbsp;0c2b</tt> represents a dictionary with two keys ("1" and "12") and three values:
+<ul>
+ <li><tt>40 03</tt> - type code and length byte, indicating 3 items
+ <li><tt>0131 0c2a</tt> - key "1" encoded as BinString (<tt>0131</tt>), value is an integer 42
+ <li><tt>0131 0c2f</tt> - key "1" encoded as BinString (<tt>0131</tt>), value is an integer 47
+ <li><tt>023132 0c2b</tt> - key "12" encoded as BinString (<tt>023132</tt>), value is an integer 43
+</ul></p>
+
+
+
+<!-- =========================================================== -->
+<h1>Message Functions</h1>
 
-<p>Frame content is always XML.</p>
+<!-- TODO: reformat messages -->
 
-<h2>Generic Messages</h2>
+<h2>Generic Functions</h2>
 
-<p>Whenever a Runner receives a message that it does not understand, it can reject it with the following notification:</p>
+<p>Whenever a Runner receives a message that it does not understand, it can reject it with the following error response:</p>
 
 <pre>
 &lt;RejectMsg reason="unknownmessage" message="ProvokeError" reqid="1234z" originalsize="456"/>