diff options
Diffstat (limited to 'contrib/libpcap/doc/pcap.txt')
| -rw-r--r-- | contrib/libpcap/doc/pcap.txt | 1680 |
1 files changed, 0 insertions, 1680 deletions
diff --git a/contrib/libpcap/doc/pcap.txt b/contrib/libpcap/doc/pcap.txt deleted file mode 100644 index cfa6645..0000000 --- a/contrib/libpcap/doc/pcap.txt +++ /dev/null @@ -1,1680 +0,0 @@ - - -Network Working Group L. Degioanni -Internet-Draft F. Risso -Expires: August 30, 2004 Politecnico di Torino - March 2004 - - - PCAP New Generation Dump File Format - pcap - -Status of this Memo - - This document is an Internet-Draft and is in full conformance with - all provisions of Section 10 of RFC2026. - - Internet-Drafts are working documents of the Internet Engineering - Task Force (IETF), its areas, and its working groups. Note that other - groups may also distribute working documents as Internet-Drafts. - - Internet-Drafts are draft documents valid for a maximum of six months - and may be updated, replaced, or obsoleted by other documents at any - time. It is inappropriate to use Internet-Drafts as reference - material or to cite them other than as "work in progress." - - The list of current Internet-Drafts can be accessed at http:// - www.ietf.org/ietf/1id-abstracts.txt. - - The list of Internet-Draft Shadow Directories can be accessed at - http://www.ietf.org/shadow.html. - - This Internet-Draft will expire on August 30, 2004. - -Copyright Notice - - Copyright (C) The Internet Society (2004). All Rights Reserved. - -Abstract - - This document describes a format to dump captured packets on a file. - This format is extensible and it is currently proposed for - implementation in the libpcap/WinPcap packet capture library. - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 1] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -Table of Contents - - 1. Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 3 - 2. General File Structure . . . . . . . . . . . . . . . . . . . . 4 - 2.1 General Block Structure . . . . . . . . . . . . . . . . . . . 4 - 2.2 Block Types . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 2.3 Block Hierarchy and Precedence . . . . . . . . . . . . . . . . 5 - 2.4 Data format . . . . . . . . . . . . . . . . . . . . . . . . . 6 - 3. Block Definition . . . . . . . . . . . . . . . . . . . . . . . 8 - 3.1 Section Header Block (mandatory) . . . . . . . . . . . . . . . 8 - 3.2 Interface Description Block (mandatory) . . . . . . . . . . . 9 - 3.3 Packet Block (optional) . . . . . . . . . . . . . . . . . . . 13 - 3.4 Simple Packet Block (optional) . . . . . . . . . . . . . . . . 15 - 3.5 Name Resolution Block (optional) . . . . . . . . . . . . . . . 16 - 3.6 Interface Statistics Block (optional) . . . . . . . . . . . . 18 - 4. Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 - 5. Experimental Blocks (deserved to a further investigation) . . 23 - 5.1 Other Packet Blocks (experimental) . . . . . . . . . . . . . . 23 - 5.2 Compression Block (experimental) . . . . . . . . . . . . . . . 23 - 5.3 Encryption Block (experimental) . . . . . . . . . . . . . . . 23 - 5.4 Fixed Length Block (experimental) . . . . . . . . . . . . . . 24 - 5.5 Directory Block (experimental) . . . . . . . . . . . . . . . . 25 - 5.6 Traffic Statistics and Monitoring Blocks (experimental) . . . 25 - 5.7 Event/Security Block (experimental) . . . . . . . . . . . . . 25 - 6. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 27 - 7. Most important open issues . . . . . . . . . . . . . . . . . . 28 - Intellectual Property and Copyright Statements . . . . . . . . 29 - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 2] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -1. Objectives - - The problem of exchanging packet traces becomes more and more - critical every day; unfortunately, no standard solutions exist for - this task right now. One of the most accepted packet interchange - formats is the one defined by libpcap, which is rather old and does - not fit for some of the nowadays applications especially in terms of - extensibility. - - This document proposes a new format for dumping packet traces. The - following goals are being pursued: - - o Extensibility: aside of some common functionalities, third parties - should be able to enrich the information embedded in the file with - proprietary extensions, which will be ignored by tools that are - not able to understand them. - - o Portability: a capture trace must contain all the information - needed to read data independently from network, hardware and - operating system of the machine that made the capture. - - o Merge/Append data: it should be possible to add data at the end of - a given file, and the resulting file must still be readable. - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 3] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -2. General File Structure - -2.1 General Block Structure - - A capture file is organized in blocks, that are appended one to - another to form the file. All the blocks share a common format, which - is shown in Figure 1. - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Block Type | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Block Total Length | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / Block Body / - / /* variable length, aligned to 32 bits */ / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Block Total Length | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 1: Basic block structure. - - The fields have the following meaning: - - o Block Type (32 bits): unique value that identifies the block. - Values whose Most Significant Bit (MSB) is equal to 1 are reserved - for local use. They allow to save private data to the file and to - extend the file format. - - o Block Total Length: total size of this block, in bytes. For - instance, a block that does not have a body has a length of 12 - bytes. - - o Block Body: content of the block. - - o Block Total Length: total size of this block, in bytes. This field - is duplicated for permitting backward file navigation. - - This structure, shared among all blocks, makes easy to process a file - and to skip unneeded or unknown blocks. Blocks can be nested one - inside the others (NOTE: needed?). Some of the blocks are mandatory, - i.e. a dump file is not valid if they are not present, other are - optional. - - The structure of the blocks allows to define other blocks if needed. - A parser that does non understand them can simply ignore their - content. - - - -Degioanni & Risso Expires August 30, 2004 [Page 4] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -2.2 Block Types - - The currently defined blocks are the following: - - 1. Section Header Block: it defines the most important - characteristics of the capture file. - - 2. Interface Description Block: it defines the most important - characteristics of the interface(s) used for capturing traffic. - - 3. Packet Block: it contains a single captured packet, or a portion - of it. - - 4. Simple Packet Block: it contains a single captured packet, or a - portion of it, with only a minimal set of information about it. - - 5. Name Resolution Block: it defines the mapping from numeric - addresses present in the packet dump and the canonical name - counterpart. - - 6. Capture Statistics Block: it defines how to store some - statistical data (e.g. packet dropped, etc) which can be useful - to undestand the conditions in which the capture has been made. - - 7. Compression Marker Block: TODO - - 8. Encryption Marker Block: TODO - - 9. Fixed Length Marker Block: TODO - - The following blocks instead are considered interesting but the - authors believe that they deserve more in-depth discussion before - being defined: - - 1. Further Packet Blocks - - 2. Directory Block - - 3. Traffic Statistics and Monitoring Blocks - - 4. Alert and Security Blocks - - TODO Currently standardized Block Type codes are specified in - Appendix 1. - -2.3 Block Hierarchy and Precedence - - The file must begin with a Section Header Block. However, more than - - - -Degioanni & Risso Expires August 30, 2004 [Page 5] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - one Section Header Block can be present on the dump, each one - covering the data following it till the next one (or the end of - file). A Section includes the data delimited by two Section Header - Blocks (or by a Section Header Block and the end of the file), - including the first Section Header Block. - - In case an application cannot read a Section because of different - version number, it must skip everything until the next Section Header - Block. Note that, in order to properly skip the blocks until the next - section, all blocks must have the fields Type and Length at the - beginning. This is a mandatory requirement that must be maintained in - future versions of the block format. - - Figure 2 shows two valid files: the first has a typical - configuration, with a single Section Header that covers the whole - file. The second one contains three headers, and is normally the - result of file concatenation. An application that understands only - version 1.0 of the file format skips the intermediate section and - restart processing the packets after the third Section Header. - - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | SHB v1.0 | Data | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Typical configuration with a single Section Header Block - - - |-- 1st Section --|-- 2nd Section --|-- 3rd Section --| - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | SHB v1.0 | Data | SHB V1.1 | Data | SHB V1.0 | Data | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Configuration with three different Section Header Blocks - - Figure 2: File structure example: the Section Header Block. - - NOTE: TO BE COMPLETED with some examples of other blocks - -2.4 Data format - - Data contained in each section will always be saved according to the - characteristics (little endian / big endian) of the dumping machine. - This refers to all fields that are saved as numbers and that span - over two or more bytes. - - The approach of having each section saved in the native format of the - generating host is more efficient because it avoids translation of - data when reading / writing on the host itself, which is the most - common case when generating/processing capture dumps. - - - -Degioanni & Risso Expires August 30, 2004 [Page 6] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - TODO Probably we have to specify something more here. Is what we're - saying enough to avoid any kind of ambiguity?. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 7] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -3. Block Definition - - This section details the format of the body of the blocks currently - defined. - -3.1 Section Header Block (mandatory) - - The Section Header Block is mandatory. It identifies the beginning of - a section of the capture dump file. Its format is shown in Figure 3. - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Magic | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Major | Minor | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 3: Section Header Block format. - - The meaning of the fields is: - - o Magic: magic number, whose value is the hexadecimal number - 0x1A2B3C4D. This number can be used to distinguish section that - have been saved on little-endian machines from the one saved on - big-endian machines. - - o Major: number of the current mayor version of the format. Current - value is 1. - - o Minor: number of the current minor version of the format. Current - value is 0. - - o Options: optionally, a list of options (formatted according to the - rules defined in Section 4) can be present. - - Aside form the options defined in Section 4, the following options - are valid within this block: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | Hardware | 2 | variable | An ascii | - | | | | string | - - - -Degioanni & Risso Expires August 30, 2004 [Page 8] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - | | | | containing the | - | | | | description of | - | | | | the hardware | - | | | | used to create | - | | | | this section. | - | | | | | - | Operating | 3 | variable | An ascii | - | System | | | string | - | | | | containing the | - | | | | name of the | - | | | | operating | - | | | | system used to | - | | | | create this | - | | | | section. | - | | | | | - | User | 3 | variable | An ascii | - | Application | | | string | - | | | | containing the | - | | | | name of the | - | | | | application | - | | | | used to create | - | | | | this section. | - +----------------+----------------+----------------+----------------+ - - Table 1 - - The Section Header Block does not contain data but it rather - identifies a list of blocks (interfaces, packets) that are logically - correlated. This block does not contain any reference to the size of - the section it is currently delimiting, therefore the reader cannot - skip a whole section at once. In case a section must be skipped, the - user has to repeatedly skip all the blocks contained within it; this - makes the parsing of the file slower but it permits to append several - capture dumps at the same file. - -3.2 Interface Description Block (mandatory) - - The Interface Description Block is mandatory. This block is needed to - specify the characteristics of the network interface on which the - capture has been made. In order to properly associate the captured - data to the corresponding interface, the Interface Description Block - must be defined before any other block that uses it; therefore, this - block is usually placed immediately after the Section Header Block. - - An Interface Description Block is valid only inside the section which - it belongs to. The structure of a Interface Description Block is - shown in Figure 4. - - - - -Degioanni & Risso Expires August 30, 2004 [Page 9] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Interface ID | LinkType | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | SnapLen | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 4: Interface Description Block format. - - The meaning of the fields is: - - o Interface ID: a progressive number that identifies uniquely any - interface inside current section. Two Interface Description Blocks - can have the same Interface ID only if they are in different - sections of the file. The Interface ID is referenced by the packet - blocks. - - o LinkType: a value that defines the link layer type of this - interface. - - o SnapLen: maximum number of bytes dumped from each packet. The - portion of each packet that exceeds this value will not be stored - in the file. - - o Options: optionally, a list of options (formatted according to the - rules defined in Section 4) can be present. - - In addition to the options defined in Section 4, the following - options are valid within this block: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | if_name | 2 | Variable | Name of the | - | | | | device used to | - | | | | capture data. | - | | | | | - | if_IPv4addr | 3 | 8 | Interface | - | | | | network | - | | | | address and | - | | | | netmask. | - | | | | | - | if_IPv6addr | 4 | 17 | Interface | - - - -Degioanni & Risso Expires August 30, 2004 [Page 10] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - | | | | network | - | | | | address and | - | | | | prefix length | - | | | | (stored in the | - | | | | last byte). | - | | | | | - | if_MACaddr | 5 | 6 | Interface | - | | | | Hardware MAC | - | | | | address (48 | - | | | | bits). | - | | | | | - | if_EUIaddr | 6 | 8 | Interface | - | | | | Hardware EUI | - | | | | address (64 | - | | | | bits), if | - | | | | available. | - | | | | | - | if_speed | 7 | 8 | Interface | - | | | | speed (in | - | | | | bps). | - | | | | | - | if_tsaccur | 8 | 1 | Precision of | - | | | | timestamps. If | - | | | | the Most | - | | | | Significant | - | | | | Bit is equal | - | | | | to zero, the | - | | | | remaining bits | - | | | | indicates the | - | | | | accuracy as as | - | | | | a negative | - | | | | power of 10 | - | | | | (e.g. 6 means | - | | | | microsecond | - | | | | accuracy). If | - | | | | the Most | - | | | | Significant | - | | | | Bit is equal | - | | | | to zero, the | - | | | | remaining bits | - | | | | indicates the | - | | | | accuracy as as | - | | | | negative power | - | | | | of 2 (e.g. 10 | - | | | | means 1/1024 | - | | | | of second). If | - | | | | this option is | - | | | | not present, a | - - - -Degioanni & Risso Expires August 30, 2004 [Page 11] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - | | | | precision of | - | | | | 10^-6 is | - | | | | assumed. | - | | | | | - | if_tzone | 9 | 4 | Time zone for | - | | | | GMT support | - | | | | (TODO: specify | - | | | | better). | - | | | | | - | if_flags | 10 | 4 | Interface | - | | | | flags. (TODO: | - | | | | specify | - | | | | better. | - | | | | Possible | - | | | | flags: | - | | | | promiscuous, | - | | | | inbound/outbou | - | | | | nd, traffic | - | | | | filtered | - | | | | during | - | | | | capture). | - | | | | | - | if_filter | 11 | variable | The filter | - | | | | (e.g. "capture | - | | | | only TCP | - | | | | traffic") used | - | | | | to capture | - | | | | traffic. The | - | | | | first byte of | - | | | | the Option | - | | | | Data keeps a | - | | | | code of the | - | | | | filter used | - | | | | (e.g. if this | - | | | | is a libpcap | - | | | | string, or BPF | - | | | | bytecode, and | - | | | | more). More | - | | | | details about | - | | | | this format | - | | | | will be | - | | | | presented in | - | | | | Appendix XXX | - | | | | (TODO). | - | | | | | - | if_opersystem | 12 | variable | An ascii | - | | | | string | - | | | | containing the | - - - -Degioanni & Risso Expires August 30, 2004 [Page 12] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - | | | | name of the | - | | | | operating | - | | | | system of the | - | | | | machine that | - | | | | hosts this | - | | | | interface. | - | | | | This can be | - | | | | different from | - | | | | the same | - | | | | information | - | | | | that can be | - | | | | contained by | - | | | | the Section | - | | | | Header Block | - | | | | (Section 3.1) | - | | | | because the | - | | | | capture can | - | | | | have been done | - | | | | on a remote | - | | | | machine. | - +----------------+----------------+----------------+----------------+ - - Table 2 - - -3.3 Packet Block (optional) - - A Packet Block is the standard container for storing the packets - coming from the network. The Packet Block is optional because packets - can be stored either by means of this block or the Simple Packet - Block, which can be used to speed up dump generation. The format of a - packet block is shown in Figure 5. - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 13] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Interface ID | Drops Count | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Timestamp (High) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Timestamp (Low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Captured Len | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Packet Len | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | | - | Packet Data | - | | - | /* variable length, byte-aligned */ | - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 5: Packet Block format. - - The Packet Block has the following fields: - - o Interface ID: Specifies the interface this packet comes from, and - corresponds to the ID of one of the Interface Description Blocks - present in this section of the file (see Figure 4). - - o Drops Count: a local drop counter. It specified the number of - packets lost (by the interface and the operating system) between - this packet and the preceding one. The value xFFFF (in - hexadecimal) is reserved for those systems in which this - information is not available. - - o Timestamp (High): the most significative part of the timestamp. in - standard Unix format, i.e. from 1/1/1970. - - o Timestamp (Low): the less significative part of the timestamp. The - way to interpret this field is specified by the 'ts_accur' option - (see Figure 4) of the Interface Description block referenced by - this packet. If the Interface Description block does not contain a - 'ts_accur' option, then this field is expressed in microseconds. - - o Captured Len: number of bytes captured from the packet (i.e. the - - - -Degioanni & Risso Expires August 30, 2004 [Page 14] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - length of the Packet Data field). It will be the minimum value - among the actual Packet Length and the snapshot length (defined in - Figure 4). - - o Packet Len: actual length of the packet when it was transmitted on - the network. Can be different from Captured Len if the user wants - only a snapshot of the packet. - - o Packet Data: the data coming from the network, including - link-layer headers. The length of this field is Captured Len. The - format of the link-layer headers depends on the LinkType field - specified in the Interface Description Block (see Section 3.2) and - it is specified in Appendix XXX (TODO). - - o Options: optionally, a list of options (formatted according to the - rules defined in Section 4) can be present. - - -3.4 Simple Packet Block (optional) - - The Simple Packet Block is a lightweight container for storing the - packets coming from the network. Its presence is optional. - - A Simple Packet Block is similar to a Packet Block (see Section 3.3), - but it is smaller, simpler to process and contains only a minimal set - of information. This block is preferred to the standard Packet Block - when performance or space occupation are critical factors, such as in - sustained traffic dump applications. A capture file can contain both - Packet Blocks and Simple Packet Blocks: for example, a capture tool - could switch from Packet Blocks to Simple Packet Blocks when the - hardware resources become critical. - - The Simple Packet Block does not contain the Interface ID field. - Therefore, it must be assumed that all the Simple Packet Blocks have - been captured on the interface previously specified in the Interface - Description Block. - - Figure 6 shows the format of the Simple Packet Block. - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 15] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Packet Len | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | | - | Packet Data | - | | - | /* variable length, byte-aligned */ | - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 6: Simple Packet Block format. - - The Packet Block has the following fields: - - o Packet Len: actual length of the packet when it was transmitted on - the network. Can be different from captured len if the packet has - been truncated. - - o Packet data: the data coming from the network, including - link-layers headers. The length of this field can be derived from - the field Block Total Length, present in the Block Header. - - The Simple Packet Block does not contain the timestamp because this - is one of the most costly operations on PCs. Additionally, there are - applications that do not require it; e.g. an Intrusion Detection - System is interested in packets, not in their timestamp. - - The Simple Packet Block is very efficient in term of disk space: a - snapshot of length 100 bytes requires only 16 bytes of overhead, - which corresponds to an efficiency of more than 86%. - -3.5 Name Resolution Block (optional) - - The Name Resolution Block is used to support the correlation of - numeric addresses (present in the captured packets) and their - corresponding canonical names and it is optional. Having the literal - names saved in the file, this prevents the need of a name resolution - in a delayed time, when the association between names and addresses - can be different from the one in use at capture time. Moreover, The - Name Resolution Block avoids the need of issuing a lot of DNS - requests every time the trace capture is opened, and allows to have - name resolution also when reading the capture with a machine not - connected to the network. - - The format of the Name Resolution Block is shown in Figure 7. - - - - -Degioanni & Risso Expires August 30, 2004 [Page 16] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Record Type | Record Length | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Record Value | - | /* variable length, byte-aligned */ | - | + + + + + + + + + + + + + + + + + + + + + + + + + - | | | | | - +-+-+-+-+-+-+-+-+ + + + + + + + + + + + + + + + + + + + + + + + + - . . . other records . . . - | Record Type == end_of_recs | Record Length == 00 | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 7: Name Resolution Block format. - - A Name Resolution Block is a zero-terminated list of records (in the - TLV format), each of which contains an association between a network - address and a name. There are three possible types of records: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | end_of_recs | 0 | 0 | End of records | - | | | | | - | ip4_rec | 1 | Variable | Specifies an | - | | | | IPv4 address | - | | | | (contained in | - | | | | the first 4 | - | | | | bytes), | - | | | | followed by | - | | | | one or more | - | | | | zero-terminate | - | | | | d strings | - | | | | containing the | - | | | | DNS entries | - | | | | for that | - | | | | address. | - | | | | | - | ip6_rec | 1 | Variable | Specifies an | - | | | | IPv6 address | - | | | | (contained in | - | | | | the first 16 | - | | | | bytes), | - - - -Degioanni & Risso Expires August 30, 2004 [Page 17] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - | | | | followed by | - | | | | one or more | - | | | | zero-terminate | - | | | | d strings | - | | | | containing the | - | | | | DNS entries | - | | | | for that | - | | | | address. | - +----------------+----------------+----------------+----------------+ - - Table 3 - - After the list or Name Resolution Records, optionally, a list of - options (formatted according to the rules defined in Section 4) can - be present. - - A Name Resolution Block is normally placed at the beginning of the - file, but no assumptions can be taken about its position. Name - Resolution Blocks can be added in a second time by tools that process - the file, like network analyzers. - - In addiction to the options defined in Section 4, the following - options are valid within this block: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | ns_dnsname | 2 | Variable | An ascii | - | | | | string | - | | | | containing the | - | | | | name of the | - | | | | machine (DNS | - | | | | server) used | - | | | | to perform the | - | | | | name | - | | | | resolution. | - +----------------+----------------+----------------+----------------+ - - -3.6 Interface Statistics Block (optional) - - The Interface Statistics Block contains the capture statistics for a - given interface and it is optional. The statistics are referred to - the interface defined in the current Section identified by the - Interface ID field. - - The format of the Interface Statistics Block is shown in Figure 8. - - - - -Degioanni & Risso Expires August 30, 2004 [Page 18] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | IfRecv | - | (high + low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | IfDrop | - | (high + low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | FilterAccept | - | (high + low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | OSDrop | - | (high + low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | UsrDelivered | - | (high + low) | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Interface ID | Reserved | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / Options (variable) / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 8: Interface Statistics Block format. - - The fields have the following meaning: - - o IfRecv: number of packets received from the interface during the - capture. This number is reported as a 64 bits value, in which the - most significat bits are located in the first four bytes of the - field. - - o IfDrop: number of packets dropped by the interface during the - capture due to lack of resources. - - o FilterAccept: number of packets accepeted by filter during current - capture. - - o OSDrop: number of packets dropped by the operating system during - the capture. - - o UsrDelivered: number of packets delivered to the user. - UsrDelivered can be different from the value 'FilterAccept - - OSDropped' because some packets could still lay in the OS buffers - when the capture ended. - - - - -Degioanni & Risso Expires August 30, 2004 [Page 19] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - o Interface ID: reference to an Interface Description Block. - - o Reserved: Reserved to future use. - - o Options: optionally, a list of options (formatted according to the - rules defined in Section 4) can be present. - - In addiction to the options defined in Section 4, the following - options are valid within this block: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | isb_starttime | 2 | 8 | Time in which | - | | | | the capture | - | | | | started; time | - | | | | will be stored | - | | | | in two blocks | - | | | | of four bytes | - | | | | each, | - | | | | containing the | - | | | | timestamp in | - | | | | seconds and | - | | | | nanoseconds. | - | | | | | - | isb_endtime | 3 | 8 | Time in which | - | | | | the capture | - | | | | started; time | - | | | | will be stored | - | | | | in two blocks | - | | | | of four bytes | - | | | | each, | - | | | | containing the | - | | | | timestamp in | - | | | | seconds and | - | | | | nanoseconds. | - +----------------+----------------+----------------+----------------+ - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 20] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -4. Options - - Almost all blocks have the possibility to embed optional fields. - Optional fields can be used to insert some information that may be - useful when reading data, but that it is not really needed for packet - processing. Therefore, each tool can be either read the content of - the optional fields (if any), or skip them at once. - - Skipping all the optional fields at once is straightforward because - most of the blocks have a fixed length, therefore the field Block - Length (present in the General Block Structure, see Section 2.1) can - be used to skip everything till the next block. - - Options are a list of Type - Length - Value fields, each one - containing a single value: - - o Option Type (2 bytes): it contains the code that specifies the - type of the current TLV record. Option types whose Most - Significant Bit is equal to one are reserved for local use; - therefore, there is no guarantee that the code used is unique - among all capture files (generated by other applications). In case - of vendor-specific extensions that have to be identified uniquely, - vendors must request an Option Code whose MSB is equal to zero. - - o Option Length (2 bytes): it contains the length of the following - 'Option Value' field. - - o Option Value (variable length): it contains the value of the given - option. The length of this field as been specified by the Option - Length field. - - Options may be repeated several times (e.g. an interface that has - several IP addresses associated to it). The option list is terminated - by a special code which is the 'End of Option'. - - The format of the optional fields is shown in Figure 9. - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 21] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Option Code | Option Length | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Option Value | - | /* variable length, byte-aligned */ | - | + + + + + + + + + + + + + + + + + + + + + + + + + - | / / / | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - / / - / . . . other options . . . / - / / - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Option Code == opt_endofopt | Option Length == 0 | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 9: Options format. - - The following codes can always be present in any optional field: - - +----------------+----------------+----------------+----------------+ - | Name | Code | Length | Description | - +----------------+----------------+----------------+----------------+ - | opt_endofopt | 0 | 0 | End of | - | | | | options: it is | - | | | | used to | - | | | | delimit the | - | | | | end of the | - | | | | optional | - | | | | fields. This | - | | | | block cannot | - | | | | be repeated | - | | | | within a given | - | | | | list of | - | | | | options. | - | | | | | - | opt_comment | 1 | variable | Comment: it is | - | | | | an ascii | - | | | | string | - | | | | containing a | - | | | | comment that | - | | | | is associated | - | | | | to the current | - | | | | block. | - +----------------+----------------+----------------+----------------+ - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 22] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -5. Experimental Blocks (deserved to a further investigation) - -5.1 Other Packet Blocks (experimental) - - Can some other packet blocks (besides the two described in the - previous paragraphs) be useful? - -5.2 Compression Block (experimental) - - The Compression Block is optional. A file can contain an arbitrary - number of these blocks. A Compression Block, as the name says, is - used to store compressed data. Its format is shown in Figure 10. - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Compr. Type | | - +-+-+-+-+-+-+-+-+ | - | | - | Compressed Data | - | | - | /* variable length, byte-aligned */ | - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 10: Compression Block format. - - The fields have the following meaning: - - o Compression Type: specifies the compression algorithm. Possible - values for this field are 0 (uncompressed), 1 (Lempel Ziv), 2 - (Gzip), other?? Probably some kind of dumb and fast compression - algorithm could be effective with some types of traffic (for - example web), but which? - - o Compressed Data: data of this block. Once decompressed, it is made - of other blocks. - - -5.3 Encryption Block (experimental) - - The Encryption Block is optional. A file can contain an arbitrary - number of these blocks. An Encryption Block is used to sotre - encrypted data. Its format is shown in Figure 11. - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 23] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Encr. Type | | - +-+-+-+-+-+-+-+-+ | - | | - | Compressed Data | - | | - | /* variable length, byte-aligned */ | - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 11: Encryption Block format. - - The fields have the following meaning: - - o Compression Type: specifies the encryption algorithm. Possible - values for this field are ??? NOTE: this block should probably - contain other fields, depending on the encryption algorithm. To be - define precisely. - - o Encrypted Data: data of this block. Once decripted, it consists of - other blocks. - - -5.4 Fixed Length Block (experimental) - - The Fixed Length Block is optional. A file can contain an arbitrary - number of these blocks. A Fixed Length Block can be used to optimize - the access to the file. Its format is shown in Figure 12. A Fixed - Length Block stores records with constant size. It contains a set of - Blocks (normally Packet Blocks or Simple Packet Blocks), of wihich it - specifies the size. Knowing this size a priori helps to scan the file - and to load some portions of it without truncating a block, and is - particularly useful with cell-based networks like ATM. - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 24] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - 0 1 2 3 - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | Cell Size | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | - | | - | Fixed Size Data | - | | - | /* variable length, byte-aligned */ | - | | - | | - +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - - Figure 12: Fixed Length Block format. - - The fields have the following meaning: - - o Cell size: the size of the blocks contained in the data field. - - o Fixed Size Data: data of this block. - - -5.5 Directory Block (experimental) - - If present, this block contains the following information: - - o number of indexed packets (N) - - o table with position and length of any indexed packet (N entries) - - A directory block must be followed by at least N packets, otherwise - it must be considered invalid. It can be used to efficiently load - portions of the file to memory and to support operations on memory - mapped files. This block can be added by tools like network analyzers - as a consequence of file processing. - -5.6 Traffic Statistics and Monitoring Blocks (experimental) - - One or more blocks could be defined to contain network statistics or - traffic monitoring information. They could be use to store data - collected from RMON or Netflow probes, or from other network - monitoring tools. - -5.7 Event/Security Block (experimental) - - This block could be used to store events. Events could contain - generic information (for example network load over 50%, server - down...) or security alerts. An event could be: - - - -Degioanni & Risso Expires August 30, 2004 [Page 25] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - o skipped, if the application doesn't know how to do with it - - o processed independently by the packets. In other words, the - applications skips the packets and processes only the alerts - - o processed in relation to packets: for example, a security tool - could load only the packets of the file that are near a security - alert; a monitorg tool could skip the packets captured while the - server was down. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 26] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -6. Conclusions - - The file format proposed in this document should be very versatile - and satisfy a wide range of applications. In the simplest case, it - can contain a raw dump of the network data, made of a series of - Simple Packet Blocks. In the most complex case, it can be used as a - repository for heterogeneous information. In every case, the file - remains easy to parse and an application can always skip the data it - is not interested in; at the same time, different applications can - share the file, and each of them can benfit of the information - produced by the others. Two or more files can be concatenated - obtaining another valid file. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 27] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -7. Most important open issues - - o Data, in the file, must be byte or word aligned? Currently, the - structure of this document is not consistent with respect to this - point. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 28] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - -Intellectual Property Statement - - The IETF takes no position regarding the validity or scope of any - intellectual property or other rights that might be claimed to - pertain to the implementation or use of the technology described in - this document or the extent to which any license under such rights - might or might not be available; neither does it represent that it - has made any effort to identify any such rights. Information on the - IETF's procedures with respect to rights in standards-track and - standards-related documentation can be found in BCP-11. Copies of - claims of rights made available for publication and any assurances of - licenses to be made available, or the result of an attempt made to - obtain a general license or permission for the use of such - proprietary rights by implementors or users of this specification can - be obtained from the IETF Secretariat. - - The IETF invites any interested party to bring to its attention any - copyrights, patents or patent applications, or other proprietary - rights which may cover technology that may be required to practice - this standard. Please address the information to the IETF Executive - Director. - - -Full Copyright Statement - - Copyright (C) The Internet Society (2004). All Rights Reserved. - - This document and translations of it may be copied and furnished to - others, and derivative works that comment on or otherwise explain it - or assist in its implementation may be prepared, copied, published - and distributed, in whole or in part, without restriction of any - kind, provided that the above copyright notice and this paragraph are - included on all such copies and derivative works. However, this - document itself may not be modified in any way, such as by removing - the copyright notice or references to the Internet Society or other - Internet organizations, except as needed for the purpose of - developing Internet standards in which case the procedures for - copyrights defined in the Internet Standards process must be - followed, or as required to translate it into languages other than - English. - - The limited permissions granted above are perpetual and will not be - revoked by the Internet Society or its successors or assignees. - - This document and the information contained herein is provided on an - "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING - TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING - BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION - - - -Degioanni & Risso Expires August 30, 2004 [Page 29] - -Internet-Draft PCAP New Generation Dump File Format March 2004 - - - HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF - MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. - - -Acknowledgment - - Funding for the RFC Editor function is currently provided by the - Internet Society. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Degioanni & Risso Expires August 30, 2004 [Page 30] - |
