draft-mcquistin-augmented-ascii-diagrams-06.txt | draft-mcquistin-augmented-ascii-diagrams-07.txt | |||
---|---|---|---|---|
Network Working Group S. McQuistin | Network Working Group S. McQuistin | |||
Internet-Draft V. Band | Internet-Draft V. Band | |||
Intended status: Experimental D. Jacob | Intended status: Experimental D. Jacob | |||
Expires: 14 January 2021 C. S. Perkins | Expires: 6 May 2021 C. S. Perkins | |||
University of Glasgow | University of Glasgow | |||
13 July 2020 | 2 November 2020 | |||
Describing Protocol Data Units with Augmented Packet Header Diagrams | Describing Protocol Data Units with Augmented Packet Header Diagrams | |||
draft-mcquistin-augmented-ascii-diagrams-06 | draft-mcquistin-augmented-ascii-diagrams-07 | |||
Abstract | Abstract | |||
This document describes a machine-readable format for specifying the | This document describes a machine-readable format for specifying the | |||
syntax of protocol data units within a protocol specification. This | syntax of protocol data units within a protocol specification. This | |||
format is comprised of a consistently formatted packet header | format is comprised of a consistently formatted packet header | |||
diagram, followed by structured explanatory text. It is designed to | diagram, followed by structured explanatory text. It is designed to | |||
maintain human readability while enabling support for automated | maintain human readability while enabling support for automated | |||
parser generation from the specification document. This document is | parser generation from the specification document. This document is | |||
itself an example of how the format can be used. | itself an example of how the format can be used. | |||
skipping to change at page 1, line 38 ¶ | skipping to change at page 1, line 38 ¶ | |||
Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
Drafts is at https://datatracker.ietf.org/drafts/current/. | Drafts is at https://datatracker.ietf.org/drafts/current/. | |||
Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
This Internet-Draft will expire on 14 January 2021. | This Internet-Draft will expire on 6 May 2021. | |||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2020 IETF Trust and the persons identified as the | Copyright (c) 2020 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents (https://trustee.ietf.org/ | Provisions Relating to IETF Documents (https://trustee.ietf.org/ | |||
license-info) in effect on the date of publication of this document. | license-info) in effect on the date of publication of this document. | |||
Please review these documents carefully, as they describe your rights | Please review these documents carefully, as they describe your rights | |||
skipping to change at page 2, line 18 ¶ | skipping to change at page 2, line 18 ¶ | |||
Table of Contents | Table of Contents | |||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 | 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 | |||
2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 | 2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 | |||
2.2. Formal languages in standards documents . . . . . . . . . 7 | 2.2. Formal languages in standards documents . . . . . . . . . 7 | |||
3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7 | 3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7 | |||
4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 10 | 4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 10 | |||
4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 10 | 4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 10 | |||
4.2. PDUs That Cross-Reference Previously Defined Fields . . . 13 | 4.2. PDUs That Cross-Reference Previously Defined Fields . . . 13 | |||
4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 15 | 4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 16 | |||
4.4. PDUs with Constraints on Field Values . . . . . . . . . . 16 | 4.4. PDUs with Constraints on Field Values . . . . . . . . . . 16 | |||
4.5. PDUs That Extend Sub-Structures . . . . . . . . . . . . . 17 | 4.5. PDUs That Extend Sub-Structures . . . . . . . . . . . . . 18 | |||
4.6. Storing Data for Parsing . . . . . . . . . . . . . . . . 18 | 4.6. Storing Data for Parsing . . . . . . . . . . . . . . . . 19 | |||
4.7. Connecting Structures with Functions . . . . . . . . . . 19 | 4.7. Connecting Structures with Functions . . . . . . . . . . 20 | |||
4.8. Specifying Enumerated Types . . . . . . . . . . . . . . . 20 | 4.8. Specifying Enumerated Types . . . . . . . . . . . . . . . 21 | |||
4.9. Specifying Protocol Data Units . . . . . . . . . . . . . 21 | 4.9. Specifying Protocol Data Units . . . . . . . . . . . . . 22 | |||
4.10. Importing PDU Definitions from Other Documents . . . . . 22 | 4.10. Importing PDU Definitions from Other Documents . . . . . 22 | |||
5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 22 | 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 22 | |||
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 | 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 | |||
7. Security Considerations . . . . . . . . . . . . . . . . . . . 22 | 7. Security Considerations . . . . . . . . . . . . . . . . . . . 23 | |||
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 | 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 | |||
9. Informative References . . . . . . . . . . . . . . . . . . . 23 | 9. Informative References . . . . . . . . . . . . . . . . . . . 23 | |||
Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 25 | Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 26 | |||
A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 25 | A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 26 | |||
A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 25 | A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 26 | |||
Appendix B. Source code repository . . . . . . . . . . . . . . . 25 | Appendix B. Tooling & source code . . . . . . . . . . . . . . . 26 | |||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 | Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 | |||
1. Introduction | 1. Introduction | |||
Packet header diagrams have become a widely used format for | Packet header diagrams have become a widely used format for | |||
describing the syntax of binary protocols. In otherwise largely | describing the syntax of binary protocols. In otherwise largely | |||
textual documents, they allow for the visualisation of packet | textual documents, they allow for the visualisation of packet | |||
formats, reducing human error, and aiding in the implementation of | formats, reducing human error, and aiding in the implementation of | |||
parsers for the protocols that they specify. | parsers for the protocols that they specify. | |||
Figure 1 gives an example of how packet header diagrams are used to | Figure 1 gives an example of how packet header diagrams are used to | |||
skipping to change at page 4, line 6 ¶ | skipping to change at page 4, line 6 ¶ | |||
is given. | is given. | |||
This document is itself an example of the approach that it describes, | This document is itself an example of the approach that it describes, | |||
with the packet header diagrams and structured text format described | with the packet header diagrams and structured text format described | |||
by example. Examples that do not form part of the protocol | by example. Examples that do not form part of the protocol | |||
description language are marked by a colon at the beginning of each | description language are marked by a colon at the beginning of each | |||
line; this prevents them from being parsed by the accompanying | line; this prevents them from being parsed by the accompanying | |||
tooling. | tooling. | |||
This draft describes early work. As consensus builds around the | This draft describes early work. As consensus builds around the | |||
particular syntax of the format described, both a formal ABNF | particular syntax of the format described, a formal ABNF | |||
specification (Appendix A) and code (Appendix B) that parses it (and, | specification (Appendix A) will be provided. | |||
as described above, this document) will be provided. | ||||
Example specifications of a number of IETF protocols described using | ||||
the Augmented Packet Header Diagram format are available. These | ||||
documents describe UDP [draft-mcquistin-augmented-udp-example], TCP | ||||
[draft-mcquistin-augmented-tcp-example], and QUIC | ||||
[draft-mcquistin-quic-augmented-diagrams]. Code that parses those | ||||
documents and automatically generates parser code for the described | ||||
protocols is described in Appendix B. | ||||
2. Background | 2. Background | |||
This section begins by considering how packet header diagrams are | This section begins by considering how packet header diagrams are | |||
used in existing documents. This exposes the limitations that the | used in existing documents. This exposes the limitations that the | |||
current usage has in terms of machine-readability, guiding the design | current usage has in terms of machine-readability, guiding the design | |||
of the format that this document proposes. | of the format that this document proposes. | |||
While this document focuses on the machine-readability of packet | While this document focuses on the machine-readability of packet | |||
format diagrams, this section also discusses the use of other | format diagrams, this section also discusses the use of other | |||
skipping to change at page 18, line 30 ¶ | skipping to change at page 19, line 10 ¶ | |||
| | | | | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
where: | where: | |||
Long Header (LH): 1 Long Header; LH.T == 3. This field is a | Long Header (LH): 1 Long Header; LH.T == 3. This field is a | |||
previously defined sub-structure. Its constraints can access | previously defined sub-structure. Its constraints can access | |||
fields in that sub-structure. In this example, the T field of the | fields in that sub-structure. In this example, the T field of the | |||
Long Header must be equal to 3. | Long Header must be equal to 3. | |||
Retry Token. This is a variable-length field as previously defined. | Retry Token This is a variable-length field as previously defined. | |||
Retry Integrity Tag: 128 bits. This is a fixed-width field as | Retry Integrity Tag: 128 bits. This is a fixed-width field as | |||
previously defined. | previously defined. | |||
As shown, the Long Header packet sub-structure is included. The | As shown, the Long Header packet sub-structure is included. The | |||
Retry Packet enforces a new value constraint on the Long Packet Type | Retry Packet enforces a new value constraint on the Long Packet Type | |||
(T) field. | (T) field. | |||
4.6. Storing Data for Parsing | 4.6. Storing Data for Parsing | |||
skipping to change at page 20, line 48 ¶ | skipping to change at page 21, line 27 ¶ | |||
comma separated list (with the last element, if there is more than | comma separated list (with the last element, if there is more than | |||
one element, preceded by 'or'), each optionally preceded by "a" or | one element, preceded by 'or'), each optionally preceded by "a" or | |||
"an". The structure names must be defined within the document or a | "an". The structure names must be defined within the document or a | |||
linked document. | linked document. | |||
Where an enumerated type has only two variants, an alternative phrase | Where an enumerated type has only two variants, an alternative phrase | |||
can be used: "The <enumerated type name> is either a <variant 1 name> | can be used: "The <enumerated type name> is either a <variant 1 name> | |||
or <variant 2 name>". The names of the variants must be defined | or <variant 2 name>". The names of the variants must be defined | |||
within the document or a linked document. | within the document or a linked document. | |||
A Frame is either a PING Frame or a HANDSHAKE_DONE Frame. | ||||
A PING Frame is formatted as follows: | A PING Frame is formatted as follows: | |||
0 1 2 3 | 0 1 2 3 | |||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| 1 | | | 1 | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
where: | where: | |||
Frame Type (FT): 1 byte; FT == 1. Frame type, set to 1 for PING | Frame Type (FT): 1 Variable-Length Integer Encoding; FT.T == 1. Fram | |||
frames. | e type, set to 1 for PING frames. | |||
A HANDSHAKE_DONE Frame is formatted as follows: | A HANDSHAKE_DONE Frame is formatted as follows: | |||
0 1 2 3 | 0 1 2 3 | |||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| 30 | | | 30 | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
where: | where: | |||
Frame Type (FT): 1 byte; FT == 30. Frame type, set to 30 for | Frame Type (FT): 1 Variable-Length Integer Encoding; FT.T == 30. Fra | |||
HANDSHAKE_DONE frames. | me type, set to 30 for HANDSHAKE_DONE frames. | |||
A Frame is either a PING Frame or a HANDSHAKE_DONE Frame. | ||||
4.9. Specifying Protocol Data Units | 4.9. Specifying Protocol Data Units | |||
A document will set out different structures that are not, on their | A document will set out different structures that are not, on their | |||
own, protocol data units. To capture the parsing or serialisation of | own, protocol data units. To capture the parsing or serialisation of | |||
a protocol, it is necessary to be able to identify or construct those | a protocol, it is necessary to be able to identify or construct those | |||
packets that are valid PDUs. As a result, it is necessary for the | packets that are valid PDUs. As a result, it is necessary for the | |||
document to identify those structures that are PDUs. | document to identify those structures that are PDUs. | |||
The PDUs that comprise a protocol are identified using the exact | The PDUs that comprise a protocol are identified using the exact | |||
skipping to change at page 25, line 5 ¶ | skipping to change at page 25, line 25 ¶ | |||
[LANGSEC] LANGSEC, "LANGSEC: Language-theoretic Security", | [LANGSEC] LANGSEC, "LANGSEC: Language-theoretic Security", | |||
<http://langsec.org>. | <http://langsec.org>. | |||
[SASSAMAN] Sassaman, L., Patterson, M. L., Bratus, S., and A. | [SASSAMAN] Sassaman, L., Patterson, M. L., Bratus, S., and A. | |||
Shubina, "The Halting Problems of Network Stack | Shubina, "The Halting Problems of Network Stack | |||
Insecurity", ;login: -- December 2011, Volume 36, Number | Insecurity", ;login: -- December 2011, Volume 36, Number | |||
6, <https://www.usenix.org/publications/login/december- | 6, <https://www.usenix.org/publications/login/december- | |||
2011-volume-36-number-6/halting-problems-network-stack- | 2011-volume-36-number-6/halting-problems-network-stack- | |||
insecurity>. | insecurity>. | |||
[draft-mcquistin-augmented-udp-example] | ||||
McQuistin, S., Band, V., Jacob, D., and C. S. Perkins, | ||||
"Describing UDP with Augmented Packet Header Diagrams", | ||||
Work in Progress, Internet-Draft, draft-mcquistin- | ||||
augmented-udp-example-00, 2 November 2020, | ||||
<http://www.ietf.org/internet-drafts/draft-mcquistin- | ||||
augmented-udp-00.txt>. | ||||
[draft-mcquistin-augmented-tcp-example] | ||||
McQuistin, S., Band, V., Jacob, D., and C. S. Perkins, | ||||
"Describing TCP with Augmented Packet Header Diagrams", | ||||
Work in Progress, Internet-Draft, draft-mcquistin- | ||||
augmented-udp-example-00, 2 November 2020, | ||||
<http://www.ietf.org/internet-drafts/draft-mcquistin- | ||||
augmented-tcp-example-00.txt>. | ||||
[draft-mcquistin-quic-augmented-diagrams] | ||||
McQuistin, S., Band, V., Jacob, D., and C. S. Perkins, | ||||
"Describing QUIC's Protocol Data Units with Augmented | ||||
Packet Header Diagrams", Work in Progress, Internet-Draft, | ||||
draft-mcquistin-quic-augmented-diagrams-03, 2 November | ||||
2020, <http://www.ietf.org/internet-drafts/draft- | ||||
mcquistin-quic-augmented-diagrams-03.txt>. | ||||
Appendix A. ABNF specification | Appendix A. ABNF specification | |||
A.1. Constraint Expressions | A.1. Constraint Expressions | |||
field-def = name ["(" short-name ")"] | ||||
[":" sp ((length [";" sp (bool-expr / | ||||
(bool-expr ";" sp presence-constraint))]) / | ||||
(bool-expr [(";" sp presence-constraint)]) / | ||||
presence-constraint)] "." | ||||
presence-constraint = "present only when " bool-expr | ||||
constant = %x31-39 *(%x30-39) ; natural numbers without leading 0s | constant = %x31-39 *(%x30-39) ; natural numbers without leading 0s | |||
short-name = ALPHA *(ALPHA / DIGIT / "-" / "_") | short-name = ALPHA *(ALPHA / DIGIT / "-" / "_") | |||
name = short-name *(" " short-name) | name = short-name *(" " short-name) | |||
sp = [" "] ; optional space in expression | sp = [" "] ; optional space in expression | |||
bool-expr = "(" sp bool-expr sp ")" / | bool-expr = "(" sp bool-expr sp ")" / | |||
"!" sp bool-expr / | "!" sp bool-expr / | |||
bool-expr sp bool-op sp bool-expr / | bool-expr sp bool-op sp bool-expr / | |||
bool-expr sp "?" sp expr sp ":" sp expr / | bool-expr sp "?" sp expr sp ":" sp expr / | |||
expr sp cmp-op sp expr | expr sp cmp-op sp expr | |||
bool-op = "&&" / "||" | bool-op = "&&" / "||" | |||
cmp-op = "==" / "!=" / "<" / "<=" / ">" / ">=" | cmp-op = "==" / "!=" / "<" / "<=" / ">" / ">=" | |||
expr = "(" sp expr sp ")" / | expr = "(" sp expr sp ")" / | |||
expr sp "^" sp expr / | expr sp op sp expr / | |||
expr sp muldiv-op sp expr / | ||||
expr sp addsub-op sp expr / | ||||
bool-expr "?" expr ":" expr / | bool-expr "?" expr ":" expr / | |||
name / short-name "." short-name / | name / short-name "." short-name / | |||
short-name "#" "Size" / | ||||
constant | constant | |||
muldiv-op = "*" / "/" / "%" | op = "+" / "-" / "*" / "/" / "%" / "^" | |||
addsub-op = "+" / "-" | length = expr sp unit / "[" sp name sp "]" | |||
length = expr " " unit / "[" sp name sp "]" | ||||
unit = %s"bit" / %s"bits" / %s"byte" / %s"bytes" / name | unit = %s"bit" / %s"bits" / %s"byte" / %s"bytes" / name | |||
A.2. Augmented packet diagrams | A.2. Augmented packet diagrams | |||
Future revisions of this draft will include an ABNF specification for | Future revisions of this draft will include an ABNF specification for | |||
the augmented packet diagram format described in Section 4. Such a | the augmented packet diagram format described in Section 4. Such a | |||
specification is omitted from this draft given that the format is | specification is omitted from this draft given that the format is | |||
likely to change as its syntax is developed. Given the visual nature | likely to change as its syntax is developed. Given the visual nature | |||
of the format, it is more appropriate for discussion to focus on the | of the format, it is more appropriate for discussion to focus on the | |||
examples given in Section 4. | examples given in Section 4. | |||
Appendix B. Source code repository | Appendix B. Tooling & source code | |||
The source for this draft is available from https://github.com/ | The source for this draft is available from https://github.com/ | |||
glasgow-ipl/draft-mcquistin-augmented-ascii-diagrams. | glasgow-ipl/draft-mcquistin-augmented-ascii-diagrams. | |||
The source code for tooling that can be used to parse this document | The source code for tooling that can be used to parse this document | |||
is available from https://github.com/glasgow-ipl/ips-protodesc-code. | is available from https://github.com/glasgow-ipl/ips-protodesc-code. | |||
This tooling supports the automatic generation of Rust parser code | ||||
from protocol descriptions written in the Augmented Packet Header | ||||
Diagram format. It also provides test harnesses that demonstrate | ||||
that example descriptions of UDP | ||||
[draft-mcquistin-augmented-udp-example] and TCP | ||||
[draft-mcquistin-augmented-udp-example] function as expected. | ||||
Authors' Addresses | Authors' Addresses | |||
Stephen McQuistin | Stephen McQuistin | |||
University of Glasgow | University of Glasgow | |||
School of Computing Science | School of Computing Science | |||
Glasgow | Glasgow | |||
G12 8QQ | G12 8QQ | |||
United Kingdom | United Kingdom | |||
End of changes. 20 change blocks. | ||||
41 lines changed or deleted | 68 lines changed or added | |||
This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |