-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathdraft-ietf-opsawg-ipfix-on-path-telemetry-10.txt
1736 lines (1087 loc) · 63.8 KB
/
draft-ietf-opsawg-ipfix-on-path-telemetry-10.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Network Working Group T. Graf
Internet-Draft Swisscom
Intended status: Standards Track B. Claise
Expires: 22 January 2025 Huawei
A. Huang Feng
INSA-Lyon
21 July 2024
Export of Delay Performance Metrics in IP Flow Information eXport
(IPFIX)
draft-ietf-opsawg-ipfix-on-path-telemetry-10
Abstract
This document specifies new IP Flow Information Export (IPFIX)
Information Elements to export the On-Path Telemetry measured delay
on the OAM transit and decapsulation nodes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 22 January 2025.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
Graf, et al. Expires 22 January 2025 [Page 1]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . 6
3.1. IP One-Way Delay Hybrid Type I Passive Performance
Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.1. Summary . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2. Description . . . . . . . . . . . . . . . . . . . . . 8
3.1.3. Change Controller . . . . . . . . . . . . . . . . . . 8
3.1.4. Version of Registry Format . . . . . . . . . . . . . 8
3.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 8
3.2.1. Reference Definition . . . . . . . . . . . . . . . . 9
3.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 9
3.3. Method of Measurement . . . . . . . . . . . . . . . . . . 9
3.3.1. Reference Methods . . . . . . . . . . . . . . . . . . 9
3.3.2. Packet Stream Generation . . . . . . . . . . . . . . 10
3.3.3. Traffic Filtering (Observation) Details . . . . . . . 10
3.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 10
3.3.5. Runtime Parameters and Data Format . . . . . . . . . 10
3.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4.2. Reference Definition . . . . . . . . . . . . . . . . 11
3.4.3. Administrative Items . . . . . . . . . . . . . . . . 14
3.4.4. Comments and Remarks . . . . . . . . . . . . . . . . 14
4. IPFIX Information Elements . . . . . . . . . . . . . . . . . 14
5. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
6.1. Performance Metrics . . . . . . . . . . . . . . . . . . . 16
6.2. IPFIX Entities . . . . . . . . . . . . . . . . . . . . . 16
6.2.1. pathDelayMeanDeltaMicroseconds . . . . . . . . . . . 17
6.2.2. pathDelayMinDeltaMicroseconds . . . . . . . . . . . . 18
6.2.3. pathDelayMaxDeltaMicroseconds . . . . . . . . . . . . 18
6.2.4. pathDelaySumDeltaMicroseconds . . . . . . . . . . . . 18
7. Operational Considerations . . . . . . . . . . . . . . . . . 19
7.1. Time Accuracy . . . . . . . . . . . . . . . . . . . . . . 19
7.2. Mean Delay . . . . . . . . . . . . . . . . . . . . . . . 19
7.3. Reduced-size encoding . . . . . . . . . . . . . . . . . . 19
Graf, et al. Expires 22 January 2025 [Page 2]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
7.4. Measurement Interval . . . . . . . . . . . . . . . . . . 19
7.5. IOAM Application . . . . . . . . . . . . . . . . . . . . 20
8. Security Considerations . . . . . . . . . . . . . . . . . . . 20
9. Implementation Status . . . . . . . . . . . . . . . . . . . . 20
9.1. FD.io VPP . . . . . . . . . . . . . . . . . . . . . . . . 21
9.2. Huawei VRP . . . . . . . . . . . . . . . . . . . . . . . 21
9.3. Fluvia . . . . . . . . . . . . . . . . . . . . . . . . . 21
9.4. Pmacct Data Collection . . . . . . . . . . . . . . . . . 22
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
11.1. Normative References . . . . . . . . . . . . . . . . . . 22
11.2. Informative References . . . . . . . . . . . . . . . . . 23
Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . 26
A.1. Aggregated On-Path Dealay Examples . . . . . . . . . . . 26
A.1.1. Template Record and Data Set with Mean Delta . . . . 26
A.1.2. Template Record and Data Set with Sum Delta . . . . . 28
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30
1. Introduction
Network operators usually gather and maintain some forms of
statistical delay view of their networks (or segments of their
networks). That view is meant to help understanding where in the
network, for which customer traffic or services, how much, and why
abnormal delay is being accumulated. To that aim, delay-related data
need to be reported from devices covering both data and control
planes. In order to understand which customer traffic is affected,
delay-related data need to be reported into customer data-plane
context. That enables network operators to quickly identify when the
control-plane updates the current path with a different set of
intermediate hops (that is, a change of the forwarding path) and
interfaces, how the path delay changes for which customer traffic.
With On-Path Telemetry, described in the Network Telemetry Framework
[RFC9232] and applied in In Situ Operations, Administration, and
Maintenance (IOAM) Deployment [RFC9378] and Alternate Marking
Deployment Framework [I-D.ietf-ippm-alt-mark-deployment], the path
delay between two endpoints is measured by inserting a timestamp in
the packet.
At least two modes of On-Path Telemetry can be distinguished.
Passport mode, where only the last hop in the forwarding path of the
On-Path Telemetry domain exposes all the metrics, and postcard mode,
where the metrics are also exposed in transit nodes. In both modes
the forwarding path exposes performance metrics allowing to determine
how much delay has been accumulated on which hop. The proposal in
this document makes more sense for the postcard mode.
Graf, et al. Expires 22 January 2025 [Page 3]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
In order to export the delay-related metrics via IPIFX [RFC7011],
this document defines four new IPFIX Information Elements (IEs),
exposing the On-Path delay on OAM transit and decapsulation nodes,
following the postcard mode principles. Since these IPFIX IEs are
performance metrics [RFC8911], they must be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
Following the guidelines for Registered Performance Metric Requesters
and Reviewers [RFC8911], the different characteristics of the
performance metrics (Identifier, Name, URI, Status, Requester,
Revision, Revision Date, Description, etc.) must be clearly specified
in the "IANA Performance Metric Registry [IANA-PERF-METRIC] in order
for the measurement results using the Performance Metrics to be
comparable even if they are performed using different implementations
and in different networks. The first performance metric
characteristic is the selection of a meaningful name, following the
"MetricType_Method_SubTypeMethod_... Spec_Units_Output" naming
convention (See Section 7.1.2 of [RFC8911]).
+------------------------------------+-------------------------------+
| Performance Metric | IPFIX Information Element |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMeanDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Mean (TBD1)|(TBD5) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMinDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Min (TBD2) |(TBD6) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMaxDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Max (TBD3) |(TBD7) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelaySumDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Sum (TBD4) |(TBD8) |
+------------------------------------+-------------------------------+
Table 1: Mapping Between IPFIX IEs and Performance Metrics
Assuming time synchronization on devices, the delay is measured by
calculating the difference between the timestamp imposed with On-Path
Telemetry in the packet at the OAM encapsulation node and the
timestamp exported in the IPFIX flow record from the OAM transit and
decapsulation nodes. The lowest, highest, mean, and/or the sum of
measured path delay can be exported, thanks to the different IPFIX IE
specifications.
Graf, et al. Expires 22 January 2025 [Page 4]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
On-Path Telemetry Domain
.........................................
. .
. D1 .
. x------> .
. .
. D2 .
. x--------------------> .
. .
. D3 .
. x-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1 Encapsulation Transit Transit Decapsulation Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
Figure 1: Delay use case. Packets flow from host 1 to host 2.
In the usecase shown in Figure 1 using On-path Telemetry to export
the delay metrics, the node R2 exports the delay D1, the node R3
exports the delay D2 and the decapsulation node R4 exports the total
delay D3 using IPFIX.
The advantages of this solution is that the delay metrics (min, max,
and mean) can be computed on the router, and aggregated directly
within the Flow Record, saving export bandwidth and computation on
the Collector. For the computation of the min, max, and mean delay
metric, to be computed locally on the router, the exporter Metering
Process require some local caching/processing computation (for each
new packets in the flow), specifically the mean value. A less
computational heavy solution for the router is the export of the
delay sum instead of the delay mean; on the Collector, the delay mean
can easily be computed by a single division operation (using the
packet count). The alternative, w ith any delay monitoring on the
router, requires the export of every single packet as a separate Flow
Record, including the timestamps information, for the Collector to
compute delay metrics (min, max, and mean) and the recompute the
aggregated Flow Record.
Graf, et al. Expires 22 January 2025 [Page 5]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document makes use of the terms defined in [RFC7011] and
[RFC9378].
The following terms are used as defined in [RFC7011]:
* IPFIX
* IPFIX Information Elements (IEs)
* Flow
* Flow Record
* Exporter
The following terms are used as defined in [RFC8911]:
* Performance Metric
* Registered Performance Metric
* Performance Metrics Registry
The following terms are used as defined in Section 5 of
[I-D.ietf-opsawg-oam-characterization]:
* Encapsulation node
* Transit node
* Decapsulation node
3. Performance Metrics
This section defines the new performance metrics following the
template defined in Section 11 of [RFC8911].
IANA Note (to be removed): RFC 8192 Section 4 was taken a guiding
example.
Graf, et al. Expires 22 January 2025 [Page 6]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
3.1. IP One-Way Delay Hybrid Type I Passive Performance Metrics
This section specifies four performance metrics for the Hybrid Type I
Passive assessment of IP One-Way Delay, to be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
All column entries besides the Identifier, Name, URI, Description,
Output, and Reference Method categories are the same; thus, this
section defines four closely related performance metrics. As a
result, IANA has assigned corresponding URIs to each of the four
registered performance metrics.
3.1.1. Summary
This category includes multiple indexes of the registered performance
metrics: the element Identifier and Metric Name.
3.1.1.1. ID (Identifier)
IANA has allocated the numeric Identifiers TBD1, TBD2, TBD3, and TBD4
for the four Named Metric Entries in the following section.
RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4.
3.1.1.2. Name
TBD1: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
TBD2: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
TBD3: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
TBD4: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace [RFC-to-be].
3.1.1.3. URI
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
Graf, et al. Expires 22 January 2025 [Page 7]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4.
3.1.2. Description
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean: This
metric assesses the mean of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min: This
metric assesses the minimum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max: This
metric assesses the maximum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum: This
metric assesses the sum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
3.1.3. Change Controller
IETF
3.1.4. Version of Registry Format
1.0
3.2. Metric Definition
This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable
document reference and values of input factors, called "Fixed
Parameters".
Graf, et al. Expires 22 January 2025 [Page 8]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
3.2.1. Reference Definition
Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-
Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC
7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-
editor.org/info/rfc7679>. [RFC7679]
Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC
6049, DOI 10.17487/RFC6049, January 2011, <https://www.rfc-
editor.org/info/rfc6049>. [RFC6049]
Section 3.4 of [RFC7679] provides the reference definition of the
singleton (single value) one-way delay metric. Section 4.4 of
[RFC7679] provides the reference definition expanded to cover a
multi-value sample. Note that terms such as "singleton" and "sample"
are defined in Section 2 of [RFC2330].
With the OP [RFC7011] typically located between the hosts
participating in the IP Flow, the one-way delay metric requires one
individual measurement between the OP and sourcing host, such that
the Spatial Composition [RFC6049] of the measurements yields a one-
way delay singleton.
This document specifies how to export the performance metric using
IPFIX.
3.2.2. Fixed Parameters
None
3.3. Method of Measurement
This category includes columns for references to relevant sections of
the RFC(s) and any supplemental information needed to ensure an
unambiguous method for implementations.
3.3.1. Reference Methods
The foundational methodology for this metric is defined in Section 4
of [RFC7323] using the Timestamps option with modifications that
allow application at a mid-path OP [RFC7011].
Graf, et al. Expires 22 January 2025 [Page 9]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
3.3.2. Packet Stream Generation
The timestamp when the packet is being received at OAM encapsulation
node. Format depends on On-Path Telemetry implementation. For IOAM,
Section 4.4.1 of [RFC9197] describes what kind of timestamps are
supported. Section 4.4.2.3 and 4.4.2.4 describe where the timestamp
is being inserted. For the Enhanced Alternate Marking Method,
Section 2 of [I-D.zhou-ippm-enhanced-alternate-marking] describes
timestamp encoding and granularity.
3.3.3. Traffic Filtering (Observation) Details
Runtime Parameters (in the following sections) may be used for
Traffic Filtering.
3.3.4. Sampling Distribution
This metric requires a partial sample of all packets that qualify
according to the Traffic Filter criteria.
3.3.5. Runtime Parameters and Data Format
Runtime Parameters are input factors that must be determined,
configured into a measurement system, and reported with the results
for the context to be complete.
The hybrid type I metering parameters must be reported to provide the
complete measurement context. As an example, if the IPFIX Metering
Process is used, then the IPFIX Metering Process parameters (IPFIX
Template Record, potential traffic filters, and potential sampling
method and parameters) that generate the Flow Records must be
reported to provide the complete measurement context. At a minimum,
the following fields are required:
Src: The IP address of the host in the host A Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see Section 4 of [RFC6991]).
Dst: The IP address of the host in the host B Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see Section 4 of [RFC6991]).
T0: T time, the start of a measurement interval (format "date/time"
Graf, et al. Expires 22 January 2025 [Page 10]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
as specified in Section 5.6 of [RFC3339]; see also "date-and-time"
in Section 3 of [RFC6991]). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", a start time is
unspecified and Tf is to be interpreted as the duration of the
measurement interval. The start time is controlled through other
means.
Tf: A time, the end of a measurement interval (format "date/time" as
specified in Section 5.6 of [RFC3339]; see also "date-and-time" in
Section 3 of [RFC6991])). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", an ending time
and date is ignored and Tf is interpreted as the duration of the
measurement interval.
3.3.6. Roles
host A: Launches an IP packet to start the Flow.
host B: Receives the IP packet to start the Flow.
Encapsulation Node: Receives the IP Flow packets and encapsulates
the timestamp into the packet.
Transit Node: Receives the IP Flow packets and measures the delay
between the timestamp in the packet and the timestamp when the
packet was received.
Decapsulation Node: Receives the IP Flow packets and computes the
delay between the timestamp in the packet and the timestamp when
the packet was received and removes the OAM header from the
packet.
3.4. Output
This category specifies all details of the output of measurements
using the metric.
3.4.1. Type
OWDelay Types are discussed in the subsections below.
3.4.2. Reference Definition
For all output types:
OWDelay_HybridType1_Passive_IP: The one-way delay of one IP packet
is a Singleton
Graf, et al. Expires 22 January 2025 [Page 11]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
For each <statistic> Singleton one of the following subsections
applies.
3.4.2.1. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
Similar to Section 7.4.2.2 of [RFC8912], the mean SHALL be calculated
using the conditional distribution of all packets with a finite value
of one-way delay (undefined delays are excluded) -- a single value,
as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.2.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.2.3 of [RFC6049].
Mean: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
3.4.2.2. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
Similar to Section 7.4.2.3 of [RFC8912], the minimum SHALL be
calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.3.3 of [RFC6049].
Min: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
Graf, et al. Expires 22 January 2025 [Page 12]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
3.4.2.3. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
Similar to Section 7.4.2.4 of [RFC8912], the maximum SHALL be
calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.2 of [RFC6049] for a closely related method for
calculating this statistic; see also Section 4.3.3 of [RFC6049]. The
formula is as follows:
Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
where all packets n = 1 through N have finite singleton delays.
Max: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
3.4.2.4. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
The sum SHALL be calculated using the conditional distribution of all
packets with a finite value of one-way delay (undefined delays are
excluded) -- a single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.5 of [RFC6049] for details on calculating this
statistic. However in this case FiniteDelay or MaxDelay MAY be used.
Sum: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
Graf, et al. Expires 22 January 2025 [Page 13]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
3.4.2.5. Metric Units
* Mean
* Min
* Max
* Sum
The one-way delay of the IP Flow singleton is expressed in seconds.
3.4.2.6. Calibration
Passive Measurements at an OP could be calibrated against an Active
Measurement at host A where the Active Measurement represents the
ground truth.
3.4.3. Administrative Items
3.4.3.1. Status
Current
3.4.3.2. Requester
This RFC
RFC EDITOR NOTE: please replace This RFC text by the RFC issued from
this document
3.4.3.3. Revision
1.0
3.4.3.4. Revision Date
RFC Date
3.4.4. Comments and Remarks
none
4. IPFIX Information Elements
This section specifies the follwing new IPFIX IEs:
Graf, et al. Expires 22 January 2025 [Page 14]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
pathDelayMeanDeltaMicroseconds
32-bit unsigned integer that identifies the mean path delay of all
packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node).
pathDelayMinDeltaMicroseconds
32-bit unsigned integer that identifies the lowest path delay of
all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node).
pathDelayMaxDeltaMicroseconds
32-bit unsigned integer that identifies the highest path delay of
all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node).
pathDelaySumDeltaMicroseconds
64-bit unsigned integer that identifies the sum of the path delay
of all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node).
5. Use Cases
The measured On-Path delay can be aggregated with Flow Aggregation as
defined in [RFC7015] to the following device and control-plane
dimensions to determine:
* With node id and egressInterface(14), on which node which logical
egress interfaces have been contributing to how much delay.
* With node id and egressPhysicalInterface(253), on which node which
physical egress interfaces have been contributing to how much
delay.
* With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the
forwarding path to which next-hop IP contributed to how much
delay.
* With mplsTopLabelIPv4Address(47) or destinationIPv6Address and
srhActiveSegmentIPv6(495), the forwarding path to which MPLS top
label IPv4 address or IPv6 destination address and SRv6 active
segment contributed to how much delay.
Graf, et al. Expires 22 January 2025 [Page 15]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
* BGP communities [RFC1997] are often used for setting a path
priority or service selection. With
bgpDestinationExtendedCommunityList(488) or
bgpDestinationCommunityList(485) or
bgpDestinationLargeCommunityList(491) which group of prefixes
accumulated at which node how much delay.
* With destinationIPv4Address(13), destinationTransportPort(11),
protocolIdentifier (4) and sourceIPv4Address(8), or equivalent
IPFIX IEs for IPv6, the forwarding path delay on each node from
each IPv4 source address to a specific application in the network.
Let us consider the example depicted in Figure 1 from Section 1 as
topology example. Below example table shows the aggregated delay per
each node, ingressInterface,(10) egressInterface(14),
destinationIPv6Address(28) and srhActiveSegmentIPv6(495).
+-----------+-----------+------+-------------+-------------+------------+
| ingress | egress | Node | destination | srhActive | Path Delay |
| Interface | Interface | | IPv6Address | SegmentIPv6 | |
+-----------+-----------+------+-------------+-------------+------------+
| 271 | 276 | R1 | 2001:db8::2 | 2001:db8::4 | 0 us |
+-----------+-----------+------+-------------+-------------+------------+
| 301 | 312 | R2 | 2001:db8::3 | 2001:db8::4 | 22 us |
+-----------+-----------+------+-------------+-------------+------------+
| 22 | 27 | R3 | 2001:db8::4 | 2001:db8::4 | 42 us |
+-----------+-----------+------+-------------+-------------+------------+
| 852 | 854 | R4 | 2001:db8::4 | 2001:db8::4 | 122 us |
+-----------+-----------+------+-------------+-------------+------------+
Table 2: Example table of measured delay. Ascending by delay.
6. IANA Considerations
6.1. Performance Metrics
This document requests IANA to add four new performance metrics under
the "Performance Metrics" registry [RFC8911] with the four templates
defined in Section 3.
6.2. IPFIX Entities
This document requests IANA to register new IPFIX IEs (see table 3)
under the "IPFIX Information Elements" registry [RFC7012] available
at "IANA IP Flow Information Export (IPFIX) Entities Registry
[IANA-IPFIX] and assign the following initial code points.
Graf, et al. Expires 22 January 2025 [Page 16]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
+-------+--------------------------------+
|Element| Name |
| ID | |
+-------+--------------------------------+
| TBD5 | pathDelayMeanDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD6 | pathDelayMinDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD7 | pathDelayMaxDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD8 | pathDelaySumDeltaMicroseconds |
| | |
+-------+--------------------------------+
Table 3: New IPFIX IEs in the "IPFIX Information Elements" Registry
Note to the RFC-Editor:
* Please replace TBD5 - TBD8 with the values allocated by IANA
* Please replace all instances of [RFC-to-be] in this section with
the RFC number assigned to this document
6.2.1. pathDelayMeanDeltaMicroseconds
Name: pathDelayMeanDeltaMicroseconds
ElementID: TBD5
Description: This Information Element identifies the mean path delay
of all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Mean in the IANA Performance Metric Registry.
Graf, et al. Expires 22 January 2025 [Page 17]
Internet-Draft Delay Performance Metrics for IPFIX July 2024
6.2.2. pathDelayMinDeltaMicroseconds
Name: pathDelayMinDeltaMicroseconds
ElementID: TBD6
Description: This Information Element identifies the lowest path
delay of all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node), according to
the OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min in
the IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Min in the IANA Performance Metric Registry.
6.2.3. pathDelayMaxDeltaMicroseconds
Name: pathDelayMaxDeltaMicroseconds
ElementID: TBD7
Description: This Information Element identifies the highest path
delay of all packets in the Flow, in microseconds, between the OAM
encapsulation node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulation node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Max in the IANA Performance Metric Registry.
6.2.4. pathDelaySumDeltaMicroseconds
Name: pathDelaySumDeltaMicroseconds