USER_GUIDE_TEST_APP.md

Test Application User Guide

This is the usage guide for the example CDI Test applications cdi_test, cdi_test_min_tx, and cdi_test_min_rx.

In addition to filing CDI-SDK bugs/issues, please use the discussion pages for Q&A, Ideas, Show and Tell or other General topics so the whole community can benefit.

• Test Application User Guide
• Running the minimal test applications
  • Minimal test application help
• Pinning cdi_test Poll Threads to Specific CPU Cores
  • EFA test
• Running the full-featured test application
  • Test application help
  • EFA test with separate transmitter and receiver
  • EFA test with audio, video, and metadata options
    • Video test with pattern
    • Video test with input file
    • Verify output files
    • Audio test
    • Metadata
    • Variable sized ancillary payloads with RIFF files
    • Multi-stream audio/video/metadata
    • Mux/Demux streams
  • Testing CDI with the sockets adapter (not recommended)
  • Testing CDI with the libfabric sockets adapter (preferred)
  • Using file-based command-line argument insertion
    • Rules for file-based command-line insertion
    • Examples
  • Multiple connections
    • Multiple connection example
    • Multiple connection example using shared poll threads
  • Connection names, logging, and display options
    • Naming a connection
    • Logging
      • Directing output to files
      • Directing output to stderr
      • Setting the log level
      • Setting log components
      • Putting it all together
    • Output display options
  • Using AWS CloudWatch with CDI
    • Using AWS CDI SDK with AWS CloudWatch
    • View results on the AWS CloudWatch dashboard
  • Example test content

Running the minimal test applications

Independent transmitter cdi_test_min_tx and receiver cdi_test_min_rx test applications provide a minimal starting point to help understand how to use the SDK from an application perspective. They provide a minimal set of features, focused exclusively on the EFA adapter. Use the commands listed in this section from the aws-cdi-sdk folder.

Note: All of the following example test commands are Linux-centric, but the basic command syntax is the same for Windows. The only difference is that for Linux, executables are found at build/debug|release/bin/cdi_test_min_tx and build/debug|release/bin/cdi_test_min_rx and for Windows executables are found at proj\x64\Debug|Release\cdi_test_min_tx.exe and proj\x64\Debug|Release\cdi_test_min_rx.exe.

Minimal test application help

Running the --help command will display all of the command-line options for the applications.

./build/debug/bin/cdi_test_min_tx --help
./build/debug/bin/cdi_test_min_rx --help

Additionally, there are several build configuration options for the AWS CDI SDK library that aid in debugging and development. For details, refer to: aws-cdi-sdk/src/cdi/configuration.h

Pinning cdi_test Poll Threads to Specific CPU Cores

This section only applies to Linux. The application called cset is used. To install it, see these instructions.

Make sure this has been run first. This command will move kernel threads, so cdi_test can use a specific set of CPU cores. In the example shown below, cdi_test will use CPU cores 1-24.

sudo cset shield -k on -c 1-24

On a system with 72 CPU cores (for example), should get output something like this:

cset: --> activating shielding:
cset: moving 1386 tasks from root into system cpuset...
[==================================================]%
cset: kthread shield activated, moving 45 tasks into system cpuset...
[==================================================]%
cset: **> 35 tasks are not movable, impossible to move
cset: "system" cpuset of CPUSPEC(0,25-71) with 1396 tasks running
cset: "user" cpuset of CPUSPEC(1-24) with 0 tasks running

Then, add the "-core" command line option to each connection that you would like to pin to a specific CPU core. For example, to pin a connection to CPU core 5 use:

-X -core 5 (other options)

To run cdi_test, must use a command line like this:

sudo cset shield -e cdi_test <cdi_test_command_line_options>

EFA test

Two EFA-capable EC2 instances are required for this test. This test runs using EFA with the receiver running on a different EC2 instance from the transmitter.

Before running the test commands, discover the IP addresses of the transmitter and receiver instances as follows:

1. Retrieve the receiver’s IP address by checking the AWS console or by running the following command on the receiver:

   ifconfig

2. The output should look like this:

   eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
   inet 203.0.113.222  netmask 255.255.255.0  broadcast 203.0.113.255
   inet6 2001:DB8::4321:5678:AAAA  prefixlen 64  scopeid 0x20<link>

3. Run the ifconfig command on the transmitter to retrieve its IP address. For this example, assume the transmitter is 203.0.113.111.

   Now run the test applications:

4. After the EC2 instances are configured, run the following command on the receiver (replace 203.0.113.222 with your local IP address):

   ./build/debug/bin/cdi_test_min_rx --local_ip 203.0.113.222 --rx RAW --dest_port 2000 --num_transactions 100 --payload_size 5184000

5. The AWS CDI SDK prints registration and log messages that show it is waiting for a connection from the transmitter:

   [18:15:54] [main:232] Initializing test.
   ...
   [18:15:54] [main:300] Waiting to establish connection with remote target...

6. Switch to the transmitter EC2 instance and run the following command, replacing 203.0.113.111 with your local IP and 203.0.113.222 with the IP address used above:

   ./build/debug/bin/cdi_test_min_tx --local_ip 203.0.113.111 --tx RAW --remote_ip 203.0.113.222 --dest_port 2000 --rate 60 --num_transactions 100 --payload_size 5184000

7. The output on the transmitter console should look like this:

   [18:17:00] [main:331] Initializing test.
   ...
   [18:17:00] [main:404] Connected. Sending payloads...

8. After running the transmit command, the resulting console output should indicate the number of payloads transferred.

Transmitter console output:

[18:17:02] [main:497] All done. Sent [100] payloads. Shutting down...

Receiver console output:

[18:17:02] [main:343] All done. Received [100] payloads. Shutting down...

---

Running the full-featured test application

The test application provides a means to quickly validate that your environment is set up correctly to use the AWS CDI SDK. Use the commands listed in this section from the aws-cdi-sdk folder.

Note: All of the following example test commands are Linux-centric, but the basic command syntax is the same for Windows. The only difference is that for Linux, the executable is found at build/debug|release/bin/cdi_test and for Windows the executable is found at proj\x64\Debug|Release\cdi_test.exe.

Test application help

Running the --help command displays all of the command-line options for the cdi_test application.

./build/debug/bin/cdi_test --help

Additionally, there are several build configuration options for the AWS CDI SDK library that aid in debugging and development. For details refer to: aws-cdi-sdk/src/cdi/configuration.h

EFA test with separate transmitter and receiver

A second EFA-capable EC2 instance is required for this test. This test runs using EFA with the receiver running on a different EC2 instance from the transmitter. It is important that the instances are in the same subnet and VPC and are using the same security group. Before the AWS CDI SDK sends data across the network using SRD, a negotiation between the two instances occurs using sockets. This is used to verify AWS CDI SDK compatibility between the two instances and to verify that the SRD connection is functional. If two instances are not already prepared, follow the instructions for Linux or Windows before continuing.

For this test, the IP addresses of the transmit and receive instances must be known. Follow the instructions in Running the Minimal Test Applications - EFA Test to get the values before continuing.

1. Next, run the following command on the receiver:

   ./build/debug/bin/cdi_test --adapter EFA --local_ip 203.0.113.222 -X --rx RAW --dest_port 2000 --rate 60 --num_transactions 100 -S --payload_size 5184000 --pattern INC

2. Implicit options not listed:

   --pat_start 0
   --buffer_type SGL
   --tx_timeout 16666

3. cdi_test prints registration and log messages that show it is waiting for a connection from the transmitter:

   [18:56:17] [main:325] -- Running CDI Test  -- 18:56:17 02/26/2020
   
   [18:56:17] [RunTestGeneric:231] Registering an adapter.
   [18:56:17] [TestRxCreateThread:633] Setting up Rx connection. Protocol[RAW] Destination Port[2000] Name[Rx_0]
   [18:56:17] [RxCreateInternal:259] Creating Rx connection. Protocol[RAW] Destination Port[2000] Name[Rx_0]
   [18:56:17] [LibFabricEndpointOpen:205] Using [fe80::c3:bcff:fe5f:de8c] for local EFA device GID
   [18:56:23] [ProbeRxControlProcessProbeMode:196] Probe Rx mode[SendReset].

4. Switch to the transmitter instance and run the following command:

   ./build/debug/bin/cdi_test --adapter EFA --local_ip 203.0.113.111 -X --tx RAW --remote_ip 203.0.113.222 --dest_port 2000 --rate 60 --num_transactions 100 -S --payload_size 5184000

5. Implied options not listed:

   --pattern INC  (--pattern defaults to INC for Tx connections)
   --pat_start 0
   --buffer_type SGL
   --tx_timeout 16666

6. The output on the transmitter console should look similar to this:

   Running CDI Test App -- 20:26:33 02/01/2020
   
       [20:26:33] [RunTestGeneric:136] Registering an adapter.

7. After running the transmit command, the resulting console output should indicate the number of successful payloads.

   Transmitter console output:

       [22:23:53] [TestTxSendAllPayloads:556] Connection[] TX Stats:
                  Number of payloads transferred[100]
                  Number of payloads dropped    [0]
                  Number of payloads late       [0]
                  Number of payloads delayed    [0]
        [22:23:53] [main:381] ** Tests PASSED **

   Receiver console output:

       [22:23:53] [TestRxVerify:195] Connection[] RX Stats:
                  Number of payloads transferred[100]
                  Number of payloads dropped    [0]
                  Number of payloads late       [0]
       [22:23:53] [main:381] ** Tests PASSED **

EFA test with audio, video, and metadata options

To test the AVM API, use the commands in this section. To display video configuration options:

./build/debug/bin/cdi_test --help_video

To display audio configuration options:

./build/debug/bin/cdi_test --help_audio

Note: The --id option is required for all AVM connection streams, and must be unique for each stream within a connection.

Video test with pattern

The following test uses the AVM protocol in the AWS CDI SDK to mimic sending video frames. This specific test sends 100 frames of 1080p60 video across two EC2 instances. The transmitter creates an incrementing data pattern and transports this data to the receiver. The receiver accumulates the data and indicate that all data was successfully received.

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Transmitter:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Video test with input file

The following test uses the AVM protocol in the AWS CDI SDK to mimic sending video frames. This specific test sends 100 frames of 1080p60 video across two EC2 instances. The transmitter reads video_out.file, and transports the contents to the receiver. The receiver accumulates the data and writes it to video_in.file. In this example, the receiver does not check incoming data, as the --pattern option is set to NONE and the --file_read option is not used. The received data are written to video_in.file via the --file_write option, allowing users to check the file contents manually by running a SHA operation on both files, as shown below. If the receiver has access to a copy of video_in.file, the --file_read video_in.file option can be used to instruct the receiver to verify the incoming data matches the contents of video_in.file.

Note: See the Example test content section for example files that can be used with the --file_read option in place of video_in.file.

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern NONE --file_write video_in.file --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Transmitter EC2:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --file_read video_out.file --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Verify output files

Perform sha256sum on video_in.file and video_out.file and the two should match.

Receiver:

sha256sum video_in.file

Transmitter:

sha256sum video_out.file

Audio test

The following test uses the AVM protocol in the AWS CDI SDK to mimic sending audio datagrams. This test sends 100 datagrams of an incrementing data pattern across two EC2 instances. To display audio configuration options:

./build/debug/bin/cdi_test --help_audio

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 6216 --pattern INC --avm_audio 51 48KHz none

Transmitter:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 6216 --pattern INC --avm_audio 51 48KHz none

Metadata

The AWS CDI SDK specifies metadata (ancillary) payloads as a collection of all ancillary packets for a given frame grouped together and wrapped as an IETF RFC 8331 payload.

The following test uses the AVM protocol in the AWS CDI SDK to mimic sending ancillary datagrams. This test sends 100 datagrams of an incrementing pattern across two EC2 instances:

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 50 --pattern SHL --pat_start 1 --avm_anc

Transmitter:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 50 --pattern SHL --pat_start 1 --avm_anc

Variable sized ancillary payloads with RIFF files

In most real world applications there is not a fixed amount of ancillary data for every frame. The number of ancillary data packets that must be sent for each frame varies so the payload size varies. To support variable size payloads, the test application supports a simple RIFF file format in which the payload size is declared in a header for every payload. RIFF files are supported for all payload types but ancillary and audio payloads are the most common use cases.

Here is an example of a RIFF file format:

BYTE POSITION	DATA TYPE	DESCRIPTION
1 to 4	FOURCC	"RIFF"
5 to 8	uint32	File size in bytes minus the size of the RIFF chunk header (8 bytes)
9 to 12	FOURCC	"CDI " - This is the Form Type field
13 to 16	FOURCC	"ANC " - Frame #1 ancillary data
17 to 21	uint32	Frame #1 chunk size in bytes
chunk_size number of bytes starting at 22	BYTE[chunk_size]	Frame #1 binary section of ancillary RTP packets

Other than the limits provided by your operating system, the RIFF file has no maximum number of "ANC" chunks in it. Each ANC chunk is sent as a payload of chunk_size bytes. When using a RIFF file, the --payload_size option indicates the max chunk_size for buffer allocation purposes instead of the actual size of the payload that is sent. If using looping content for testing, the RIFF file continues looping until --num_transactions are sent. As long as the RIFF file ends on a completed chunk, it loops back to the beginning of the file.

The following test sends variably sized payloads of ancillary data sourced from a RIFF file and writes back the received data back as a RIFF file. These commands create an ancillary_data_out.cdi file identical to the test content ancillary_data.cdi file:

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 30 -S --id 1 --payload_size 1024 --riff --file_write ancillary_data_out.cdi --avm_anc

Transmitter:

Note: The use of RIFF files as transmit sources requires --buffer_type LINEAR to be added to the transmitter command string.

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 30 --buffer_type LINEAR -S --id 1 --payload_size 1024 --riff --file_read <Test Content Directory>/ancillary_data.cdi --avm_anc

Multi-stream audio/video/metadata

With the multi-stream option, the test application can send multiple AVM data types across the same connection. The following test uses the AVM protocol in the AWS CDI SDK to mimic sending audio, video, and ancillary datagrams across one connection. Note the use of a unique stream ID (--id) to differentiate streams in the connection.

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0 -S --id 2 --payload_size 6216 --pattern INC --avm_audio 51 48KHz EN -S --id 3 --payload_size 50 --pattern SHL --pat_start 1 --avm_anc

Transmitter:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -X --tx AVM --remote_ip <rx-ipv4> --dest_port 2000 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0 -S --id 2 --payload_size 6216 --pattern INC --avm_audio 51 48KHz EN -S --id 3 --payload_size 50 --pattern SHL --pat_start 1 --avm_anc

Mux/Demux streams

The AWS CDI SDK supports muxing and demuxing of multiple streams using a single connection. A diagram that shows this is located at: doc/multi_endpoint_flow.jpg

With the multi-endpoint option (-XS or -new_conns), the test application can send multiple AVM data types to different endpoints. The following test uses the AVM protocol in the AWS CDI SDK to mimic sending video datagrams from a test application and audio datagrams from another test application to unique endpoints within a single connection. This results in both streams being muxed by the receiver into a single connection. Note the use of a unique stream ID (--id) to differentiate streams in the connection.

Receiver:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 3100 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern IGNORE --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0 -S --id 2 --payload_size 2944 --pattern IGNORE --avm_audio "ST" 48kHz none

Transmitter #1 (video datagrams):

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -XS --tx AVM --rate 60 --num_transactions 100 -S --id 1 --remote_ip <rx-ipv4> --dest_port 3100 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Transmitter #2 (audio datagrams):

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -XS --tx AVM --rate 60 --num_transactions 100 -S --id 2 --remote_ip <rx-ipv4> --dest_port 3100 --payload_size 2944 --pattern INC --avm_audio "ST" 48kHz none

The following test uses the AVM protocol in the AWS CDI SDK to mimic sending video and audio datagrams from a single connection within a test application to unique endpoints. This results in the streams being demuxed by the transmitter and sent to different endpoints. Note the use of a unique stream ID (--id) to differentiate streams in the connection.

Receiver #1 (video datagrams):

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 3100 --rate 60 --num_transactions 100 -S --id 1 --payload_size 5184000 --pattern IGNORE --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

Receiver #2 (audio datagrams):

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --rx AVM --dest_port 3200 --rate 60 --num_transactions 100 -S --id 2 --payload_size 2944 --pattern IGNORE --avm_audio "ST" 48kHz none

Transmitter:

./build/debug/bin/cdi_test --adapter EFA --local_ip <tx-ipv4> -XS --tx AVM --rate 60 --num_transactions 100 -S --id 1 --remote_ip <rx-ipv4> --dest_port 3100 --payload_size 5184000 --pattern INC --avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0 -S --id 2 --remote_ip <rx-ipv4> --dest_port 3200 --payload_size 2944 --pattern INC --avm_audio "ST" 48kHz none

Testing CDI with the sockets adapter (not recommended)

The sockets adapter makes it possible to run cdi_test without using underlying EFA hardware by transmitting data through the UDP stack in the kernel. Note that UDP is inherently an unreliable form of transmission. If reliable transmission is required while still prototyping without the EFA adapter, it is recommended to use the libfabric sockets adapter. Furthermore, UDP sockets incur a higher overhead than the libfabric interface, limiting the payload size and increasing latency when compared with using the EFA adapter.

The sockets adapter is provided through the SDK and exposed in the cdi_test application for prototyping purposes and can be utilized by specifying the following command-line option: --adapter SOCKET.

When using the sockets adapter, it is suggested to use payload sizes less than or equal to 58,130 bytes. Larger sizes are not guaranteed to work with the sockets adapter.

Below are example command-line options for cdi_test using the sockets adapter.

Receiver:

./build/debug/bin/cdi_test --adapter SOCKET --local_ip <rx-ipv4> -X --rx RAW --dest_port 2000 --num_transactions 1000 --rate 30 --keep_alive -S --pattern INC --payload_size 1000

Transmitter:

./build/debug/bin/cdi_test --adapter SOCKET --local_ip <tx-ipv4> -X --tx RAW --dest_port 2000 --remote_ip <rx-ipv4> --num_transactions 1000 --rate 30 --keep_alive -S --pattern INC --payload_size 1000

Testing CDI with the libfabric sockets adapter (preferred)

The libfabric sockets adapter provides reliable transport over UDP and is recommended for prototyping on non-EFA platforms because it eliminates unreliable transport as a source of errors that will not occur in production environments. Similar to the EFA adapter, transmitting and receiving larger payload sizes is possible with the libfabric sockets adapter. However, much like the sockets adapter, libfabric sockets will suffer from a latency penalty. It is suggested to only use this adapter for prototyping applications. In contrast to the EFA adapter, which uses only a single port, this adapter uses a consecutive range of ten ports, starting with the destination port.

The libfabric sockets adapter is provided through the SDK and exposed in the cdi_test application by specifying the following command-line option: --adapter SOCKET_LIBFABRIC.

Below are example command-line options for cdi_test using the libfabric sockets adapter.

Receiver:

./build/debug/bin/cdi_test --adapter SOCKET_LIBFABRIC --local_ip <rx-ipv4> -X --rx RAW --dest_port 2000 --num_transactions 1000 --rate 30 --keep_alive -S --pattern INC --payload_size 20000

Transmitter:

./build/debug/bin/cdi_test --adapter SOCKET_LIBFABRIC --local_ip <tx-ipv4> -X --tx RAW --dest_port 2000 --remote_ip <rx-ipv4> --num_transactions 1000 --rate 30 -name single_raw --keep_alive -S --pattern INC --payload_size 20000

Note: If using a Windows instance without an EFA adapter, please see here.

Using file-based command-line argument insertion

In addition to parsing command-line options directly, the cdi_test application can read commands from a file. To use file-based command-line arguments, use the following format in place of usual arguments:

./build/debug/bin/cdi_test @<cmd-filename>

Rules for file-based command-line insertion

• The ‘#’ character is a comment delimiter. Anything after this on the line is ignored.
• Commands can be placed across multiple lines.
• In Windows, if invoking the application on Powershell, the ‘@’ character must be escaped with a backwards tick mark: ‘`’.

Examples

An example of the contents of a file for a typical local loopback test:

# Raw loopback example
# Adapter
--adapter EFA --local_ip 127.0.0.1
# Tx Connection
-X --tx RAW --rate 60 --remote_ip 127.0.0.1 --dest_port 2000 --num_transactions 100
-S --payload_size 5184000 --pattern INC
# Rx Connection
-X --rx RAW --rate 60 --dest_port 2000 --num_transactions 100
-S --payload_size 5184000 --pattern INC

An example of the contents of a file with a RAW transmitter profile and multiple comments:

# Adapter
--adapter EFA                      # adapter type is EFA
--local_ip <tx-ipv4>               # local ip

# Tx Connection
-X
--tx RAW
--remote_ip <rx-ipv4>              # remote ip
--dest_port 2000                   # destination port
--num_transactions 100 --rate 60   # number of transactions and rate
# stream criteria
-S
--payload_size 5184000

Multiple connections

The AWS CDI SDK is designed to allow the creation of multiple connections between EC2 instances. This is advantageous because the EC2 instance can now send data of multiple types, raw data, video, audio, and ancillary data simultaneously.

The --new_conn or -X option creates a new connection for which subsequent options are attached until a new connection is specified or until the command-line options are finished. See Multiple Connection Example for an example of how to use multiple connections for a transmitting EC2 instance and a receiving EC2 instance.

Multiple connection example

The following example uses two EC2 instances. One is a dedicated transmitter with two connections: one sending video and the other sending audio. The other EC2 instance is a dedicated receiver with two connections: one receiving video and the other receiving audio. The transmitter and receiver have separate command files that are invoked by using the method shown in the file-based command-line section.

Receiver command file:

# adapter
--adapter EFA --local_ip <rx-ipv4>

# connection 1: video
-X --rx AVM
--connection_name my_video_rx_1
--dest_port 2000
--rate 60000/1001 --num_transactions 100
-S --id 1
--payload_size 5184000
--pattern INC
--avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

# connection 2: audio
-X --rx AVM
--connection_name my_audio_rx_2
--dest_port 2001
--rate 60 --num_transactions 100
-S --id 1
--payload_size 6216
--pattern INC
--avm_audio 51 48KHz EN

Transmitter command file:

# adapter
--adapter EFA --local_ip <tx-ipv4>

# connection 1: video
-X --tx AVM
--connection_name my_video_tx_1
--remote_ip <rx-ipv4>
--dest_port 2000
--rate 60000/1001 --num_transactions 100
-S --id 1
--payload_size 5184000
--pattern INC
--avm_video 1920 1080 YCbCr422 Unused 10bit 60 1 BT2020 true false PQ Narrow 16 9 0 1080 0 0

# connection 2: audio
-X --tx AVM
--connection_name my_audio_tx_2
--remote_ip <rx-ipv4>
--dest_port 2001
--rate 60 --num_transactions 100
-S --id 1
--payload_size 6216
--pattern INC
--avm_audio 51 48KHz EN

Multiple connection example using shared poll threads

To share a single poll thread across multiple connections, use the -tc <ID> command-line option with the same ID value for each connection. For example, to share a single poll thread for both Rx connections using the receiver command file shown above, add the -tc option as shown below:

...
# connection 1: video
-X -tc 1 --rx AVM
...
# connection 2: audio
-X -tc 1 --rx AVM
...

Connection names, logging, and display options

Naming a connection

By default, the cdi_test application labels connection names with numbers representing the order in which connections were created on the command-line. Use the option --connection_name (or -name) to name connections using more meaningful names.

For example:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> -X --connection_name big_xfer --rx RAW --dest_port 2000 --rate 60 --num_transactions 100 -S --payload_size 5184000 --pattern INC -X --connection_name small_xfer --rx RAW --dest_port 2001 --rate 60 --num_transactions 100 -S --payload_size 1000 --pattern INC

Logging

Directing output to files

The cdi_test application outputs to the console by default, but users can direct output to user-defined log files if desired. There are two options for directing log file output, --log <file path/name> and --logs <file path/name>. The former directs log output to a pair of log files for test application output and SDK output, respectively. The latter option creates the same two files as the --log option, but any connection specific messages are output to a file that is unique to that connection.

For example, --log my.log would create a file called my.log for cdi_test application output, and a file called SDK_my.log for SDK output.

The --logs my.log command would create the files listed above, but neither would contain any connection-specific messages. Instead, connection-specific messages would be output to my.log_0.log, my.log_1.log, etc., for cdi_test application messages and to SDK_my.log_0.log and SDK_my.log_1.log, etc., for SDK messages.

When logging options are used in conjunction with the --connection_name option described above, the connection numbers are replaced with the names provided by the user.

For example:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> --logs my.log -X --connection_name big_xfer --rx RAW --dest_port 2000 --rate 60 --num_transactions 100 -S --payload_size 5184000 --pattern INC -X --connection_name small_xfer --rx RAW --dest_port 2001 --rate 60 --num_transactions 100 -S --payload_size 1000 --pattern INC

This command produces several files, which are shown below:

• my.log – cdi_test messages that are not associated with any particular connection
• my.log_big_xfer.log – cdi_test messages that are associated with connection “big_xfer”
• my.log_small_xfer.log – cdi_test messages that are associated with connection “small_xfer”
• SDK_my.log – SDK messages that are not associated with any particular connection
• SDK_my.log_big_xfer.log – SDK messages that are associated with connection “big_xfer”
• SDK_my.log_small_xfer.log – SDK messages that are associated with connection “small_xfer”

Directing output to stderr

In addition to sending messages to user log files, messages can be directed to stderr using the --stderr option.

Setting the log level

The logging level can be set with the --log_level option. By default the DEBUG log level is enabled, which is the most verbose. See the cdi_test usage message for all log levels.

Setting log components

Some log messages are disabled by default to keep log messages from overwhelming the SDK output. However, there are several groups of log messages, called “components,” that can be enabled using the --log_component command-line option. Multiple components can be enabled by putting them in quotes, separated by spaces. See the cdi_test usage message for all log components.

Putting it all together

Here is an example of a command that sets the output to both a log file as well as stderr, and also sets the logging level to VERBOSE while enabling the PROBE, ENDPOINT_MANAGER, and PERFORMANCE_METRICS log components:

--log log_tx.log --log_component "PROBE ENDPOINT_MANAGER PERFORMANCE_METRICS"
--log_level DEBUG --stderr

Output display options

By default, the cdi_test application outputs to the console. This can be cumbersome to watch when users are interested in tracking then number of payloads sent as well as other statistics about the test run (such as payload latencies, number of errors, etc). To simplify this kind of monitoring, the cdi_test application includes an option called --multiwindow, which uses the curses library to provide a stable display pane that shows statistics and a scrolling pane that shows other cdi_test console output.

Here is an example of the output at the top of the console when using the --multiwindow option:

| Elapsed Time: 00:00:40  |                         Payload Latency (us)                  |      | Connection | Control |
|      Payload Counts     |    Overall    |                 Most Recent Series            |      |            | Command |
| Success | Errors | Late |  Min  |  Max  |  Min  |  P50  |  P90  |  P99  |  Max  | Count | CPU% | Drop Count | Retries |
|     500 |      0 |    0 |  1186 |  1532 |  1195 |  1230 |  1249 |  1280 |  1280 |    27 |    8 |       0    |     0   |
-------------------------------------------------------------------------------------------------------------------------

Below that, regular console messages scroll without interrupting the statistics panel at the top.

Using AWS CloudWatch with CDI

The AWS CloudWatch feature is available to use through the AWS CDI SDK. AWS CloudWatch allows you to monitor the health of the application through specific metrics such as CPU Utilization, Network Traffic In/Out, etc. For a high-level overview of AWS CloudWatch, visit Amazon CloudWatch: Observability of your AWS resources and applications on AWS and on-premises and the AWS CloudWatch Users Guide.

AWS CloudWatch is a service that has both free and paid tiers. For more information on pricing, visit the AWS CloudWatch pricing page.

For the AWS CDI SDK, the statistics generated by the AWS CDI SDK can be published to AWS CloudWatch. To use this functionality, the AWS CLI and AWS SDK C++ must be downloaded and installed using these instructions for Linux or Windows before continuing.

Using AWS CDI SDK with AWS CloudWatch

With the AWS CloudWatch libraries installed, the AWS CDI SDK can publish metrics to AWS CloudWatch. There are dedicated AWS CloudWatch options exposed to the user in the AWS CDI SDK: namespace, dimension domain, and region. Visit Amazon CloudWatch Concepts for more information concerning metrics terminology.

Note: EC2 instances that publish metrics data to CloudWatch must have an IAM role attached that allows them to publish CloudWatch data. See Linux or Windows for IAM policy guidance.

Note: Prefix any cdi_test commands with sudo to reveal any IAM permission errors, which might not be displayed when run without elevated privileges.

Run the following option in the cdi_test application to view AWS CloudWatch usage and statistics:

./build/debug/bin/cdi_test --help_stats

The cdi_test application can be configured to send metrics using the --stats_cloudwatch option, which has the following three required arguments: namespace, region, and dimension.

• The namespace controls the naming of the overall metrics container. This value defaults to CloudDigitalInterface if AWS CloudWatch is enabled and the namespace is set to NULL.
• The region specifies the location to send the metrics, eg. us-west-2. If this value is set to NULL, it defaults to the region in which the SDK is running.
• The dimension domain provides a unique name/value pair for the metric. The domain defaults to the connection name.

Here is an example of using AWS CDI SDK command-line arguments to publish AWS CloudWatch metrics:

./build/debug/bin/cdi_test --adapter EFA --local_ip <rx-ipv4> --stats_cloudwatch CDIStats us-west-2 Stream1 -X --connection_name small_xfer --rx RAW --dest_port 2000 --rate 60 --num_transactions 100 -S --payload_size 1000 --pattern INC --stats_period 1

View results on the AWS CloudWatch dashboard

1. To view the resulting AWS CDI SDK metrics on the AWS CloudWatch dashboard, log on to the AWS console.
2. From the AWS Management Console, locate Management & Governance and then select CloudWatch.
3. From the primary AWS CloudWatch dashboard, select Metrics in the navigation pane.
4. If the AWS CDI SDK successfully posted data to AWS CloudWatch, locate the Namespace that was given to AWS CDI SDK under Custom Namespaces.
5. To interact with the graphing feature, see the AWS CloudWatch User Guide.

Example test content

Test files for use with the cdi_test application's --file_read option may be downloaded from cdi.elemental.com

Content generation tools

The src/tools/convert_image_to_rgb10.py can be used to convert an image in typical formats such as PNG to an uncompressed payload in CDI's 10-bit RGB format.

To install the dependencies needed by this script, run the following command:

pip3 install Pillow bitstring

To convert an image file to an uncompressed payload, run the script as follows:

./src/tools/convert_image_to_rgb10.py <input_image.png> <output_payload.rgb10>

---

[Return to README]