Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ndpiReader: print how many packets (per flow) were needed to perform full DPI #1891

Merged
merged 1 commit into from
Mar 1, 2023
+8,881 −8,878
Merged

ndpiReader: print how many packets (per flow) were needed to perform full DPI #1891

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions example/ndpiReader.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1425,6 +1425,7 @@ static void printFlow(u_int32_t id, struct ndpi_flow_info *flow, u_int16_t threa
#if 0
fprintf(out, "[Num calls: %d]", flow->num_dissector_calls);
#endif
fprintf(out, "[DPI packets: %d]", flow->dpi_packets);

if(flow->detected_protocol.category != 0)
fprintf(out, "[cat: %s/%u]",
Expand Down
1 change: 1 addition & 0 deletions example/reader_util.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1662,6 +1662,7 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
workflow->stats.dpi_packet_count[1]++;
else
workflow->stats.dpi_packet_count[2]++;
flow->dpi_packets++;

memset(&input_info, '\0', sizeof(input_info)); /* To be sure to set to "unknown" any fields */
/* Set here any information (easily) available; in this trivial example we don't have any */
Expand Down
1 change: 1 addition & 0 deletions example/reader_util.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@ typedef struct ndpi_flow_info {
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
u_int16_t num_dissector_calls;
u_int16_t dpi_packets;

// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
Expand Down
394 changes: 197 additions & 197 deletions tests/result/1kxun.pcap.out
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion tests/result/443-chrome.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ Patricia protocols: 2/0 (search/found)

TLS 1 1506 1

1 TCP 178.62.197.130:443 -> 192.168.1.13:53059 [proto: 91/TLS][IP: 0/Unknown][Encrypted][Confidence: Match by port][cat: Web/5][1 pkts/1506 bytes -> 0 pkts/0 bytes][Goodput ratio: 96/0][< 1 sec][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No client to server traffic][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0]
1 TCP 178.62.197.130:443 -> 192.168.1.13:53059 [proto: 91/TLS][IP: 0/Unknown][Encrypted][Confidence: Match by port][DPI packets: 1][cat: Web/5][1 pkts/1506 bytes -> 0 pkts/0 bytes][Goodput ratio: 96/0][< 1 sec][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No client to server traffic][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0]
2 changes: 1 addition & 1 deletion tests/result/443-curl.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ JA3 Host Stats:
1 192.168.1.13 1


1 TCP 192.168.1.13:55523 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Network/14][51 pkts/4260 bytes <-> 58 pkts/69722 bytes][Goodput ratio: 22/94][1.10 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;http/1.1][(Negotiated) ALPN: h2][bytes ratio: -0.885 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/19 784/784 122/114][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 84/1202 583/1506 74/562][TLSv1.2][JA3C: 2a26b1a62e40d25d4de3babc9d532f30][ServerNames: www.ntop.org][JA3S: ae53107a2e47ea20c72ac44821a728bf][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Firefox][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 3,13,1,1,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,73,0,0]
1 TCP 192.168.1.13:55523 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][DPI packets: 7][cat: Network/14][51 pkts/4260 bytes <-> 58 pkts/69722 bytes][Goodput ratio: 22/94][1.10 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;http/1.1][(Negotiated) ALPN: h2][bytes ratio: -0.885 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/19 784/784 122/114][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 84/1202 583/1506 74/562][TLSv1.2][JA3C: 2a26b1a62e40d25d4de3babc9d532f30][ServerNames: www.ntop.org][JA3S: ae53107a2e47ea20c72ac44821a728bf][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Firefox][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 3,13,1,1,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,0,73,0,0]
2 changes: 1 addition & 1 deletion tests/result/443-firefox.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ JA3 Host Stats:
1 192.168.1.13 1


1 TCP 192.168.1.13:53096 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Network/14][316 pkts/28495 bytes <-> 351 pkts/429572 bytes][Goodput ratio: 27/95][8.44 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;http/1.1][(Negotiated) ALPN: h2][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.876 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 32/20 4007/4045 285/250][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 90/1224 583/1506 58/472][TLSv1.2][JA3C: b20b44b18b853ef29ab773e921b03422][ServerNames: www.ntop.org][JA3S: 3653a20186a5b490426131a611e01992][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Firefox][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256][Plen Bins: 1,0,1,6,7,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,1,0,0,0,0,1,70,0,0]
1 TCP 192.168.1.13:53096 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][DPI packets: 7][cat: Network/14][316 pkts/28495 bytes <-> 351 pkts/429572 bytes][Goodput ratio: 27/95][8.44 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;http/1.1][(Negotiated) ALPN: h2][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.876 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 32/20 4007/4045 285/250][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 90/1224 583/1506 58/472][TLSv1.2][JA3C: b20b44b18b853ef29ab773e921b03422][ServerNames: www.ntop.org][JA3S: 3653a20186a5b490426131a611e01992][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Firefox][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256][Plen Bins: 1,0,1,6,7,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,1,0,0,0,0,1,70,0,0]
2 changes: 1 addition & 1 deletion tests/result/443-git.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ JA3 Host Stats:
1 192.168.1.13 1


1 TCP 192.168.1.13:55744 <-> 140.82.114.4:443 [proto: 91.203/TLS.Github][IP: 203/Github][Encrypted][Confidence: DPI][cat: Collaborative/15][35 pkts/3167 bytes <-> 35 pkts/34022 bytes][Goodput ratio: 28/93][0.82 sec][Hostname/SNI: github.com][(Advertised) ALPNs: http/1.1][(Negotiated) ALPN: http/1.1][bytes ratio: -0.830 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/15 143/143 48/43][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 90/972 583/1490 94/616][TLSv1.2][JA3C: 2a26b1a62e40d25d4de3babc9d532f30][ServerNames: github.com,www.github.com][JA3S: ae53107a2e47ea20c72ac44821a728bf][Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert SHA2 Extended Validation Server CA][Subject: C=US, ST=California, L=San Francisco, O=GitHub, Inc., CN=github.com][Certificate SHA-1: CA:06:F5:6B:25:8B:7A:0D:4F:2B:05:47:09:39:47:86:51:15:19:84][Firefox][Validity: 2018-05-08 00:00:00 - 2020-06-03 12:00:00][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 5,8,2,5,0,0,2,0,0,2,0,2,0,2,0,0,2,2,0,2,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,54,0,0,0]
1 TCP 192.168.1.13:55744 <-> 140.82.114.4:443 [proto: 91.203/TLS.Github][IP: 203/Github][Encrypted][Confidence: DPI][DPI packets: 7][cat: Collaborative/15][35 pkts/3167 bytes <-> 35 pkts/34022 bytes][Goodput ratio: 28/93][0.82 sec][Hostname/SNI: github.com][(Advertised) ALPNs: http/1.1][(Negotiated) ALPN: http/1.1][bytes ratio: -0.830 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/15 143/143 48/43][Pkt Len c2s/s2c min/avg/max/stddev: 54/66 90/972 583/1490 94/616][TLSv1.2][JA3C: 2a26b1a62e40d25d4de3babc9d532f30][ServerNames: github.com,www.github.com][JA3S: ae53107a2e47ea20c72ac44821a728bf][Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert SHA2 Extended Validation Server CA][Subject: C=US, ST=California, L=San Francisco, O=GitHub, Inc., CN=github.com][Certificate SHA-1: CA:06:F5:6B:25:8B:7A:0D:4F:2B:05:47:09:39:47:86:51:15:19:84][Firefox][Validity: 2018-05-08 00:00:00 - 2020-06-03 12:00:00][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 5,8,2,5,0,0,2,0,0,2,0,2,0,2,0,0,2,2,0,2,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,54,0,0,0]
2 changes: 1 addition & 1 deletion tests/result/443-opvn.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ Patricia protocols: 2/0 (search/found)

OpenVPN 46 11573 1

1 TCP 192.168.1.84:52973 <-> 192.12.192.103:1194 [proto: 159/OpenVPN][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: VPN/2][25 pkts/5636 bytes <-> 21 pkts/5937 bytes][Goodput ratio: 70/77][8.96 sec][bytes ratio: -0.026 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 443/427 3959/4015 926/1024][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 225/283 1506/1506 330/399][PLAIN TEXT (Registro.it)][Plen Bins: 4,41,4,8,0,0,0,0,0,4,4,0,0,0,4,0,0,4,0,8,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,8,0,0]
1 TCP 192.168.1.84:52973 <-> 192.12.192.103:1194 [proto: 159/OpenVPN][IP: 0/Unknown][Encrypted][Confidence: DPI][DPI packets: 6][cat: VPN/2][25 pkts/5636 bytes <-> 21 pkts/5937 bytes][Goodput ratio: 70/77][8.96 sec][bytes ratio: -0.026 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 443/427 3959/4015 926/1024][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 225/283 1506/1506 330/399][PLAIN TEXT (Registro.it)][Plen Bins: 4,41,4,8,0,0,0,0,0,4,4,0,0,0,4,0,0,4,0,8,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,8,0,0]
2 changes: 1 addition & 1 deletion tests/result/443-safari.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ JA3 Host Stats:
1 192.168.1.13 1


1 TCP 192.168.1.13:53031 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Network/14][21 pkts/2195 bytes <-> 20 pkts/17734 bytes][Goodput ratio: 36/93][1.10 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;h2-16;h2-15;h2-14;spdy/3.1;spdy/3;http/1.1][(Negotiated) ALPN: h2][bytes ratio: -0.780 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 51/47 695/695 167/168][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 105/887 394/1506 83/661][TLSv1.2][JA3C: a69708a64f853c3bcc214c2c5faf84f3][ServerNames: www.ntop.org][JA3S: f9fcb52580329fb6a9b61d7542087b90][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Safari][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256][Plen Bins: 8,21,4,4,0,0,0,4,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,4,0,0,0,0,40,0,0]
1 TCP 192.168.1.13:53031 <-> 178.62.197.130:443 [proto: 91.26/TLS.ntop][IP: 0/Unknown][Encrypted][Confidence: DPI][DPI packets: 7][cat: Network/14][21 pkts/2195 bytes <-> 20 pkts/17734 bytes][Goodput ratio: 36/93][1.10 sec][Hostname/SNI: www.ntop.org][(Advertised) ALPNs: h2;h2-16;h2-15;h2-14;spdy/3.1;spdy/3;http/1.1][(Negotiated) ALPN: h2][bytes ratio: -0.780 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 51/47 695/695 167/168][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 105/887 394/1506 83/661][TLSv1.2][JA3C: a69708a64f853c3bcc214c2c5faf84f3][ServerNames: www.ntop.org][JA3S: f9fcb52580329fb6a9b61d7542087b90][Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3][Subject: CN=www.ntop.org][Certificate SHA-1: DB:A7:E4:3E:6D:BB:21:AB:68:47:35:E8:0B:8F:15:DF:DB:C7:C9:6F][Safari][Validity: 2019-12-17 01:17:28 - 2020-03-16 01:17:28][Cipher: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256][Plen Bins: 8,21,4,4,0,0,0,4,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,4,0,0,0,0,40,0,0]
2 changes: 1 addition & 1 deletion tests/result/4in4tunnel.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ Unknown 5 850 1


Undetected flows:
1 UDP 10.10.100.25:62367 -> 10.10.101.2:17000 [VLAN: 505][proto: 0/Unknown][IP: 0/Unknown][ClearText][Confidence: Unknown][5 pkts/850 bytes -> 0 pkts/0 bytes][Goodput ratio: 59/0][121572.07 sec][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No server to client traffic][Plen Bins: 0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
1 UDP 10.10.100.25:62367 -> 10.10.101.2:17000 [VLAN: 505][proto: 0/Unknown][IP: 0/Unknown][ClearText][Confidence: Unknown][DPI packets: 5][5 pkts/850 bytes -> 0 pkts/0 bytes][Goodput ratio: 59/0][121572.07 sec][Risk: ** Unidirectional Traffic **][Risk Score: 10][Risk Info: No server to client traffic][Plen Bins: 0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
2 changes: 1 addition & 1 deletion tests/result/4in6tunnel.pcap.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ JA3 Host Stats:
1 192.168.0.1 1


1 TCP 192.168.0.1:64455 <-> 10.10.10.1:443 [proto: 91.212/TLS.Microsoft][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Web/5][2 pkts/520 bytes <-> 2 pkts/1668 bytes][Goodput ratio: 43/82][< 1 sec][Hostname/SNI: www.bing.com][(Advertised) ALPNs: h2;http/1.1][TLSv1.2][JA3C: 9e10692f1b7f78228b2d4e424db3a98c][Firefox][Plen Bins: 0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0]
1 TCP 192.168.0.1:64455 <-> 10.10.10.1:443 [proto: 91.212/TLS.Microsoft][IP: 0/Unknown][Encrypted][Confidence: DPI][DPI packets: 4][cat: Web/5][2 pkts/520 bytes <-> 2 pkts/1668 bytes][Goodput ratio: 43/82][< 1 sec][Hostname/SNI: www.bing.com][(Advertised) ALPNs: h2;http/1.1][TLSv1.2][JA3C: 9e10692f1b7f78228b2d4e424db3a98c][Firefox][Plen Bins: 0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0]
Loading