4.14 Stable

nDPI 4.14 (Apr 2025)

Major Changes

• Introduce QoE (Quality of Experience) classification

New Supported Protocols and Services

• Add DigitalOcean protocol
• Add GearUP Booster application protocol/dissector (heuristic based) (#2764 #2765)
• Add LagoFast protocol dissector. (#2743)
• Add RUTUBE (#2725)
• Add Vivox support (#2668)
• Add new protocol ID to handle Mozilla/Firefox generic traffic (#2740)
• Add health category
• Unify "Skype" and "Teams" IDs (#2687)

Information about all protocols are available at https://github.com/ntop/nDPI/blob/dev/doc/protocols.rst

New features

• Add ndpi_find_protocol_qoe() API call
• Add ndpi_network_ptree6_match() API call
• Add ndpi_data_jitter() API call

New configuration knobs

• Add configuration parameter to enable/disable export of flow risk info (#2761)
• Add a specific configuration for classification only (#2689)
• Add the ability to enable/disable every specific flow risks (#2653)
• Extend configuration to enable/disable export of flow risk info (#2780)
• bittorrent: add configuration for "hash" metadata (#2706)
• HTTP: add configuration for some metadata (#2704)
• SSDP: add configuration for disabling metadata extraction (#2736)

Further information available aathttps://github.com/ntop/nDPI/blob/dev/doc/configuration_parameters.md

Improvements

• armagetron: reworked dissector (#2777)
• blizzard: add detection of Overwatch2, improve detection of generic battle.net traffic
• Rework the old Starcraft code to identify traffic from generic Blizzard games (#2776)
• DNS: code rework
  • Rework adding entries to the FPC-DNS cache (#2730)
  • Improve detection and handling of TCP packets (#2728)
  • Set NDPI_MALFORMED_PACKET risk if the answer message is invalid (#2724)
  • Rework/isolate code to process domain name (#2721)
  • Faster exclusion (#2719)
  • Disable subclassification by default (#2715)
  • Evaluate all flow risks even if sub-classification is disabled (#2714)
  • Export transactionId
• FPC: save all addresses from DNS to fpc_dns cache (#2792)
• HTTP: extract host and referer metadata
• RTP: improve dissection with EVS and other mobile voice codecs
  • Add ndpi_rtp_payload_type2str() API call
  • Export RTP payload in packet metadata
  • Improve detection of multimedia type for Signal calls (#2697)
• Path of Exile 2 support (#2654)
• QUIC: extract "max idle timeout" parameter (#2649)
• SMBv1: improve heuristic to avoid triggering risks for SMBv1 broadcast messages when used to browse (old) network devices
• STUN: improve detection of Telegram calls (#2671)
• STUN/RTP: extend extracted metadata (#2798)
• TLS: avoid sub-classification for RDP flows (#2769)
• TOR: update IP lists (#2748), improve detection, improve exit node download and add IPv6 support
• UBNTAC2,Ookla: improve detection (#2793 #2744)
• WoW: update detection
• Add a new specific ID for generic Ubiquity traffic (#2796)
• Add support for UTF-8 encoding in JSON serialization
• Add ndpi_str_to_utf8() API call to convert an ISO 8859 stirng to UTF-8
• Add API calls to load TCP fingeprints
• Add initial LLM traffic recognition
• Add secondary single exponential smoothing implementation
• Add Autonomous System Organization to geoip (#2763)
• Add city as a geoip possibility (#2746)
• Add additional VK ASNs
• Add Windows fingerprints
• Add missing Dropbox domain (#2685)
• Add support for loading a list of JA4C malicious fingerprints (#2678)
• Add ICMP risk checks for valid packet payloads
• Auto-generate Microsoft-related list of domains (#2688)
• Enhanced Cybersecurity protocol
• Extend list of domains for SNI matching (#2791)
• Flow risk infos are always exported "in order" (by flow risk id)
• Implement detection of the latest Signal video/audio calls leveraging on Cloudflare CDN
• Improve Google PlayStore detection
• Improve DICOM detection
• Improve WebSocket-over-HTTP detection (#2664)
• Implement SSDP Metadata export (#2729)
• Rework MapleStory support to identify traffic from generic Nexon games (#2773)
• Update SNI for YandexMetrica and YandexAlice (#2711)

Bug Fixes

• Address cache: fix a use-of-uninitialized-value error on cache restore
• Address cache: fix some bugs on cache traversal
• DNS: fix message parsing (#2732)
• DNS: fix parsing of hostname for empty response messages (#2731)
• DNS: fix dissection (#2726)
• DNS: fix check for DGA domain (#2716)
• DNS: fix writing to flow->protos.dns
• DNS: fix dissection when there is only the response message
• DNS: fix relationship between FPC and subclassification (#2702 #2709)
• DNS: fix extraction of transactionID field (#2703)
• Flute: fix heap-buffer-overflow
• HTTP: fix entropy calculation (#2666)
• SSH: fix how the flow risk is set (#2652)
• TLS: fix NDPI_TLS_WEAK_CIPHER flow risk (#2647)
• Wireguard: fix configuration of sub-classification
• Fix JA4 SSL 2 version and remove fictional SSL 1 version along with mis-mapping to s3 (#2684)
• Fix a stack-buffer-overflow error (#2782)
• Fix function checking if a packet is multicast
• Fix CSV serialization
• Fix bad IPv6 format (#1890 #2651)
• Fix bug in domain name computation
• Fix code scanning alert no. 13: Multiplication result converted to larger type (#2675)
• Fix code scanning alert no. 12: Multiplication result converted to larger type (#2676)
• Fix code scanning alert no. 7: Multiplication result converted to larger type (#2677)
• Fix code scanning alert no. 14: Redundant null check due to previous dereference (#2674)
• Fix CodeQL GitHub action (#2665)
• Fix classification "by-port" (#2655)
• Fix compilation on latest mac versions with external libraries (#2669)

Misc

• TLS: avoid exporting TLS heuristic fingerprint as metadata (#2783)
• Add extra check to trap application that mix on the same flow different protocols (#2762)
• Add 2 new fuzzers for KD-trees and Ball-trees (#2670)
• Extend fuzz coverage (#2786)
• Move rtp info out of flow->protos (#2739)
• Update all IP/domain lists (#2795)
• ndpiReader: print more DNS information (#2717)
• ndpiReader: add some global statistics about FPC (#2680)
• Remove extraction of QUIC user-agent (#2650)
• Remove Cobalt strike
• Remove JA3C (#2679)
• Remove TLS ESNI support (#2648)
• Remove NDPI_FULLY_ENCRYPTED flow risk (#2779)
• Remove NDPI_TLS_SUSPICIOUS_ESNI_USAGE flow risk (#2778)
• Rename ndpi_search_tls_udp to ndpi_search_dtls
• Rename ips_match to ndpi_ips_match
• Added 14 new categories

Full Changelog: 4.12...4.14

4.12 Stable

Major Changes

• Added detection of encrypted/obfuscated OpenVPN flows (#2547, #2560)
• Added detection of encrypted/obfuscated/proxied TLS flows (#2553)
• Implemented nDPI TCP fingerprint (6b6dad4)

For further details on these three topics, see https://www.ntop.org/ntop/a-deep-dive-into-traffic-fingerprints/

New Supported Protocols and Services

This is the list of the new supported protocols, grouped by category.
Information about these new protocols is available on https://github.com/ntop/nDPI/blob/dev/doc/protocols.rst

• NDPI_PROTOCOL_CATEGORY_IOT_SCADA:
  • NDPI_PROTOCOL_CNP_IP (#2521, #2531)
  • NDPI_PROTOCOL_ATG (#2527)
  • NDPI_PROTOCOL_TRDP (#2528)
  • NDPI_PROTOCOL_DICOM (4fd1227)
  • NDPI_PROTOCOL_CATEGORY_DATA_TRANSFER:
  • NDPI_PROTOCOL_LUSTRE (#2544)
• NDPI_PROTOCOL_CATEGORY_VPN:
• NDPI_PROTOCOL_NORDVPN (f350379)
• NDPI_PROTOCOL_SURFSHARK (5b0374c)
• NDPI_PROTOCOL_CACTUSVPN (c99646e)
• NDPI_PROTOCOL_WINDSCRIBE (2964c23)
• NDPI_PROTOCOL_CATEGORY_MUSIC:
• NDPI_PROTOCOL_SONOS (806f473)
• NDPI_PROTOCOL_CATEGORY_CHAT:
• NDPI_PROTOCOL_DINGTALK (#2581)
• NDPI_PROTOCOL_PALTALK (#2606)
• NDPI_PROTOCOL_CATEGORY_WEB:
  • NDPI_PROTOCOL_NAVER (#2610)
• NDPI_PROTOCOL_CATEGORY_SHOPPING:
  • NDPI_PROTOCOL_SHEIN (#2615)
  • NDPI_PROTOCOL_TEMU (#2615)
  • NDPI_PROTOCOL_TAOBAO (#2615)
• NDPI_PROTOCOL_CATEGORY_NETWORK:
  • NDPI_PROTOCOL_MIKROTIK (#2618)
• NDPI_PROTOCOL_CATEGORY_STREAMING:
  • NDPI_PROTOCOL_PARAMOUNTPLUS (#2628)
• NDPI_PROTOCOL_CATEGORY_VIRTUAL_ASSISTANT:
  • NDPI_PROTOCOL_YANDEX_ALICE (#2633)

New features

• Implemented JA4 raw fingerprint (42ded07)
• Add monitoring capability (for STUN flows) (#2588). See also: https://github.com/ntop/nDPI/blob/dev/doc/monitoring.md
• Implemented DNS host cache (55fa924)
• Add a configuration file to ndpiReader (#2629)

New algorithms

• Implemented algorithms for K-Nearest Neighbor Search (KNN) (#2554)
• Added ndpi_quick_encrypt() and ndpi_quick_decrypt() API calls (#2568)

New configuration knobs

Further information is available on https://github.com/ntop/nDPI/blob/dev/doc/configuration_parameters.md

• tls,subclassification, quic,subclassification, http,subclassification: enable/disable subclassification (#2533)
• openvpn,subclassification_by_ip, wiregurad,subclassification_by_ip: enable/disable sub-classification using server IP. Useful to detect the specific VPN application/app (85ebda4)
• openvpn,dpi.heuristics, openvpn,dpi.heuristics.num_messages: configure heuristics to detect OpenVPN flows (#2547)
• dpi.guess_ip_before_port: enable/disable guessing by IP first when guessing flow classifcation (#2562)
• tls,dpi.heuristics, tls,dpi.heuristics.max_packets_extra_dissection: configure heuristics to detect TLS flows (#2553)
• flow.use_client_ip_in_guess, flow.use_client_port_in_guess: configure guessing algorithm (#2569)
• $PROTO_NAME,monitoring: enable/disable monitoring state (#2588)
• metadata.tcp_fingerprint, tls,metadata.ja4r_fingerprint: enable/disable some fingerprints (6b6dad4, 42ded07)
• sip,metadata.attribute.XXX: enable/disable extraction of some SIP metadata (#2614)

Improvements

• Fixed probing attempt risk that was creating false positives (fc4fb4d)
• Fixes Viber false positive detection (5610145)
• ahocorasick: fix mem leaked AC_NODE_T object (#2258, #2522)
• Endian-independent implementation of IEEE 802.3 CRC32 (#2529)
• Improved Yahoo matching for Japanese traffic (#2539)
• HTTP, QUIC, TLS: allow to disable sub-classification (#2533)
• Bittorrent fixes (#2538)
• bins: fix ndpi_set_bin, ndpi_inc_bin and ndpi_get_bin_value (#2536)
• TLS: better state about handshake (#2534)
• OpenVPN: improve detection (c713c89)
• OpenVPN, Wireguard: improve sub-classification (85ebda4)
• oracle: fix dissector (#2548)
• RTMP: improve detection (#2549)
• RTP: fix identification over TCP (def86ba)
• QUIC: add a basic heuristic to detect mid-flows (#2550)
• Enhanced DHCP fingerprint (b77d3e3)
• dns: add a check before setting NDPI_MALFORMED_PACKET risk (#2558)
• Tls out of order (#2561)
• Added DHCP class identifier (7cc2432)
• Improved fingerprint serialization (40fefd5)
• Fixed handling of spurious TCP retransmissions (eeb1c28)
• TLS: improve handling of Change Cipher message (#2564)
• Added pki.goog domain name (26b1899)
• TTL Cache Fix (#2582)
• Added STUN fingerprint code (ab3e073)
• TLS: heuristics: fix memory allocations (#2577)
• TLS: detect abnormal padding usage (#2579)
• Enhanced DHCP fingerprint (4df60a8)
• STUN: fix monitoring of Whatsapp and Zoom flows (#2590
• Exports DNS A/AAAA responses (up to 4 addresses) (45323e3)
• Added new API calls for serializing/restoring the DNS cache (b9348e9)
• Fixed JA4 invalid computation due to code bug and uninitialized values (2b40611)
• Add configuration of TCP fingerprint computation (#2598)
• STUN: if the same metadata is found multiple times, keep the first value (#2591)
• STUN: minor fix for RTCP traffic (#2593)
• Added support for RDP over TLS (6dc4533)
• STUN: fix monitoring with RTCP flows (#2603)
• Fixes TCP fingerprint calculation when multiple EOL are specified (d5236c0)
• Added DHCP fingerprint (fecc378)
• DNS reponse addresses are now serialized in JSON (0d4c1e9)
• TikTok cleanup (a97a130)
• Added HTTP credentials extraction (412ca87)
• TLS: export heuristic fingerprint as metadata (#2609)
• SIP: rework detection and extract metadata (#2614)
• Zoom: fix heap-buffer-overflow (#2621)
• Small updates on domains list (#2623)
• RTP, STUN: improve detection of multimedia flow type (#2620)
• Update flow->flow_multimedia_types to a bitmask (#2625)
• Improved TCP probing attempt (https://github.com/...

4.10 Stable

Major Changes

• Initial work towards First Packet Classification (FPC)

New Supported Protocols and Services

• Add OpenWire support (#2513)
• FPC: add DNS correlation (#2497)
• ipaddr2list.py, ndpi2timeline.py: reformatted (#2509)
• Add Nano (XNO) protocol support (#2508)
• Added ClickHouse protocol
• Add HLS support (#2502)
• Add infrastructure for explicit support of Fist Packet Classification (#2488)
• Add detection of Twitter bot (#2487)
• Added default port mappings to ndpiReader help -H (#2477)
• Add Ripe Atlas probe protocol. (#2473)
• Add ZUG consensus protocol dissector. (#2458)
• Added NDPI_PROBING_ATTEMPT risk
• DTLS: add support for DTLS 1.3 (#2445)
• Added dpi.compute_entropy configuration parameter
• Add Call of Duty Mobile support (#2438)
• Add Ethernet Global Data support (#2437)
• Viber: add detection of voip calls and avoid false positives (#2434)
• Add support for Mastodon, Bluesky and (FB-)Threads (#2418)
• Fixes JA4 computation adding a better GREASE detect funzion
• DTLS: add support for Alert message type (similar to TLS) (#2406)
• Add Adobe Connect support (#2407)
• Remove PPStream protocol and add iQIYI (#2403)
• Add BFCP protocol support (#2401)
• Add strlcpy implementation (#2395)
• Add KNXnet/IP protocol support (#2397)
• STUN: add support for ipv6 in some metadata (#2389)
• Implemented STUN peer_address, relayed_address, response_origin, other_address parsing Added code to ignore invalid STUN realm Extended JSON output with STUN information
• Add Label Distribution Protocol support (#2385)
• Add The Elder Scrolls Online support (#2376)
• Add Shellscript risk detection. (#2375)
• Add PE32/PE32+ risk detection (detect transmitted windows executables). (#2312)
• Added support for STUN Mapped IP address
• Added binary data transfer risk alert
• Add LoL: Wild Rift detection (#2356)
• STUN: add dissection of XOR-PEER-ADDRESS with ipv6 address
• Add FLUTE protocol dissector (#2351)
• Add PFCP protocol dissector (#2342)
• Add Path of Exile protocol dissector (#2337)
• Add NetEase Games detection support (#2335)
• Add Naraka Bladepoint detection support (#2334)
• Add BFD protocol dissector (#2332)
• Add DLEP protocol dissector (#2326)
• Add ANSI C12.22 protocol dissector (#2317)
• TLS: add configuration of JA* fingerprints (#2313)
• Add detection of Gaijin Entertainment games (#2311)
• Add new AppsFlyer domain (#2307)
• Add TencentGames protocol dissector (#2306)
• Add Gearman protocol dissector (#2297)
• Add Raft protocol dissector. (#2286)
• Add Radmin protocol dissector (#2283)
• Add STOMP protocol dissector (#2280)
• Add ElectronicArts detection support (#2274)
• Add Yojimbo (netcode) protocol dissector (#2277)
• Add a dedicated dissector for Zoom (#2265)
• Add Mumble detection support (#2269)
• Add KCP protocol dissector. (#2257)
• Add PIA (Private Internet Access) support (#2250)
• Add more adult content hostnames (#2247)
• Add Roughtime protocol dissector. (#2248)
• Add realtime protocol output to ndpiReader. (#2197)
• Add Google Chat support (#2244)
• ndpiReader: add breed stats on output used for CI (#2236)
• Add Ceph protocol dissector (#2242)
• Add HL7 protocol dissector (#2240)
• Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
• Add NoMachine NX protocol dissector (#2234)
• Add Apache Kafka protocol dissector (#2226)
• Add WebDAV detection support (#2224)
• Add JSON-RPC protocol dissector (#2217)
• Add OpenFlow protocol dissector (#2222)
• Add UFTP protocol dissector (#2215)
• Add HiSLIP protocol dissector (#2214)
• Add PROFINET/IO protocol dissector (#2213)
• Add Monero protocol classification. (#2196)
• Add Ether-S-Bus protocol dissector (#2200)
• Add IEEE C37.118 protocol dissector (#2193)
• Add ISO 9506-1 MMS protocol dissector (#2189)
• Add Beckhoff ADS protocol dissector (#2181)
• Add Schneider Electric’s UMAS detection support (#2180)
• Add Ether-S-I/O protocol dissector (#2174)
• Add Omron FINS protocol dissector (#2172)
• Rework S7Comm dissector; add S7Comm Plus support (#2165)
• Add OPC UA protocol dissector (#2169)
• Add RTPS protocol dissector (#2168)
• Add HART-IP protocol dissector (#2163)
• Add IEEE 1588-2008 (PTPv2) dissector (#2156)
• Added TeslaServices and improved TikTok host names. Fixes #2140. (#2144)
• Add ethereum protocol dissector. (#2111)
• Added generic Google Protobuf dissector. (#2109)
• Add CAN over Ethernet dissector.

Improvements

• Enhanced PrimeVideo detection
• Enhanced ookla tracing
• Improved ICMP malformed packet risk description
• Improve detection of Cloudflare WARP traffic (#2491)
• tunnelbear: improve detection over wireguard (#2485)
• Improve detection of Twitter/X (#2482)
• Zoom: fix detection of screen sharing (#2476)
• Improved detection of Android connectiity checks
• Zoom: fix integer overflow (#2469)
• RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)
• Zoom: faster detection of P2P flows (#2467)
• Added NDPI_PROTOCOL_NTOP assert and removed percentage comparison (#2460)
• Add extra entropy checks and more precise(?) analysis. (#2383)
• STUN: improve extraction of Mapped-Address metadata (#2370)
• Added support for roaring bitmap v3 (#2355)
• Add more TencentGames signatures (#2354)
• Added DGA exception for Dropbox
• QUIC: add heuristic to detect unidirectional GQUIC flows (#2207)
• fuzzing: improve coverage (#2495)
• Improve detection of Cloudflare WARP traffic (#2491)
• fuzz: improve fuzzers using pl7m (#2486)
• wireshark: lua: minor improvements
• Improved logic for checking invalid DNS queries
• fuzz: improve fuzzing coverage (#2474)
• Improved Kafka dissector. (#2456)
• H323: improve detection and avoid false positives (#2432)
• Fix/improve fuzzing (#2426) (#2400)
• eDonkey: improve/update classification (#2410)
• Domain Classification Improvements (#2396)
• STUN: improve extraction of Mapped-Address metadata (#2370)
• Improve LoL: Wild Rift detection (#2359)
• Improve TencentGames detection (#2353)
• STUN: improve heurstic to detect old classic-stun
• ahocorasick: improve matching with subdomains (#2331)
• Improved alert on suspicious DNS traffic
• Telegram: improve identification
• Improved Telegram detection
• Improved modbus dissection to discard false positives
• Improved Polish gambling sites fetch script. (#2315)
• fuzz: improve fuzzing coverage (#2309)
• Improve normalization of flow->host_server_name (#2310)
• Improve ndpi_set_config error printing. (#2300)
• Improve MySQL detection (#2279)
• Improve handling of custom rules (#2276)
• Zoom: improve detection (#2270)
• Improved ndpi_get_host_domain
• Bittorrent: improve detection of UTPv1 (#2259)
• Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
• fuzz: improve fuzzing coverage (#2239)
• fuzz: improve fuzzing coverage (#2220)
• Improved belgium gambling sites regex. (#2184)
• Improve CORBA detection (#2167)
• STUN: improve demultiplexing of DTLS packets (#2153)
• Improved TFTP. Fixes #2075. (#2149)
• fuzz: improve coverage and remove dead code (#2135)
• Improved Protobuf dissector. (#2119)
• Improved detection as non DGA for hostnames belnging to a CDN (#2068)
• Improved CryNetwork protocol dissector.

Tools

• Make the CI faster (#2475)
• Add a script to download/update the domain suffix list (#2321)
• Add identification of Huawei generic and cloud traffic (#2325)
• ndpiReader: improve the check on max number of pkts processed per flow (#2261)
• Added default port mappings to ndpiReader help -H (#2477)
• ndpiReader: restore ndpiReader -x $DOMAIN_NAME functionality (#2329)
• ndpiReader: improve the check on max number of pkts processed per flow (#2261)
• ndpiReader: fix memory leak
• Add realtime protocol output to ndpiReader. (#2197)
• ndpiReader: add breed stats on output used for CI (#2236)
• ndpiReader: avoid creating two detection modules when processing traffic/traces (#2209)
• ndpiReader: fix guessed_flow_protocols statistic (#2203)

Misc

• Improved tests coverage
• Varisous performance improvements
• Added stress test
• Added new API calls - ndpi_load_domain_suffixes() - ndpi_get_host_domain_suffix()
• Add some fast CRC16 algorithms implementation (#2195)
• Add a FAQ for the project (#2185)
• Ip address list: aggregate Mullvad and Tor lists too (#2154)
• IP lists: aggregate addresses wherever possible (#2152)
• Added malicious sites from the polish cert. (#2121)
• IPv6: add support for custom categories (#2126)
• IPv6: add support for IPv6 risk exceptions (#2122)
• IPv6: add support for custom rules (#2120)
• IPv6: add support for IPv6 risk tree (#2118)
• ipv6: add support for ipv6 addresses lists (#2113)

4.8 Stable

Major Changes

• Reworked lists implementation that decreased memory usage of orders of magnitude
• Improved code robustness via extensive code fuzzing
• Various improvements to overall library performance
• Extended IPv6 support

New Supported Protocols and Services

• Add "Heroes of the Storm" video game signature detection. (#1949)
• Add Apache Thrift protocol dissector. (#2007)
• Add Remote Management Control Protocol (RMCP).
• Add Service Location Protocol dissector. (#2036)
• Add VK detection (#1880)
• Add Yandex services detection (#1882)
• Add a new protocol id for generic Adult Content traffic (#1906)
• Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)
• Add bitcoing protocol dissector. (#1992)
• Add detection of Roblox games (#2054)
• Add support for (un-encrypted) HTTP/2 (#2087)
• Add support for Epic Games and GeForceNow/Nvidia (#1990)
• Add support for SRTP (#1977)
• Added BACnet dissector. (#1940)
• Added HAProxy protocol. (#2088)
• Added OICQ dissector. (#1950)
• Added OperaVPN detection
• ProtonVPN: add basic detection (#2006)
• Added detection of Facebook Reels and Stories
• Add an heuristic to detect fully encrypted flows (#2058)
• Added NDPI_MALWARE_HOST_CONTACTED flow risk
• Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk

Improvements

• Improve protocol detection for:
• FreeBSD compilation fix (C) update
• Gnutella: improve detection (#2019)
• H323: fix false positives (#1916)
• HTTP: fix another memory access error (#2049)
• HTTP: fix extraction of filename (#2046)
• HTTP: fix heap-buffer-overflow (#2044)
• HTTP: improve extraction of metadata and of flow risks (#1959)
• HTTP: remove useless code about XBOX (#1958)
• HTTP: rework state machine (#1966)
• Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
• Enhance DNS risk for long hostnames (> 32)
• Enhanced MS teams STUN/Azure detection
• Enhanced custom port definition and improved error reporting in case of duplications
• Improve detection of Alibaba flows (#1991)
• Improve detection of crawler/bot traffic (#1956)
• Improve detection of crawlers/bots (#1968)
• Improved MGCP detection by allowing '\r' as line feed.
• Improved MS Teams detection with heuristic
• Improved Steam detection by adding steamdiscover pattern. (#2105)
• Improved Wireguard detection
• Improved checks for duplicated entries in protocols file
• Improved classification further reducing memory used
• Improved detection of invalid chars in DNS names
• Improved domain search tet unit
• Improved helper scripts. (#1986)
• MS Teams enhancement
• MySql: improve detection (#1928)
• zabbix: improve detection (#2055)

Tools

• ndpiReader: allow to configure LRU caches TTL and size (#2004)
• ndpiReader: fix VXLAN de-tunneling (#1913)
• ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
• ndpiReader: fix export of HTTP attributes (#1982)
• ndpiReader: fix flow stats (#1943)
• ndpiReader: fix print of flow payload (#1960)
• ndpiReader: improve printing of payload statistics (#1989)
• ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
• ndpireader: fix detection of DoH traffic based on packet distributions (#2045)

Misc

• ARM compilation fix
• Add ndpi_domain_classify_finalize() function (#2084)
• Add a configuration knob to enable/disable loading of gambling list (#2047)
• Add a new flow risk about literal IP addresses used as SNI (#1892)
• Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
• Add another example of custom rules (#1923)
• Add support for multiline json
• Add support for roaring_bitmap_xor_inplace (#1983)
• Add support for vxlan decapsulation (#1441) (#1900)
• Added Source Engine dissector. (#1937)
• Added lists/gambling.list to extra dist.
• Added slackb.com SNI. (#2067)
• Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2)
• Added check to avoid skype heuristic false positives
• Added comment
• Added coverage targets to Makefile.am for convenience. (#2039)
• Added fix for better handling exceptions rollback in case of later match
• Added hyperlink
• Added ndpi_binary_bitmap data structure
• Added ndpi_bitmap64 support
• Added ndpi_bitmap_andnot API call
• Added ndpi_bitmap_copy() API call
• Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
• Added ndpi_domain_classify_XXX(0 API
• Added ndpi_filter_add_multi() API call
• Added ndpi_murmur_hash to the nDPI API
• Added new API calls for implementing Bloom-filter like data structures
• Added printf/fprintf replacement for some internal modules. (#1974)
• Added scripts to auto generate hostname/SNI *.inc files. (#1984)
• Added sub-domain classification fix
• Added the ability to define custom protocols with arbitrary Ids in proto.txt
• Added vlan_id in ndpi_flow2json() prototype
• Adds new pcap for testing "funny" HTTP servers
• All protocols should be excluded sooner or later (#1969)
• Allow init of app protocols w/o any hostnames set. (#2057)
• Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
• Boundary check
• CI: fix Performance job (#1936)
• Centos7 fixes
• Changed logging callback function sig. (#2000)
• Changes for supporting more efficient sub-string matching
• Classification fixes
• DNS: extract geolocation information, if available (#2065)
• Debian 12 fixes
• Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted
• DisneyPlus/Hulu ip lists should be auto-generated (#1905)
• Extend content list of Microsoft protocols (#1930)
• Extend content-match list (#1967)
• Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934)
• Fix MS Teams detection with heuristic (#1972)
• Fix access to packet/flow information (#2013)
• Fix an heap-buffer-overflow (#1994)
• Fix classification-by-ip in ndpi_detection_giveup (#1981)
• Fix compilation (#2011)
• Fix compilation in CI jobs (#2048)
• Fix compilation on Windows (#2072)
• Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
• Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)
• Fix export/serialization of flow->risk (#1885)
• Fix for buffer overflow in serialization
• Fix insert of ip addresses into patricia tree(s) (#1895)
• Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
• Fix packet counters (#1884)
• Fix some errors found by fuzzers (#2078)
• Fix some memory errors triggered by allocation failures (#1995)
• Fix some prototypes (#2085)
• Fix string truncation. (#2056)
• Fixed OpenWRT arm related build issues. (#2104)
• Fixed heap-buffer-overflow issue
• Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
• Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
• Fixed missing AS_HELP_STRING in configure.ac. (#1893)
• Fixed two OpenWRT arm related build issues. (#2103)
• Fixes matches with domain name strings that start with a dot
• Fixes risk mask exception handling while improving the overall performance
• Implemented Count-Min Sketch [count how many times a value has been observed]
• Implemented Zoom/Teams stream type detection
• Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw
• Implemented ndpi_predict_linear() for predicting a timeseries value overtime
• Improved debug output. (#1951)
• Improved invalid logging via printf().
• Improved line protocol dissection with heuristic
• Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
• Improved protocol detection exploiting IP-based guess Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
• Improvement for reducing false positives
• Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
• Keep master protocol in ndpi_reconcile_protocols
• Leak fix
• Language fix
• Line: fix heap-buffer-overflow error (#2015)
• Made VK protocol detection more strict
• Make Bittorrent LRU cache IPv6 aware. (#1909)
• Merged new and old version of ndpi_domain_classify.c code
• Mullvad VPN service added (based on entry node IP addresses) (#2062)
• Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
• Numeric truncation at tls.c:1010 (#2005)
• Ookla: rework detection (#1922)
• Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4()
• ProtonVPN: split the ip list (#2060)
• QUIC: add support for QUIC version 2
• QUIC: export QUIC version as metadata
• QUIC: fix a memory access error
• QUIC: fix dissection of packets forcing VN
• RDP: improve detection over UDP (#2043)
• RTP: remove dead-code (#1953)
• RTP: rework code (#2021)
• Refreshed ASN lists Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
• Remove some useless checks (#1993)
• Remove special handling of some TCP flows without SYN (#1965)
• Removed overlapping port
• Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI
• Replaces free() with ndpi_free()
• Rework CI jobs to try reducing CI duration (#1903)
• Reworked domain classification based on binary filters
• Reworked initialization
• Reworked ndpi_filter_xxx implementation using compressed bitmaps
• Reworked teams handling
• RiotGames: add detection of flows (#1935)
• STUN: add dissection of DTLS handshake (#2018)
• STUN: avoid FacebookVoip false positives (#2029)
• STUN: fix Skype/MsTeams detection and monitoring logic (#2028)
• STUN: fix detection of Google Voip apps (#2031)
• STUN: fix detection over TCP
• STUN: i...

4.6 Stable

nDPI 4.6 (Feb 2023)

New Features

• New support for custom BPF protocol definition using nBPF (see example/protos.txt)
• Improved dissection performace
• Added fuzzing all over

New Supported Protocols and Services

• Add protocol detection for:
  • Activision
  • AliCloud server access
  • AVAST
  • CryNetwork
  • Discord
  • EDNS
  • Elasticsearch
  • FastCGI
  • Kismet
  • Line App and Line Voip valls
  • Meraki Cloud
  • Munin
  • NATPMP
  • Syncthing
  • TP-LINK Smart Home
  • TUYA LAN
  • SoftEther VPN
  • Tailscale
  • TiVoConnect

Improvements

• Improve protocol detection for:
  • Anydesk
  • Bittorrent (fix confidence, detection over TCP)
  • DNS, add ability to decode DNS PTR records used for reverse address resolution
  • DTLS (handle certificate fragments)
  • Facebook Voip calls
  • FastCGI (dissect PARAMS)
  • FortiClient (update default ports)
  • Zoom
    • Add Zoom screen share detection
    • Add detection of Zoom peer-to-peer flows in STUN
  • Hangout/Duo Voip calls detection, optimize lookups in the protocol tree
  • HTTP
    • Handling of HTTP-Proxy and HTTP-Connect
    • HTTP subclassification
    • Check for empty/missing user-agent in HTTP
  • IRC (credentials check)
  • Jabber/XMPP
  • Kerberos (support for Krb-Error messages)
  • LDAP
  • MGCP
  • MONGODB (avoid false positives)
  • Postgres
  • POP3
  • QUIC (support for 0-RTT packets received before the initial)
  • Snapchat Voip calls
  • SIP
  • SNMP
  • SMB (support for messages split into multiple TCP segments)
  • SMTP (support for X-ANONYMOUSTLS command)
  • STUN
  • SKYPE (improve detection over UDP, remove detection over TCP)
  • Teamspeak3 (License/Weblist detection)
  • Threema Messenger
  • TINC (avoid processing SYN packets)
  • TLS
    • improve reassembler
    • handling of ALPN(s) and subclassification
    • ignore invalid Content Type values
  • WindowsUpdate
• Add flow risk:
  • NDPI_HTTP_OBSOLETE_SERVER
  • NDPI_MINOR_ISSUES (generic/relevant information about issues found on traffic)
  • NDPI_HTTP_OBSOLETE_SERVER (Apache and nginx are supported)
  • NDPI_PERIODIC_FLOW (reserved bit to be used by apps based on nDPI)
  • NDPI_TCP_ISSUES
• Improve detection of WebShell and PHP code in HTTP URLs that is reported via flow risk
• Improve DGA detection
• Improve AES-NI check
• Improve nDPI JSON serialization
• Improve export/print of L4 protocol information
• Improve connection refused detection
• Add statistics for Patricia tree, Ahocarasick automa, LRU cache
• Add a generic (optional and configurable) expiration logic in LRU caches
• Add RTP stream type in flow metadata
• LRU cache is now IPv6 aware

Tools

• ndpiReader
  • Add support for Linux Cooked Capture v2
  • Fix packet dissection (CAPWAP and TSO)
  • Fix Discarded bytes statistics

Fixes

• Fix classification by-port
• Fix exclusion of DTLS protocol
• Fix undefined-behaviour in ahocorasick callback
• Fix infinite loop when a custom rule has port 65535
• Fix undefined-behavior when setting empty user-agent
• Fix infinite loop in DNS dissector (due to an integer overflow)
• Fix JSON export of IPv6 addresses
• Fix memory corruptions in Bittorrent, HTTP, SoftEther, Florensia, QUIC, IRC, TFTP dissectors
• Fix stop of extra dissection in HTTP, Bittorrent, Kerberos
• Fix signed integer overflow in ASN1/BER dissector
• Fix char/uchar bug in ahocorasick
• Fix endianess in IP-Port lookup
• Fix FastCGI memory allocation issue
• Fix metadata extraction in NAT-PMP
• Fix invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow)

Misc

• Support for Rocky Linux 9
• Enhance fuzzers to test nDPI configurations, memory allocation failures, serialization/deserialization, algorithms and data structures
• GitHub Actions: update to Node.js 16
• Size of LRU caches is now configurable

4.4 Stable

nDPI 4.4 (July 2022)

New Features

• Add risk information that describes why a specific risk was triggered also providing metadata
• Added API call ndpi_check_flow_risk_exceptions() for handling risk exceptions
• Split protocols in: network (e.g. TLS) and application protocols (e.g. Google)
• Extended confidence level with two new values (NDPI_CONFIDENCE_DPI_PARTIAL and NDPI_CONFIDENCE_DPI_PARTIAL_CACHE)
• Added ndpi_get_flow_error_code() API call

New Supported Protocols and Services

• Add protocol detection for:
  • UltraSurf
  • i3D
  • RiotGames
  • TSAN
  • TunnelBear VPN
  • collectd
  • PIM (Protocol Indipendent Multicast)
  • Pragmatic General Multicast (PGM)
  • RSH
  • GoTo products (mainly GoToMeeting)
  • Dazn
  • MPEG-DASH
  • Agora Software Defined Real-time Network (SD-RTN)
  • Toca Boca
  • VXLAN
  • MDNS/LLMNR

Improvements

• Improve protocol detection for:
  • SMTP/SMTPS now supports STARTTLS
  • OCSP
  • TargusDataspeed
  • Usenet
  • DTLS (added support for old versions)
  • TFTP
  • SOAP via HTTP
  • GenshinImpact
  • IPSec/ISAKMP
  • DNS
  • syslog
  • DHCP (various bug fixes and improvements)
  • NATS
  • Viber
  • Xiaomi
  • Raknet
  • gnutella
  • Kerberos
  • QUIC (Added support for v2drft 01)
  • SSDP
  • SNMP
• Improved DGA detection
• Improved AES-NI check
• Add flow risk:
  • NDPI_PUNYCODE_IDN
  • NDPI_ERROR_CODE_DETECTED
  • NDPI_HTTP_CRAWLER_BOT
  • NDPI_ANONYMOUS_SUBSCRIBER
• NDPI_UNIDIRECTIONAL_TRAFFIC

Changes

• Added support for 64 bit bins
• Added Cloudflare WARP detection patterns
• Renamed Z39.50 -> Z3950
• Replaced nDPI's internal hashmap with uthash
• Reimplemented 1kxun application protoco
• Renamed SkypeCall to Skype_TeamsCall
• Updated Python Bindings
• Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation

Fixes

• Fixes for some protocol classification families
• Fixed default protocol ports for email protocols
• Various memory and overflow fixes
• Disabled various risks for specific protocols (e.g. disable missing ALPN for CiscoVPN)
• Fix TZSP decapsulation

Misc

• Update ASN/IPs lists
• Improved code profiling
• Use Doxygen to generate the API documentation
• Added Edgecast and Cachefly CDNs.

Raw Changelog

• Label SMTP w/ STARTTLS as SMTPS and dissect TLS clho. (#1639)
• Compilation fix
• Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636)
• SMTP with STARTTLS is now identified as SMTPS
• Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes #1630. (#1637)
• Run regression tests from different locations at the same time w/o side effects on the results. (#1638)
• Exported username in flow information
• Updated ndpi_check_flow_risk_exceptions() signature
• Cleaned-up issuer DN check code adding u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);
• Set CiscoVPN as a network protocol
• Updated JA3/SSL fingerprints.
• Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
• Added UltraSurf protocol dissector. (#1618)
• Add two new confidence values: confidence by partial DPI (#1632)
• Update host content list match (#1633)
• Sync Psiphon unit test. (#1634)
• Added Psiphon detection patterns. See #566 and #1099. (#1631)
• OCSP: improve detection (#1629)
• Added i3D and RiotGames protocol dissectors. (#1609)
• TargusDataspeed: avoid false positives (#1628)
• Update ASN/IPs lists (#1627)
• bins: add support for 64bit bins (#1626)
• Skinny: rework and improve classification (#1625)
• Skype_Teams, Mining, SnapchatCall: fix flow category (#1624)
• Minor changes in how classification results are set (#1623)
• Usenet: improve dissection (#1622)
• Fix category for mail sessions (#1621)
• TLS: add support for old DTLS versions and for detection of mid-sessions (#1619)
• Fix a compilation warning (#1620)
• Generate profiling results as PNG.
• gprof test/CI integration
• Improved TFTP. Dissect Read/Write Request filenames. (#1617)
• Added TSAN support. (#1613)
• Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. Fixes #1608. (#1614)
• Added Cloudflare WARP detection patterns. (#1615) (#1616)
• Fixed SMTP default port 587
• Added TunnelBear VPN detection patterns. (#1615)
• Updated (C)
• Removed space from "Genshin Impact"
• sync unit tests (#1612)
• Fix after the protocol name update
• Renamed Z39.50 -> Z3950 as the '.' breaks the naming convention QUIC is a network protocol
• Enhanced TLS risk info reported to users
• Added default port for syslog TCP
• Fix compilation and sync unit tests results (#1606)
• Added unidirectional traffic flow risk
• Improved SOAP via HTTP. (#1605)
• Improved GenshinImpact protocol dissector. (#1604)
• Added collectd dissector (again). (#1601)
• Replaced nDPI's internal hashmap with uthash. (#1602)
• Improved IPSec/ISAKMP detection. (#1600)
• Added new test pcaps
• Add some statistics to ndpiReader (#1587)
• Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
• Improved WhatsApp detection. (#1595)
• Fix invalid memory access (#1596)
• DNS: fix TTL check and sync unit test results (#1594)
• Updated DNS alert triggered only with TTL == 0
• Restored ndpi_set_proto_defaults() prototype Updated test results
• Added check for DGA names that resolve to a valid record
• Improved DNS traffic analysis Added ability to identify application and network protocols
• Added DNS record TTL check
• Added gprof CPU/HEAP profiling support. (#1592)
• Removed Makefile references to legacy code. (#1589)
• Added Pragmatic General Multicast (PGM) protocol detection
• Dissect host line if SSDP contains such. (#1586)
• Reimplemented 1kxun application protocol. (#1585)
• Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is empty. (#1584)
• Fixed syslog false negatives. (#1582)
• Fix some debug messages (#1583)
• Updated test results
• Fixed invalid DHCP dissection
• Fixed DHCP dissection bug
• Added RSH dissector. Fixes #202. (#1581)
• Add support for GoTo products (mainly GoToMeeting) (#1580)
• Fix syslog heap overflow introduced in 09fbe0a. (#1579)
• Fixed syslog false positives. (#1577)
• Fix heap buffer overflow mentioned in #1574. (#1576)
• TLS: fix use-of-uninitialized-value error (#1573)
• Removed README.nDPI as it does not provide any new information not covered by README.md (#1572)
• Removed LGTM ql query for packet payload integer arithmetic. (#1570)
• Force roaring bitmap to use ndpi memory wrappers. (#1569)
• TLS: fix stack-buffer-overflow error (#1567)
• Updated risk results
• Improved message for known proto on non std port
• Added check
• Updated README.md (#1562)
• TLS: fix use-of-uninitialized-value error (#1566)
• Redefined type name to avoid conflicts
• Added ability to return risk info in JSON format in ndpi_get_flow_risk_info()
• Support word diff for tests/do.sh for better readability. (#1565)
• Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
• HTTP: fix heap-buffer-overflow error (#1564)
• Certificate timestamps should be printed in UTC (#1563)
• Fixed dispay bug for risk_info
• Updated tests results Code cleanup
• Added RiskInfo string
• Fix dissection of IPv4 header (#1561)
• Dazn: add support for Dazn streaming service (#1559)
• Compilation fixes for old ggc's
• Comment
• Added detection for WordPress exploits Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
• Use Doxygen to generate the API documentation. (#1558)
• Added MPEG-DASH dissector. Fixes #1223. (#1555)
• Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
• Compilation fixes for old gcc compiler
• Compilation fixes
• Version cut fix
• Fixes compilation issues on RedHat systems
• Sync unit test results (#1554)
• Updated SkypeCall -> Skype_TeamsCall
• Fixed false positives with NATS
• Added script to compare and verify the output of `make dist'. (#1551)
• Replaced obsolete autoconf macros. (#1553)
• Fixed windows-latest build error. (#1552)
• Improved invalid host detection
• Added invalid SNI check in QUIC
• Improved detection of invalid SNI and hostnames in TLS, HTTP
• Added room for storing information used by custom third-party dissectors
• Moved RTSP http patterns to the protocol source file.
• Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).
• Removed MacOS XCode integration.
• Moved mgcp.pcapng to tests/pcap/ instead of tests/
• DNS-over-QUIC: update default port (#1548)
• Improved Viber (TCP) detection. (#1547)
• Improved Xiaomi HTTP detection. (#1546)
• Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
• Added Softether(-VPN) DDNS service detection. (#1544)
• Improved TLS alert detection. (#1542)
• Improved TLS application data detection. (#1541)
• Added Edgecast and Cachefly CDNs. (#1540)
• Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
• Fix compilation (if --enable-debug-messages is used) (#1539)
• Added extra check to make sure that the guessed protocol is the one we expect and not another one
• Fixes bug that prevents triggering alerts for traffic on non-standard ports that have been defined in the custom protocols file
• Fixes outdated description
• Modified risk labels
• Added some Pluralsight Hostnames/SNIs. May fix #1501. (#1538)
• Updated RRD dependencies
• Improved suspicious http user agent detection. (#1537)
• Added ndpi_get_flow_error_code() API call Fixed typo
• Improved AES-NI check. (#1536)
• Improved AES-NI chec...

4.2 Stable

nDPI 4.2 (Feb 2022)

New Features

• Add a "confidence" field indicating the reliability of the classification
• Add risk exceptions for services and domain names via ndpi_add_domain_risk_exceptions()
• Add ability to report whether a protocol is encrypted

New Supported Protocols and Services

• Add protocol detection for:
  • Badoo
  • Cassandra
  • EthernetIP

Improvements

• Significantly reduced memory footprint from 2.94 KB to 688 B per flow
• Improve protocol detection for:
  • BitTorrent
  • ICloud Private Relay
  • IMAP, POP3, SMTP
  • Log4J/Log4Shell
  • Microsoft Azure
  • Pandora TV
  • RTP
  • RTSP
  • Salesforce
  • STUN
  • Whatsapp
  • QUICv2
  • Zoom
• Add flow risk:
  • NDPI_CLEAR_TEXT_CREDENTIALS
  • NDPI_POSSIBLE_EXPLOIT (Log4J)
  • NDPI_TLS_FATAL_ALERT
  • NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
• Update WhatsAPP and Instagram addresses
• Update the list of default ports for QUIC
• Update WindowsUpdate URLs
• Add support for the .goog Google TLD
• Add googletagmanager.com
• Add bitmaps and API for handling compressed bitmaps
• Add JA3 in risk exceptions
• Add entropy calculation to check for suspicious (encrypted) payload
• Add extraction of hostname in SMTP
• Add RDP over UDP dissection
• Add support for TLS over IPV6 in Subject Alt Names field
• Improve JSON and CSV serialization
• Improve IPv6 support for almost all dissectors
• Improve CI and unit tests, add arm64, armhf and s390x as part of CI
• Improve WHOIS detection, reduce false positives
• Improve DGA detection for skipping potential DGAs of known/popular domain names
• Improve user agent analysis
• Reworked HTTP protocol dissection including HTTP proxy and HTTP connect

Changes

• TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1)
• Numeric IPs are not considered for DGA checks
• Differentiate between standard Amazon stuff (i.e market) and AWS
• Remove Playstation VUE protocol
• Remove pandora.tv from Pandora protocol
• Remove outdated SoulSeek dissector

Fixes

• Fix race conditions
• Fix dissectors to be big-endian friendly
• Fix heap overflow in realloc wrapper
• Fix errors in Kerberos, TLS, H323, Netbios, CSGO, Bittorrent
• Fix wrong tuple comparison
• Fix ndpi_serialize_string_int64
• Fix Grease values parsing
• Fix certificate mismatch check
• Fix null-dereference read for Zattoo with IPv6
• Fix dissectors initialization for XBox, Diameter
• Fix confidence for STUN classifications
• Fix FreeBSD support
• Fix old GQUIC versions on big-endian machines
• Fix aho-corasick on big-endian machines
• Fix DGA false positive
• Fix integer overflow for QUIC
• Fix HTTP false positives
• Fix SonarCloud-CI support
• Fix clashes setting the hostname on similar protocols (FTP, SMTP)
• Fix some invalid TLS guesses
• Fix crash on ARM (Raspberry)
• Fix DNS (including fragmented DNS) dissection
• Fix parsing of IPv6 packets with extension headers
• Fix extraction of Realm attribute in STUN
• Fix support for START-TLS sessions in FTP
• Fix TCP retransmissions for multiple dissectors
• Fix DES initialisation
• Fix Git protocol dissection
• Fix certificate mismatch for TLS flows with no client hello observed
• Fix old versions of GQUIC on big-endian machines

Misc

• Add tool for generating automatically the Azure IP list

4.0 Stable

New Features

• Add API for computing RSI (Relative Strenght Index)
• Add GeoIP support
• Add fragments management
• Add API for jitter calculation
• Add single exponential smoothing API
• Add timeseries forecasting support implementing Holt-Winters with confidence interval
• Add support for MAC to radix tree and expose the full API to applications
• Add JA3+, with ALPN and elliptic curve
• Add double exponential smoothing implementation
• Extended API for managing flow risks
• Add flow risk score
• New flow risks:
  • Desktop or File Sharing Session
  • HTTP suspicious content (useful for tracking trickbot)
  • Malicious JA3
  • Malicious SHA1
  • Risky domain
  • Risky AS
  • TLS Certificate Validity Too Long
  • TLS Suspicious Extension

New Supported Protocols and Services

• New protocols:
  • AmongUs
  • AVAST SecureDNS
  • CPHA (CheckPoint High Availability Protocol)
  • DisneyPlus
  • DTLS
  • Genshin Impact
  • HP Virtual Machine Group Management (hpvirtgrp)
  • Mongodb
  • Pinterest
  • Reddit
  • Snapchat VoIP calls
  • Tumblr
  • Virtual Asssitant (Alexa, Siri)
  • Z39.50
• Add protocols to HTTP as subprotocols
• Add detection of TLS browser type
• Add connectionless DCE/RPC detection

Improvements

• 2.5x speed bump. Example ndpiReader with a long mixed pcap
  v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec
  v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
• Improve detection/dissection of:
• AnyDesk
• DNS
• Hulu
• DCE/RPC (avoid false positives)
• dnscrypt
• Facebook (add new networks)
• Fortigate
• FTP Control
• HTTP
  • Fix user-agent parsing
  • Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
• IEC104
• IEC60870
• IRC
• Netbios
• Netflix
• Ookla speedtest (detection over IPv6)
• openspeedtest.com
• Outlook / MicrosoftMail
• QUIC
  • update to draft-33
  • improve handling of SNI
  • support for fragmented Client Hello
  • support for DNS-over-QUIC
• RTSP
• RTSP via HTTP
• SNMP (reimplemented)
• Skype
• SSH
• Steam (Steam Datagram Relay - SDR)
• STUN (avoid false positives, improved Skype detection)
• TeamViewer (add new hosts)
• TOR (update hosts)
• TLS
  • Certificate Subject matching
  • Check for common ALPNs
  • Reworked fingerprint calculation
  • Fix extraction for TLS signature algorithms
  • Fix ClientHello parsing
• UPnP
• wireguard
• Improve DGA detection
• Improve JA3
• Improve Mining detection
• Improve string matching algorithm
• Improve ndpi_pref_enable_tls_block_dissection
• Optimize speed and memory size
• Update ahocorasick library
• Improve subprotocols detection

Fixes

• Fix partial application matching
• Fix multiple segfault and leaks
• Fix uninitialized memory use
• Fix release of patterns allocated in ndpi_add_string_to_automa
• Fix return value of ndpi_match_string_subprotocol
• Fix setting of flow risks on 32 bit machines
• Fix TLS certificate threshold
• Fix a memory error in TLS JA3 code
• Fix false positives in Z39.50
• Fix off-by-one memory error for TLS-JA3
• Fix bug in ndpi_lru_find_cache
• Fix invalid xbox and playstation port guesses
• Fix CAPWAP tunnel decoding
• Fix parsing of DLT_PPP datalink type
• Fix dissection of QUIC initial packets coalesced with 0-RTT one
• Fix parsing of GTP headers
• Add bitmap boundary checks

Misc

• Update download category name
• Update category labels
• Renamed Skype in Skype_Teams (the protocol is now shared across these apps)
• Add IEC analysis wireshark plugin
• Flow risk visualization in Wireshark
• ndpiReader
  • add statistics about nDPI performance
  • fix memory leak
  • fix collecting of risks statistics
• Move installed libraries from /usr/local to /usr
• Improve NDPI_API_VERSION generation
• Update ndpi_ptree_match_addr prototype

3.4 Stable

New Features

• Completely reworked and extended QUIC dissector
• Added flow risk concept to move nDPI towards result interpretation
• Added ndpi_dpi2json() API call
• Added DGA risk for names that look like a DGA
• Added HyperLogLog cardinality estimator API calls
• Added ndpi_bin_XXX API calls to handle bin handling
• Fully fuzzy tested code that has greatly improved reliability and robustness

New Supported Protocols and Services

• QUIC
• SMBv1
• WebSocket
• TLS: added ESNI support
• SOAP
• DNScrypt

Improvements

• Python CFFI bindings
• Various TLS extensions and fixes including extendede metadata support
• Added various pcap files for testing corner cases in protocols
• Various improvements in JSON/Binary data serialization
• CiscoVPN
• H323
• MDNS
• MySQL 8
• IEC 60870-5-104
• DoH/DoT dissection improvements
• Office365 renamed to Microsoft365
• Major protocol dissection improvement in particular with unknwon traffic
• Improvement in Telegram v6 protocol support
• HTTP improvements to detect file download/upload and binary files
• BitTorrent and WhatsApp dissection improvement
• Spotify
• Added detection of malformed packets
• Fuzzy testing support has been greatly improved
• SSH code cleanup

Fixes

• Fixed various memory leaks and race conditions in protocol decoding
• NATS, CAPWAP dissector
• Removed HyperScan support that greatly simplified the code
• ARM platform fixes on memory alignment
• Wireshark extcap support
• DPDK support
• OpenWRT, OpenBSD support
• MINGW compiler support

MISC

• Created demo app for nDPI newcomers
• Removed obsolete pplive and pando protocols

3.2 Stable

New Features

• New API calls
• Protocol detection: ndpi_is_protocol_detected
• Categories: ndpi_load_categories_file / ndpi_load_category
• JSON/TLV serialization: ndpi_serialize_string_boolean / ndpi_serialize_uint32_boolean
• Patricia tree: ndpi_load_ipv4_ptree
• Module initialization: ndpi_init_detection_module / ndpi_finalize_initalization
• Base64 encoding: ndpi_base64_encode
• JSON export: ndpi_flow2json
• Print protocol: ndpi_get_l4_proto_name / ndpi_get_l4_proto_info
• Libfuzz integration
• Implemented Community ID hash (API call ndpi_flowv6_flow_hash and ndpi_flowv4_flow_hash)
• Detection of RCE in HTTP GET requests via PCRE
• Integration of the libinjection library to detect SQL injections and XSS type attacks in HTTP requests

New Supported Protocols and Services

• TLS: new decode
• Added ALPN support
• Added export of supported version in TLS header
• Added Telnet dissector with metadata extraction
• Added Zabbix dissector
• Added POP3/IMAP metadata extraction
• Added FTP user/password extraction
• Added NetBIOS metadata extraction
• Added Kerberos metadata extraction
• Implemented SQL Injection and XSS attack detection
• Host-based detection improvements and changes
• Added Microsoft range
• Added twitch.tv website
• Added brasilbandalarga.com.br and .eaqbr.com.br as EAQ
• Added 20.180.0.0/14, 20.184.0.0/13 range as Skype
• Added 52.84.0.0/14 range as Amazon
• Added pastebin.com
• Changed 13.64.0.0/11 range from Skype to Microsoft
• Refreshed Whatsapp server list, added whatsapp-.fbcdn.net IPs
• Added public DNSoverHTTPS servers

Improvements

• Reworked and improved the TLS dissector
• Reworked Kerberos dissector
• Improved DNS response decoding
• Support for DNS continuous flow dissection
• Improved Python bindings
• Improved Ethereum support
• Improved categories detection with streaming and HTTP
• Support for IP-based detection to compute the application protocol
• Renamed protocol 104 to IEC60870 (more meaningful)
• Added failed authentication support with FTP
• Renamed DNSoverHTTPS to handle bot DoH and DoT
• Implemented stacked DPI decoding
• Improvements for CapWAP and Bloomberg
• Improved SMB dissection
• Improved SSH dissection
• Added capwap support
• Modified API signatures for ndpi_ssl_version2str / ndpi_detection_giveup
• Removed ndpi_pref_http_dont_dissect_response / ndpi_pref_dns_dont_dissect_response (replaced by ndpi_extra_dissection_possible)

Fixes

• Fixed memory invalid access in SMTP and leaks in TLS
• Fixed a few memory leaks
• Fixed invalid memory access in a few protocol dissectors (HTTP, memcached, Citrix, STUN, DNS, Amazon Video, TLS, Viber)
• Fixed IPv6 address format across the various platforms/distributions
• Fixed infinite loop in ndpi_workflow_process_packet
• Fixed SHA1 certificate detection
• Fixed custom protocol detection
• Fixed SMTP dissection (including email)
• Fixed Telnet dissection and invalid password report
• Fixed invalid category matching in HTTP
• Fixed Skype and STUN false positives
• Fixed SQL Injection detection
• Fixed invalid SMBv1 detection
• Fixed SSH dissection
• Fixed ndpi_ssl_version2str
• Fixed ndpi_extra_dissection_possible
• Fixed out of bounds read in ndpi_match_custom_category

Misc

ndpiReader

• CSV output enhancements
• Added tunnelling decapsulation
• Improved HTTP reporting
• Added scan and HTTP attacks (XSS, SQL Injection) detection