NFDUMP(1)

NAME
nfdump - flow display and analysis program

SYNOPSIS
nfdump -r flowpath [-w outfile] [-f filterfile] [-C config] [-R filelist] [-M dirlist] [-O order] [-t timewin] [-c num] [-a] [-A aggregation] [-b] [-B] [-I] [-D nameserver] [-G geoDB] [-H torDB] [-s statistic] [-n num] [-o format] [-6] [-q] [-N] [-i ident] [-v flowfile] [-E flowfile] [-x flowfile] [-W workers] [-z=<compress>] [-J compress] [-X] [-Z] [-T] [-V] [-h] [filter]

DESCRIPTION
nfdump reads the flow data from one or more binary files, created by any nfdump collector nfcapd, nfpcapd and sfcapd. It processes and lists the flows in many different output formats and can create a wide range of statistics.

nfdump has a very powerful flow filter to process flows. The filter syntax is very similar to tcpdump, but adapted and extended for flow filtering. A flow filter may also contain arrays of many thousand IP addresses etc. to search for specific records.

nfdump can aggreagte flows according to a user defined number of elements. This masks certain elements and allows to sum up flow records matching the same values.

The combination of flow filtering and aggregation as input for any flow statistics allows complex flow processing. Pre-filtered and aggregated flow data may also be written back into a binary flow file, which again may be processed with nfdump

nfdump can enrich the listing of flows with geo location information and AS information, unless AS information is already available in the flow records. IP addresses can be tagged with a two letter country code, or with a longer location label containing the geographic region, country and city. The geo location and AS information is retrieved from the optional geoDB database, created by the geolookup program from the nfdump tools. geolookup uses the Maxmind database GeoDB or GeoLite2 to create a binary lookup database for nfdump Please check the geolooup(1) man page for more details.

The options are as follows:

-r flowpath
    Reads flow records from this path. flowpath may be a single file, or a directory containing any number of flow files or sub directories. All files are processed in the order, as listed by the OS.

-w outfile
    Writes all processed records into outfile instead of printing. The flowfile is a binary flow file and may be processed again with nfdump This can be useful to limit flows according to a flow filter and/or specific flow aggregation.

-f filterfile
    Reads the flow filter from filterfile. This can be useful for very long or structured filters, with comments and long lists. Note: Any filter specified directly on the command line takes precedence over the filterfile.

-C config
    Read more options from file config. nfdump tries to read by default %prefix/etc/nfdump.conf. This may be overwritten by the environment variable NFCONF which again may be overwritten by this option -C. In order to prevent reading any config file, even if it would exist set -C none. A config file is not required, but may be handy for often used output formats etc.

-O order
    Sets an output order for records to be printed as text output. This order applies after all records processing, such as filtering, and aggregation and before printing.
    flows
        Sort according to the number of flows
    packets
        Sort according to (in)packets
    ipkg
        Same as packets
    opkg
        Sort according to output packets
    bytes
        Sort according to (in)bytes
    ibyte
        Same as bytes
    obyte
        Sort according to output bytes
    pps Sort according to (in)packets per second
    ipps
        Same as ipps
    opps
        Sort according to out packets per second
    bps Sort according to (in)bytes per second
    ibps
        Same as bps
    obps
        Sort according to output bytes per second
    bpp Sort according to (in)bytes per packet
    ibpp
        Same as bpp
    obpp
        Sort according to output packets
    tstart
        Sort according to start time of flow - former -m
    tend
        Sort according to end time of flows
    duration
        Sort according to duration of flows

-t timewin
    Set time window to process flows. This option is considered legacy and may be replaced with a filter primitive in future releases. The time window is specified as: YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss]. Any parts of the time spec may be omitted e.g YYYY/MM/dd expands to YYYY/MM/dd.00:00:00-infinity and processes all flow from a given day onwards.

-c num
    Limit the number of records to be processed to the first num records, which passwd the filter.

-a  Aggregate flow records. The default aggregation is done at connection level by taking the 5-tuple protocol, srcip, dstip, srcport and dstport. This way of aggregation may be overwritten by option -A

-A aggregation
    Sets the list of elements in a flow record to be aggregated. aggregation is a ',' separated list of any number of v9/ipfix elements. The following elements are accepted:
    proto
        IP protocol
    srcip
        Source IP address
    dstip
        Destination IP address
    srcip4/net
        IPv4 source IP address with applied netmask
    srcip6/net
        IPv6 source IP address with applied netmask
    dstip4/net
        IPv4 destination IP address with applied netmask
    dstip6/net
        IPv6 destination IP address with applied netmask
    srcnet
        Apply netmask srcmask in netflow record for source IP
    dstnet
        Apply netmask dstmask in netflow record for dest IP
    srcport
        Source port
    dstport
        Destination port
    srcmask
        Source mask
    dstmask
        Destination mask
    srcvlan
        Source vlan label
    dstvlan
        Destination vlan label
    srcas
        Source AS number
    dstas
        Destination AS number
    nextas
        BGP Next AS
    prevas
        BGP Previous AS
    inif
        SNMP input interface number
    outif
        SNMP output interface number
    next
        IP next hop
    bgpnext
        BGP next hop
    insrcmac
        In source MAC address
    outdstmac
        out destination MAC address
    indstmac
        In destination MAC address
    outsrcmac
        Out source MAC address
    tos Source type of service
    srctos
        Source type of Service
    dsttos
        Destination type of Service
    flags
        TCP flags
    mpls1
        MPLS label 1
    mpls2
        MPLS label 2
    mpls3
        MPLS label 3
    mpls4
        MPLS label 4
    mpls5
        MPLS label 5
    mpls6
        MPLS label 6
    mpls7
        MPLS label 7
    mpls8
        MPLS label 8
    mpls9
        MPLS label 9
    mpls10
        MPLS label 10
    router
        IP address of exporting router
    odid
        observation domain ID
    opid
        observation point ID
    xsrcip
        X-late source IP address, if compiled with NSEL support
    xdstip
        X-late destination IP address, if compiled with NSEL support
    xsrcport
        X-late source port, if compiled with NSEL support
    xdstport
        X-late destination port, if compiled with NSEL support

    nfdump automatically compiles the appropriate output format for the selected aggregation elements unless an explicit output format -o is given. The automatic output format is identical to

        -o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'

    where <fields> represents the selected aggregation tags.

-b  Aggregate flow records as bidirectional flows. This automatically implies -a. Aggregation is done on connection level by taking the 5-tuple protocol, srcip, dstip, srcport and dstport The reverse order applies for the corresponding reverse flow. Input and output packets/bytes are counted and reported separately. Both flows are merged into a single record with corresponding input and output counters. An appropriate output format is selected automatically, which may be overwritten by any -o format option.

-B  Similar to option -b but tries to guess the correct client to server direction. Automagically swaps flows if src port is < dst port for TCP and UDP flows and src port < 1024 and dst port > 1024. Some exporters do not really care sending the flows in proper order. It's considered to be a conveniency option.

-I  Print flow statistics of a single file or the summary of all the files specified by -r flowpath.

-g  Print for each flow file given by -r flowpath a one line summary, which can be easily used by gnu plot.

-D nameserver
    Sets the nameserver to translate hostnames into IP addresses in filter expressions. See filter below for more details.

-G geoDB
    Use geoDB as geo lookup DB for geo location and AS lookups. nfdump tries to read the environment variable NFGEODB for the path of geoDB. The option -G overwrites NFGEODB or geodb.path in nfdump.conf. In order to prevent reading any geoDB file, even if it would exist set -G none. See also geolookup(1)

-H torDB
    Use torDB as tor lookup DB for tor exit node lookups. nfdump tries to read the environment variable NFTORDB for the path of torDB. The option -H overwrites NFTORDB or tordb.path in nfdump.conf. In order to prevent reading any torDB file, even if it would exist set -H none. See also torlookup(1)

-s statistic [:p [/orderby]]
    Generate the Top N flow record or flow element statistic. By optionally adding:p to statistic, the statistic is additionally split up into the transport layer protocols. By default the statistic is transport protocol independent. Each statistic may be ordered by the optional parameter orderby This can be flows, packets, bytes, pps, bps or bpp. You may specify more than one orderby option, which results in the same statistic but ordered differently. If no orderby is given, the statistic is ordered by flows. You can specify as many -s flow element statistics as needed on the command line for the same run.

    statistic can be:

    record
        aggregated netflow records.
    srcip
        source IP addresses
    dstip
        destination IP addresses
    ip  any (src or dst) IP addresses
    srcgeo
        Two letter src geo country code of src IP address
    dstgeo
        Two letter dst geo country code of dst IP address
    geo Two letter src/dst geo country code of src/dst IP address
    nhip
        next hop IP addresses
    nhbip
        BGP next hop IP addresses
    router
        exporting router IP address
    srcport
        source ports
    dstport
        destination ports
    port
        any (source or destination) ports
    flags
        TCP flags
    tos type of service - default src
    srctos
        src type of service
    dsttos
        dst type of service
    dir flow directions ingress/egress
    srcas
        source AS numbers
    dstas
        destination AS numbers
    srcasn
        source AS organisations and numbers
    dstasn
        destination AS organisations and numbers
    as  any (source or destination) AS numbers
    asn any (source or destination) AS org and numbers
    inif
        input interface
    outif
        output interface
    if  any interface
    inam
        input interface name
    onam
        output interface name
    srcmask
        src mask
    dstmask
        dst mask
    srcvlan
        src vlan label
    dstvlan
        dst vlan label
    vlan
        any vlan label
    insrcmac
        input src MAC address
    outdstmac
        output dst MAC address
    indstmac
        input dst MAC address
    outsrcmac
        output src MAC address
    srcmac
        any src MAC address
    dstmac
        any dst MAC address
    inmac
        any input MAC address
    outmac
        any output MAC address
    mask
        any mask
    proto
        IP protocols
    tunproto
        IP tunnel protocols
    mpls1
        MPLS label 1
    mpls2
        MPLS label 2
    mpls3
        MPLS label 3
    mpls4
        MPLS label 4
    mpls5
        MPLS label 5
    mpls6
        MPLS label 6
    mpls7
        MPLS label 7
    mpls8
        MPLS label 8
    mpls9
        MPLS label 9
    mpls10
        MPLS label 10
    sysid
        Internal SysID of exporter
    nbar
        nbar ID
    ja3 ja3 hashes
    odid
        observation domain ID
    opid
        observation point ID
    vrf/ivrf
        ingress vrf
    evrf
        egress vrf
    ivrfnam
        ingress vrf name
    evrfnam
        egress vrf name

        NSEL/ASA statistics
    event
        NSEL/ASA event
    xevent
        NSEL/ASA extended event
    natsrcip
        NSEL/ASA nat src IP address
    natdstip
        NSEL/ASA nat dst IP address
    natip
        NSEL/ASA nat src/dst IP addresses
    nat srcport
        NSEL/ASA nat src port
    natdstport
        NSEL/ASA nat dst port
    natport
        NSEL/ASA nat src/dst port
    iacl
        NSEL/ASA ingress ACL
    iace
        NSEL/ASA ingress ACE
    ixace
        NSEL/ASA ingress xACE
    eacl
        NSEL/ASA egress ACL
    eace
        NSEL/ASA egress ACE
    exace
        NSEL/ASA egress xACE

        NAT statistics
    nevent
        NAT event
    nsrcip
        NAT src IP address
    nsrcport
        NAT src port
    ndstip
        NAT dst IP address
    ndstport
        NAT dst port

        % nfdump -s srcip -s ip/flows/bytes -s record/bytes

-n num
    Set the number of records to be printed to num. This option applies to -s statistics as well as to ordered output -O -or -aggregated -records -a The default is set to 10 for statistics and unlimited for the other use cases. To disable the limit, set num to 0.

-o format
    Sets the output format to print flow records. has many different output formats already predefined. format may be one of the options below:

    raw Print the full flow record on multiple lines. This prints all available information.
    fmt: user
        Print the flow records according the format user. This is a very flexible and powerful way to format flow records. See the section OUTPUT below for more details on how to compile your own format.
    csv: user
        Print the flow records as user defined csv format. Use a ',' separated list of user defined output token compatible with fmt format. See the section OUTPUT below for more details on how to compile your own csv format.
    json
        Print full record as a separate json object.
    ndjson
        Print full record as a one line json object, separated by newline. Suitable for log processors such as logstash.
    csv Print reocrd in csv format - format compatible to fmt line format.
    csv-fast
        Replaces old pipe format. Basic record information only. Fast implementation.

    Already predefined fmt formats:

    line
        Print each flow on one line. Default format.
    long
        Print each flow on one line with more details
    biline
        Same as line, but for bi-directional flows
    bilong
        Same as long, but for bi-directional flows
    gline
        Same as line, but add country code to IPs. If a geoDB file is supplied this is the default output format
    glong
        Same as long, but add country code to IPs
    extended
        Print each flow on one line with even more details.
    nsel
        Print format for NSEL event records. Default format if NSEL/NAT support has been compiled in.
    nel Print format for NAT event records.

    The nfdump config file may contain additional formats. If you want to add new formats or change existing ones, check the config file.

    IPv6 addresses are printed condensed in any fmt defined format to prevent cluttering the output with large blank blocks. A condensed IPV6 uses max 16 characters. If it is longer, then the middle part of the IP is cut out and replaced be "..". For previewing an output, this fits most needs. For a listing with the full IPV6 addresses add option -6.

-6  Print full length of IPv6 addresses in output instead of condensed.

-q  Quiet mode. Suppress the header line and the statistics at the bottom of text outputs.

-N  Print plain numbers in output without scaling. Easier for output parsing with 3rd party tools.

-i ident
    Change the ident label in the file, specified by -r to ident

-v flowfile
    Verify the consistency of flowfile and print the file parameters and number of records.

-E flowfile
    Print the exporter and sampler list if found in flowfile. Additional statistics per exporter are printed with number of flows, packets and sequence errors.

-x flowfile
    This options works on nfdump version 1.6.x files only and may get removed in future. Scans and prints extension maps located in flowfile

-z=lzo
    Compress flow files with LZO1X-1 compression. Fastest compression.

-z=bz2
    Compress flow files with bz2 compression. Slow but most efficient. May be used for archiving files or if you are really short of spce.

-z=lz4[:level]
    Compress flow files with LZ4 compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See -W.

-z=zstd[:level]
    Compress flow files with ZSTD compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See -W.

-W num
    Sets the number of workers to compress flows. Defaults to 4. Must not be greater than the number of cores online. Useful for higher levels of compression for lz4 or zstd and large amount of flows per second. Please not, -W affects only writing flows.

-J compress
    Change compression for any number of files given by option -r flowpath Set compress to 0 for no compression or to any of: 1 or LZO, 2 or BZ2, 3 or LZ4. This option may be used for archiving flow files and changing the compression to use less disk space.

-X  Compiles the filter syntax and dumps the filter engine table to stdout. This is for debugging purpose only.

-Z  Check filter syntax and exit. Sets the return value accordingly.

-R filelist
    Select a range of files. This option is mainly used by old NfSen and documented here as legacy option.
    /any/dir Read recursively all files in directory dir.
    /dir/file Read all files beginning with file.
    /dir/file1:file2 Read all files from file1 to file2.
    When using in combination with a sub hierarchy: /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from sub1/sub2/file1 sub3/sub4/file2 iterating over all required hierarchy levels. Note: files are read in alphabetical order.

-M dirlist
    Read the same file hierarchy from multiple directories. This option is mainly used by old NfSen and documented here as legacy option. Example: /any/path/to/dir1:dir2:dir3 etc. and will be expanded to the directories: /any/path/to/dir1, /any/path/to/dir2 and /any/path/to/dir3. Any number of colon separated directories may be given. A path ending with a wildcard '@' such as /any/path/to@ will expand automatically in a list of all existing sub directories /any/path/to/dir1:dir2:dir3. The files to read are specified by -r or -R and are expected to exist in all the given directories. The options -r and -R must not contain any directories when used in combination with -M.

-T  Tag IP addresses with a prepending cntrl-A character, to allow output parsers to hook in. This option is mainly used by old NfSen and documented here as legacy option.

-V  Print nfdump version and exit.

-h  Print help text on stdout with all options and exit.

filter selects, which records will be further processed. If no filter is given, all records will be processed. Otherwise, only those flows matching the filter will be processed. Any IP address in a filter may be specified as IPv4 or IPv6.

The filter syntax is similar to tcpdump but adapted and extended for flow records. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case insensitive.

A single filter primitive filters a single element of a flow record. A filter consists of one or more primitives, which are linked together:

    expr and expr
    expr or expr
    not expr and (expr)

In all expressions, where a number is a valid argument, the number may be given as a normal decimal number or as a hex number prefixed by 0x such as 0x22. A decimal number may also contain a multiplication factor such as K, M, G, T which multiplies the number by the corresponding factor. For example 1K, 2G etc.

String arguments may be single or double quoted or not quoted at all, if the string is not a reserved filter key word - src geo CH but src geo 'IN'

Possible filter primitives:
@include file
    Expands the content of file into the current filter

count comp number
    True if the comparison with the record counter matches number Each record gets assigned a record number at the time it is read from file. Therefore this record number is not unique and may change, depending on the order files are read.
ident string
    True if the record ident field matches string. This filter can be used to filter out different sources.

inet
ipv4
    True if source and destination IP of a record are IPv4 IPs.

inet6
ipv6
    True if source and destination IP of a record are IPv6 IPs.

min ttl comp num
max ttl comp num
    True if min/max IP ttl matches comparison.

ttl comp num
    True if min or max IP ttl matches comparison.

ttl equal
    True if min and max IP ttl have the same value

proto protocol
    True if the record protocol field matches protocol. protocol can be a string such as tcp, udp, icmp, ah, esp, ipip, and many more or a protocol number, such as 6, 17 for protocol tcp and udp.

tun proto protocol
    True if the record tunnel protocol field matches protocol. protocol may be a string or protocol number.

ip ipaddr
src ip ipaddr
dst ip ipaddr
    True if the respective IP field of the record matches ipaddr. ipaddr may be an IPv4 or IPv6 address or a symbolic hostname. In this case a DNS lookup resolves the hostname to one or more IP addresses. If more than one IP results, all IPs are chained together in an or chain. (IP or IP or IP). If ip is not specified with src or dst the source or destination IP may match. If ipaddr is set to tor then flows are listed, if the respective ip addr is a to tor exit not. For this filter to work, you need a working nftordb.
host ipaddr
    host is just a synonym for ip (See above)

ip in [iplist]
src in ip [iplist]
dst ip [iplist]
    True if the respective IP field of the record is in iplist. iplist is a space or ',' separated list of IP addresses or networks in CIDR notation. This is the preferred way to search in large list of IP addresses and networks and is much more efficient than to chain all IP addresses together. (IP1 or IP2 or IP3). The iplist may contain several hundreds to thousand IPs and/or networks. For just a few IPs use an or chain, otherwise use an iplist If ip is not specified with src or dst the source or destination IP may match.

net network netmask
src net network netmask
dst net network netmask
net network/netbits
src net network/netbits
dst net network/netbits
    True if the respective IP field of the record matches the network if the corresponding netmask or netbits are applied to the IP address. If net is not specified with src or dst the source or destination IP may match.

geo string
src geo string
dst geo string
    True, if the 2-letter country code resolved by geolookup of the source or destination IP address matches string. This filter works only, if a valid geoDB is specified. See geo location option above. The 2-letter country code corresponds to the maxmind DB definitions. if geo is not specified with src or dst the source or destination geo location code may match. Please note: country codes, which match nfdump filter language reserved words such as IN, LT etc must be explicitly quoted to be recognized as string.

tun ip ipaddr
src tun ip ipaddr
dst tun ip ipaddr
    True if the respective tunnel IP field of the record matches ipaddr. If tun ip is not specified with src or dst the source or destination tunnel IP may match.

port comp num
src port comp num
dst port comp num
    True if the comparison of the respective port field matches num See comp for the comparator details. If port is not specified with src or dst the source or destination port may match.

port in [portlist]
src port in [portlist]
dst port in [portlist]
    True if the respective port field of the record is in portlist. portlist is a space or ',' separated list of port numbers. This is the preferred way to search in large list of port numbers and is much more efficient than to chain all ports together. (PORT1 or PORT2 or PORT3). portlist may contain several hundreds to thousand of port numbers. If port is not specified with src or dst the source or destination port may match.

icmp type num
icmp code num
    True if the respective icmp field of the record matches num. This automatically implies proto icmp.

engine type num
engine id num
sysid num
    True if the respective fields of the record matches num engine type and ID are set by the exporting device, sysid refers to the nfdump collector internal assigned number. See also option -E above.

if num
in if num
out if num
    True if the respective interface fields of the record matches num. This ID may correspond to the SNMP ID of the interface but depends on the exporter. If if is not specified with in or out the input or output interface may match.

as comp num
src as comp num
dst as comp num
prev as comp num
next as comp num
    True if the comparison of the respective AS fields matches nfdump supports 32-bit AS numbers every where. Without or the source or destination AS may match. See comp for the comparator details.

as in [aslist]
src as in [aslist]
dst as in [aslist]
prev as in [aslist]
next as in [aslist]
    True if the respective AS field of the record is in aslist. aslist is a space or ',' separated list of AS numbers. This is the preferred way to search in large list of AS numbers and is much more efficient than to chain all ports together. aslist may contain several hundreds to thousand of AS numbers. If as is not specified with src, dst, prev or next the source or destination AS may match.

mask bits
src mask bits
dst mask bits
    True if the respective mask bit field of the record matches bits If mask is not specified with src or dst the source or destination mask bits may match.

vlan num
src vlan num
dst vlan num
    True if the respective vlan field of the record matches num If vlan is not specified with src or dst the source or destination vlan may match.

flags tcpflags
    True if the respective tcp flags field of the record matches any of the given tcpflags. tcpflags is a string combination of all flags to be tested:
    A   ACK.
    S   SYN.
    F   FIN.
    R   Reset.
    P   Push.
    U   Urgent.
    X   All flags on.
    The order of the flags within tcpflags is not relevant. Flags not mentioned are treated as don't care. In order to get those flows with only the SYN flag set, use the syntax

        flags S and not flags AFRPU

router ip ipaddr
    True if the ip address of the sending router matches ipaddr as valid IPv4/IPv6 address.

next ip ipaddr
    True if the field next-ip of the record matches ipaddr as valid IPv4/IPv6 address.

bgp next ip ipaddr
    True if the field bgpnext-ip of the record matches ipaddr as valid IPv4/IPv6 address.

mac macaddr
in mac macaddr
in src mac macaddr
in dst mac macaddr
out mac macaddr
out src mac macaddr
out dst mac macaddr
    True if the respective mac address field of the record matches macaddr By prepending mac with any combination of a direction specifier as defined by CISCO v9 the test is limited to those mac addresses only. Otherwise multiple matches are possible. Without any specifiers any mac address is tested against macaddr

mpls labelN comp number
    True if the comparison of the mpls label N with N as mpls label number 1..10 matches number Filters according a specific number in the mpls label stack.

mpls eos comp number
    True if the comparison of the end of stack mpls label matches number

mpls expN comp number
    True if the comparison of the experimental bits 0..7 of mpls label N with N as mpls label number 1..10 matches number

packets comp num
in packets comp num
out packets comp num
    True if the comparison of the packet counter in the flow record matches num. num may contain any valid scaling factor such as k, m, g Example: packets > 1k. For a single flow packets and in packets is equivalent and describes the number of packets from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option --B ) the packet counter for the reverse flow can be tested with out packet

bytes comp num
in bytes comp num
out bytes comp num
    True if the comparison of the byte counter in the flow record matches num. num may contain any valid scaling factor such as k, m, g Example: bytes > 1k bytes and in bytes is equivalent and describes the number of bytes from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option --B ) the byte counter for the reverse flow can be tested with out bytes

flows comp num
    True if the comparison of the flow counter in the flow record matches num. num may contain any valid scaling factor such as k, m, g For each received flow, the flow counter is set to 1, unless the exporter sends this information. If multiple flows are aggregated, this counter is increased respectively.

tos num
    True if the type of service field of the flow record matches num

flowdir direction
    True, if the flow direction field in the flow record matches direction. direction may be ingress, egress, 0 for ingress, or 1 for egress

duration comp time
    True if the calculated duration of a flow (tend - tstart) compares to time. The duration is specified in msec (milliseconds)

first seen comp timeISO8601
last seen comp timeISO8601
    True if the start/end time of a flow compares to timeISO8601. The time follows ISO8601 format. e.g. 2024-07-11T09:15:10.020. Up to msec may be specified. A string may be incomplete e.g. 2024-07-11T09 which is automatically extended to 2024-07-11T09:00:00.000. At least a valid year is required.

pps comp num
    True if the calculated value of in-packets/duration (packets per second) compares with the number num. num may contain any valid scaling factor such as k, m, g

bps comp num
    True if the calculated value of 8*in-bytes/duration (bits per second) compares with the number num. num may contain any valid scaling factor such as k, m, g

bpp comp num
    True if the calculated value of in-bytes/in-packets (bytes per packet) compares with the number num. num may contain any valid scaling factor such as k, m, g

observation domain id comp number
observation point id comp number
    True if the comparison of the observation domain ID or point ID field respectively matches number

payload filters
    Some exporters, such as yaf or the nfdump collector nfpcap can send payload data along the netflow information. If such payloads are sent it can be filtered according the filter primitives below:

payload content 'string'
    True if the string string is found in the payload data. string must be quoted with single or double quotes: 'string', “string”

payload regex 'regex'
payload regex 'regex' flags
    True if regex matches the payload data. regex searches over the full payload length. A ' ' byte does not stop the match process. regex must be quoted with single or double quotes: 'regex' or “regex” The regex engine understands the following reduced syntax:

    (...) subexpressions/capture ranges
    | the "or" operator
    ^ and $ anchors
    [...] and [^...] character classes
    ?, *, +, simple quantifiers
    *?, +?, ?? lazy quantifiers
    {<num>}, {<num1>,<num2>} complex quantifiers
    \d numeric character
    \w alpha numeric character
    \r carrige return
    \n new line

    flags are optional can be:
    m multiline
    i case insensitive matching
    s dot '.' includes newlines

payload dns defined
    True, if the payload decodes to a valid dns query/answer. Implies src or dst port 53

payload dns name dnsname
    True, if any of the payload dns queries contain the string dnsname. It may be the full name or a sub string. Implies src or dst port 53

payload dns address IP
    True, if any of the payload dns answers or additional records contain the string IP. It may be the full or partial IP address. Implies src or dst port 53

payload ssl defined
    True, if the payload contains the start of a valid SSL/TLS handshake

payload ssl version version
    True, if the payload contains the start of a valid SSL handshake and the SSL/TLS version matches version Valid versions are 1.0, 2.0, 3.0

payload tls version version
    True, if the payload contains the start of a valid TLS handshake and the TLS version matches version Valid versions are 1.0, 1.1, 1.2, 1.3

payload tls sni sniname
    True, if the payload contains the start of a valid TLS handshake and the TLS sni name contains the string sniname

payload ja3 md5string
    True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja3 value of the handshake matches md5string Depending on client or server SSL handshake, either ja3 or ja3s is calculated.

payload ja3 defined
    True, if the payload contains the start of a valid SSL/TLS handshake and a valid ja3 value can be calculated. Useful to mask out all flow records with no SSL/TLS traffic in order to generate a -s ja3 statistic

payload ja4 ja4string
    True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja4 value of the client TLS handshake matches ja4string

payload jas4 ja4Sstring
    True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja4 value of the server TLS handshake matches ja4Sstring

payload ja4 defined
    True, if the payload contains the start of a valid SSL/TLS handshake and a valid ja4/ja4s value can be calculated. Useful to mask out all flow records with no SSL/TLS traffic in order to generate a -s ja3 or a -s ja3s statistic

OpenBSD pflog implemented elements

pf action action
    True, if the respective pflog action field compares to one of pass, block, scrub, noscrub, nat, nonat, binat, nobinat, rdr, nordr, synblock, defer, match, divert, rt, afrt
pf reason reason
    True, if the respective pflog reason field compares to one of match, bad-offset, fragment, short, normalize, memory, bad-timestamp, congestion, ip-option, proto-cksum, state-mismatch, state-insert, state-limit, src-limit, synproxy, translate, no-route
pf rule ruleNr
    True, if the respective pflog rule number field matches ruleNr
pf dir in|out
    True, if the respective pflog rule direction field matches in or out
pf interface interfaceName
    True, if the respective pflog rule interface name field matches the string interfaceName

nprobe implemented elements

client latency comp time
server latency comp time
    True, if the respective latency field in the flow record compares to time. time is specified in msec.

CISCO ASA, network security event logging (NSEL) and NAT event logging (NEL) specific filters:

asa event event
    True if the NSEL event type of an event record matches event which may be: ignore, create, term, delete, deny

asa event comp number
    True if the comparison of the NSEL event type of an event records matches number as a number.

asa event denied reason
    True if the event denied type of an event records matches reason which may be ingress, egress, interface, nosyn

asa xevent comp num
    True, if the comparison of the extended event field of the event record matches num

nat event event
    True if the NEL event type of an event record matches event. event may be add, delete

nat event comp number
    True if the comparison of the NEL event type of an event records matches number as a number.

nat ip ipaddr
src nat ip ipaddr
dst nat ip ipaddr
    True, if the field of the translated source or destination IP address matches ipaddr. If nat ip is specified without src or dst both IP addresses may match.

nat port port
src nat port port
dst nat port port
    True, if the field of the translated source or destination port matches port. If port is specified without src or dst both ports may match.

nat net network/mask
src nat net network/mask
dst nat net network/mask
    True if the translated source or destination IP address matches network if mask mask is applied. if xnet is specified without src or dst both IP addresses may match.

pblock start comp number
pblock step comp number
pblock end comp number
    True if the comparison of the start, step or end of the NAT port block in the event record matches number
port in pblock
src port in pblock
dst port in pblock
    True, if the source or destination port field matches the NAT port block range

ingress ACL comp number
ingress ACE comp number
ingress XACE comp number
    True if the comparison of the respective ingress field matches number

egress ACL comp number
    True if the comparison of the egress field matches number

ingress vrf number
    True, if the field of the ingess vrf field of the event record matches number

comp
    Many filter elements support the comparison with a number. The following comparators are supported for each of those filters: =, ==, >, <, >=, <= To prevent collisions with bash interpretation, alternative comparators are available: EQ, LT, GT, LE, GE If comp is omitted, '==' is assumed.

OUTPUT FORMAT
This section describes how output formats are compiled. nfdump has a lot of already pre-defined output formats such as raw, json, ndjson, csv etc. One line formats supplied with option -o can be compiled from various elements of a flow record. As a flow record contains many different elements it is often useful to compile an output format for specific needs.

  Format description
The output format is specified by -o “fmt: string” for line output or -o “csv: string” for csv output. String contains the field tags to be printed as well as other characters if needed. A tag starts with a % sign followed by the field name. tags are separated by spaces for line format or by ',' for csv format. Characters or other strings, not starting with a % sign are copied literally to the output.

Example:
    -o “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
    -o “csv:%ts,%td,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl”

The difference between fmt and csv is the printing of the header line. Furthermore csv does not have a flow summary at the end. csv output can be processed by any other post processing tool, which understands csv data. The examples above show the definitions of the predined formats line and csv. It adds the elements tstart duration protocol source IP address/port followed by the literal characters -> and destination ip address/port packets, bytes, flows counter. Depending on the task, different output formats are required to see the required fields of a flow record. You can either extend a predefined format or specify a new one at the command line.

Example: Extend the predefined format long with the the IP address of the sending router
    -o “fmt:%long %ra”

Predefined formats can be extended by simply add their name with a % sign somewhere in the format string. As described under the output option -o

  Format definition
nfdump has already many formats predefined. Most of the time, these format are good enough. Sometimes you may need different formats, which can be compiled as described above. In order to prevent adding the same often used output format each time you run nfdump a new output format may be define in the config file nfdump.conf The file nfdump.conf.dist contains the definition of the already hard coded formats. These may be uncommented and changed according to the specific needs. New formats may be added using the following syntax:

    fmt.newname = “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”

with newname any new or existing definition of output formats. Existing formats are overwritten with the new definition. You may also define a custom csv output format such as:

    csv.newname = “csv:%tsr,%ter,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl”

Please note, that newname must be unique for all formats in fmt as well as csv

  Tag definition
The following list contains all tags, which are available to compile the output format:

%<format>
    Inserts the predefined format at this position. e.g. %line
%cnt
    Record counter. record numbers are assigned dynamically assigned while reading read from file.
%idt
    Ident string. Source identifier string -i to nfcapd and constant for all flows in a file.
%nfv
    Netflow version.
%ts Start Time - first seen
%tfs
    First seen - identical to %ts
%tsr
    Start Time, but in fractional seconds since the epoch (1970-01-01) UNIX format.
%tsg
    Start Time GMT - first seem
%te End Time - last seen
%ter
    End Time, in fractional seconds
%teg
    End Time GMT - last seen
%tr Time the flow was received by the collector
%trr
    Time the flow was received, in fractional seconds
%trg
    Time GMT the flow was received by the collector
%td Duration of flow. Displayed in ddHHMMSS.msec
%pr Transort protocol
%exp
    Exporter ID
%eng
    Engine Type/ID
%lbl
    Flowlabel
%sa Source Address
%da Destination Address
%sap
    Source Address:Port
%dap
    Destination Address:Port
%gsap
    Source Address(country code):Port
%gdap
    Destination Address(country code):Port
%sp Source Port
%dp Destination Port
%minttl
    IP min ttl
%maxttl
    IP max ttl
%ttl
    IP min/max ttl
%it ICMP-type
%ic ICMP-code
%sn Source Network, mask applied
%dn Destination Network, mask applied
%nh Next-hop IP Address
%nhb
    BGP Next-hop IP Address
%ra Router IP Address
%sas
    Source AS
%das
    Destination AS
%nas
    Next AS
%pas
    Previous AS
%in Input Interface num
%out
    Output Interface num
%pkt
    Packets - default input
%ipkt
    Input Packets
%opkt
    Output Packets
%byt
    Bytes - default input
%ibyt
    Input Bytes
%obyt
    Output Bytes
%fl Flows
%flg
    TCP Flags
%tos
    Tos - default src
%stos
    Src Tos
%dtos
    Dst Tos
%dir
    Direction: ingress, egress
%smk
    Src mask
%dmk
    Dst mask
%fwd
    Forwarding Status
%svln
    Src vlan label
%dvln
    Dst vlan label
%scvln
    Src customer vlan label
%dcvln
    Dst customer vlan label
%ismc
    Input Src Mac Addr
%odmc
    Output Dst Mac Addr
%idmc
    Input Dst Mac Addr
%osmc
    Output Src Mac Addr
%mpls1
    MPLS label 1
%mpls2
    MPLS label 2
%mpls3
    MPLS label 3
%mpls4
    MPLS label 4
%mpls5
    MPLS label 5
%mpls6
    MPLS label 6
%mpls7
    MPLS label 7
%mpls8
    MPLS label 8
%mpls9
    MPLS label 9
%mpls10
    MPLS label 10
%mpls
    MPLS labels 1-10
%bps
    bps - bits per second
%pps
    pps - packets per second
%bpp
    bps - Bytes per package
%sc src IP 2 letter country code
%dc dst IP 2 letter country code
%sloc
    src IP geo location info
%dloc
    dst IP geo location info
%sasn
    src AS organisation name
%dasn
    dst AS organisation name
%stor
    src IP 2 letter tor exit info: TX tor exit node
%dtor
    dst IP 2 letter tor exit info: TX tor exit node
%n  new line char \n
%ipl
    input payload
%opl
    output payload
%nbid
    nbar ID
%ja3
    ja3 hash
%sni
    sni name in tls handshake
%nbnam
    nbar name
%odid
    observation domainID
%opid
    observation pointID

OpenBSD pflog specific formats
%pfifn
    pflog interface name
%pfact
    pflog action
%pfrea
    pflog reason
%pfdir
    pflog direction
%pfrule
    pflog rule nr

NSEL specific formats
%nfc
    NSEL connection ID
%evt
    NSEL event
%xevt
    NSEL extended event
%sgt
    NSEL Source security group tag
%msec
    NSEL event time in msec
%iacl
    NSEL ingress ACL
%eacl
    NSEL egress ACL
%xsa
    NSEL XLATE src IP address
%xda
    NSEL XLATE dst IP address
%xsp
    NSEL XLATE src port
%xdp
    NSEL SLATE dst port
%xsap
    Xlate Source Address:Port
%xdap
    Xlate Destination Address:Port
%uname
    NSEL user name

NEL/NAT specific formats
%nevt
    NAT event - same as %evt
%ivrf
    NAT ingress VRF ID
%evrf
    NAT egress VRF ID
%nsa
    NAT src IP address
%nda
    NAT dst IP address
%nsp
    NAT src port
%ndp
    NAT dst port
%pbstart
    NAT pool block start
%pbend
    NAT pool block end
%pbstep
    NAT pool block step
%pbsize
    NAT pool block size
%flid
    Flow ID
%isid
    Nokia NAT in service ID
%osid
    Nokia NAT out service ID
%nats
    Nokia NAT string

Nprobe formats
%cl Client latency
%sl Server latency
%al Application latency

EXAMPLES
nfdump processes files created by any previous version of nfdump 1.6.x with some limitations for versions < 1.6.17. In order to convert flow files to the new 1.7.x binary format use the following command to read//write files:

    % nfdump -r oldfile -w newfile

Print a statistic about the top 20 IP addresses, once sorted by flows and once by bytes

    % nfdump -r flowfile -s ip/flows/bytes -n 20

Print two statistics, one about the source IP and one about the destination IP address limited to flow with either source or destination port 443

    % nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port 443'

Print a statistic about the IP pairs, which exchanged most traffic.

    % nfdump -r flowfile -s record/bytes -A srcip,dstip

Print all flows in raw format with a HTTP header in the payload even if flow is not on port 80.

    % nfdump -r flowfile -o raw “payload regex 'GET|POST'”

Print a statistic about all ja3 md5 sums for those flows, which a valid ja3 can be calculated

    % nfdump -r flowfile -s ja5 -n 0 'payload ja3 defined'

Aggregate all flows and write the result back to a binary file, sorted by the start time

    % nfdump -r flowfile -a -Otstart -w newfile

RETURN VALUES
nfdump returns 0 on success and 255 if processing failed.

SEE ALSO
https://www.iana.org/assignments/ipfix/ipfix.xhtml

https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html

nfcapd(1) nfpcapd(1) sfcapd(1) geolookup(1)

BUGS
No software without bugs! Please report any bugs back to me.