14.1.13 Netflow measurement serversThe Netflow measurement system is non-active and non-discoverable PVSR module. The PVSR system supports Netflow version 1, 5 and 9 as well IPFIX (“Netflow 10”). It expects packets right from the netflow source (i.e. Cisco router, Linux server) thus there is no need for an additional netflow analyzer application. The administrator can separately specify the level of parallelism for the normal measurement collection and for the Netflow analysis (14.1.13.1). The first can be set with NETFLOW_PARALLEL_PROCESSING_WEB, the second with NETFLOW_PARALLEL_PROCESSING_COLLECTOR in the CONFIG_INI.pm. Both must be positive integer numbers. By default the measurements are calculated by only one process while the analysis is run by two processes (per analysis). The maximum number shouldn’t be bigger than the total number of CPU cores on the PVSR collector server. However it is important to note that more parallelism requires more memory as well. Similarly the NETFLOW_PARALLEL_PROCESSING_TOP_TALKERS parameter can be used to configure the parallel processing of top talkers data. To the data collector arbitrary Source and Destination IP address/port/interface ranges as well as the used protocol (i.e. TCP) can be specified, which summarizes the traffic according to these parameters. Additionally to their basic parameters the devices have the following parameters (providing the name codes of the given parameters in brackets, see chapter 7.5.6 on Parameters of non-SNMP data collectors):
If version 1 type Netflow packets are coming from the source, then a setting where the data will be unprocessed in case of missed packets shall not be set because at this Netflow version the packets are not numbered.
For the Netflow measurement server three device level and two range level measurement variables are available (providing the variable name used at measurement definition in parenthesis):
There are four measurement level parameters in the system. None of them are displayed to non-admin users and neither of them is mandatory. If any of them are omitted then there will be no filtering by that parameter:
<!><tos_filter:><ifnumber:><ASnumber:><ip></subnet><:port1><-port2> More than one of these conditions can be specified for both the Source and Destination parameters in separate lines. If the condition starts with an exclamation mark then that means negation, that is, the session traffic is not counted in the measurement if the condition is true for it. The system evaluates the negating conditions first, and if any of them is true for the session, then the processing does not continues. The remaining non-negating conditions are processed with the logical OR connective between them. None of the elements are mandatory and the separating characters must be set only if the given element is used (i.e. the : (colon) character in case of the ifnumber element). The descriptions of the elements are:
Examples:
At the installation of measurement server the following graphical templates are created:
14.1.13.1 Netflow analysisPVSR not only does the measuring based on the Netflow information received but it also stores the detailed source-destination session information every 5 minutes for some time (this can be specified per equipment). Using this data the administrators and the restricted administrators can create on-demand reports, but a restricted administrator can only use this feature if he is also allowed to modify the equipment. These reports can show the collected statistics in several ways, using different groupings, filtering, etc… The Netflow reports are accessible through the Measurement menu, if we open a Netflow equipment or a charts containing Netflow measurements or a virtual equipment which has Netflow charts. If we are viewing a chart or a virtual equipment then PVSR automatically filters the session by the filters specified for the measurements and a flow is only shown if it matches at least one criteria. These implicit criterias are always shown on the top of the page. The results can be filtered additionally with the filtering fields on the Browser area. The acceptable filtering values are the same as for the measurement settings. The application can resolve the IP addresses and represent host names instead of them (this is the default setting), however keep in mind that by using this feature the report generation takes more time and that the resolving is done by the machine on which the WEB server is running. The administrators can also easily create new Netflow measurements using these reports by clicking on an [add new] link in the result table. PVSR will try to create a new measurement with the same filtering as the aggregation, although sometimes the grouping cannot be exactly turned into a measurement filtering. In such a case PVSR warns the user that the filtering presented might not be exactly the same as the grouping. If the session data comes from different Netflow sources then the user has the option to either group those sessions together or to differentiate them based on the Netflow source. For example if we have two routers and the traffic only flows through either of them then mostly it makes sense to group the sessions together and do not care from which source the informate came. However if the flows go through both of them then by grouping them together we would “duplicate” the session and would see twice the throughput for any given row in the report. This can be changed using the Grouping checkbox. If we are using additional filtering then we can choose between unidirectional or bidirectional interpretation. In the first case if we specify a value for the source filtering then only those sessions are taken into account which have a matching source address, while in the second case those sessions also count which have a matching destination address. The bidirectional case make more sense if we are looking for example for any traffic from or to a given host, answering the question “With whom did the host speek?”. The unidirectional case makes more sense if we are looking for example for those hosts which are responsible for the highest downloads on an xDSL link, by specifying the local subnet in the destination filter. The system offers several different tabular reports, the type of the report can be changed using the Report parameter on the Browser area. All reports mean some kind of aggregation, the difference is the parameters used for grouping the flows 1 Host: the grouping is done by the IP address, the report shows the input and output traffic for the hosts 2 Host and type: the grouping is done by the IP address and the used port (see later), the report shows the input and output traffic for the hosts separating the different protocols 3 Type: the grouping is done by the used port (see later), the report shows the sum traffic for the protocols (there are no separate in and out values in this case) 4 Host-host: the grouping is done by the source and destination IP address pairs, the report shows the traffic data between the host pairs 5 Host-host and type: the grouping is done by the source and destination IP address pairs as well by the used port (see later), the report shows the traffic data between the host pairs separating the different protocols 6 Interface-interface: the grouping is done by input and output interface 7 Interface-interface and type: the grouping is done by input and output interface and the used port (see later) 8 AS: Autonomous System 9 AS and type: Autonomous System and the used port (see later) 10 AS-AS: Autonomous System source and Autonomous System destination 11 AS-AS and type: Autonomous System source and Autonomous System destination and the used port (see later) 12 Host and ToS: the grouping is done by the IP address and the ToS byte, the report shows the input and output traffic for the hosts 13 Host and type and ToS: the grouping is done by the IP address, the used port (see later) and the ToS byte, the report shows the input and output traffic for the hosts separating the different protocols 14 Type and ToS: the grouping is done by the used port (see later) and the ToS byte, the report shows the sum traffic for the protocols (there are no separate in and out values in this case) 15 ToS: the grouping is done by the ToS byte, the report shows the sum traffic for the type of services (there are no separate in and out values in this case) 16 Host-host and ToS: the grouping is done by the source and destination IP address pairs and the ToS byte, the report shows the traffic data between the host pairs 17 Host-host and type and ToS: the grouping is done by the source and destination IP address pairs as well by the used port (see later) and the ToS byte, the report shows the traffic data between the host pairs separating the different protocols 18 Interface-interface and ToS: the grouping is done by input and output interface and the ToS byte If the grouping is done by the ToS byte as well then the user must also specify the way PVSR should interpret the ToS byte: · DSCP: ToS is interpreted as a DSCP value, so only the first six bits are significant · Precedence: ToS is interpreted as a precedence value, so only the first three bits are significant · ToS: ToS is interpreted as Type of Service so only bits 4-1 are significant · Precedence+ToS: ToS is interpreted as Precedence and Type of Service, so only the first seven bits are significant PVSR maintains a known port list. This list can be edited by opening a Netflow equipment under the Setting -> Site and equipment configuration menu. The setting page is accessible using the “Netflow known-port configuration” from the Operation drill-down list. Only an administrator can access that page and the modifications affect every Netflow equipments not just that one which was opened. If the sessions should by grouped by the their type then PVSR determines during the report processing for each of them whether it should regard the source and/or the destination port as a known-port. Based on this decision it represents the session in one or two rows in the report: 1 If one of the port is below 1024, and the other is above 1024, then the later one isn’t regarded as a known port 2 If only of the two ports is known then it will be added only to its statistics 3 If both of them are known or neither of them are then: 3.1 For the Host-host and type report: both of them will be presented in the type field in the report 3.2 For the Host-host and for the Type report: the traffic data will be added to both statistics Example: consider two session information: A host X port -> B host Y port A host Z port -> B host Y port If every port is above 1024, X and Y are known ports as x and y, and Z isn’t known, then: 1 Type report: 1.1 x: the data for the first session will be added to it 1.2 y: the data for both sessions will be added to it 2 Host and type report: 2.1 A, x: the data for the first session will be added to its output value 2.2 A, y: the data for the second session will be added to its output value 2.3 B, y: the data for both sessions will be added to its input value 3 Host-host and type report: 3.1 A, B, x-y: the data for the first session will be added to its source -> destination value 3.2 A, B, y: the data for the second session will be added to its source -> destination value If every port is above 1024, and X is known as x, Y is known as y and Z is known as x too, then: 1 Type report: 1.1 x: data for both sessions will be added to it 1.2 y: data for both sessions will be added to it 2 Host and type report: 2.1 A, x: the data for both sessions will be added to its output value 2.2 B, y: the data for both sessions will be added to its input value 3 Host-host and type report: 3.1 A, B, x-y: the data for the both sessions will be added to its source -> destination value The tabular reports are TopN reports in every case, the sorting can be changed by clicking on the icons in the heading of the table. The analysis page also contains a pie chart based on the values in the table below it. The pie chart represents at any given time only one column from the table (by default the one used for ordering), this can be changed by clicking on the column name next to the pie chart. By clicking on the links in the byte and packet cells, PVSR displays a chart depicting the detailed traffic information for that particular entry.
|