Using the DMF Recorder Node with Analytics

This chapter describes Arista Analytics to use with the DMF Recorder Node. It includes the following sections.

Overview

The DMF Recorder Node records packets from the network to disk and recalls specific from disk quickly, efficiently, and at scale. A single DANZ Monitoring Fabric controller can manage multiple DMF Recorder Nodes, delivering packets for recording through DANZ Monitoring Fabric policies. The controller also provides central APIs for interacting with DMF Recorder Nodes to perform packet queries across one or multiple recorders and for viewing errors, warnings, statistics, and the status of connected recorder nodes.

A DANZ Monitoring Fabric policy directs matching packets to one or more recorder interfaces. The DMF Recorder Node interface defines the switch and port used to attach the recorder to the fabric. A DANZ Monitoring Fabric policy treats these as delivery interfaces.

Both NetFlow and TCPflow dashboards have the recorder node visualization.

General Operation

To retrieve packets from the DMF Recorder Node for analysis using Arista Analytics, select the controller and log in from Recorder Node window on the NetFlow or Flows dashboard. To add a new controller, click the small Select Controller icon and add the controller. After logging in to the DMF Recorder Node, the system displays the following dialog:
Figure 1. DMF Recorder Node

The Recorder Node window lets you compose and submit a query to the DMF Recorder Node. Use any of the fields shown to create a query and click Submit. The Switch Controller link at the bottom of the dialog lets you log in to a different DMF Recorder Node.

You can use the Recorder Summary query to determine the number of packets in the recorder database, and then apply filters to retrieve a reasonable number of packets that include the most interesting information.

You can modify the filters in the recorder query until a Size query returns the most useful number of packets.

Query Parameters

The following parameters are available for queries:
  • Query Type
    • Size: Retrieve a summary of the matching packets based on the contents and search criteria stored in the recorder node. Here Size refers to the total frame size of the packet.
    • AppID: Retrieve details about the matching packets based on the contents and search query in the recorder node datastore, where the actual packets are stored. Use this query to see what applications are in encrypted packets.
    • Packet Data: Retrieve the raw packets that match the query. At the end of search query a URL is generated pointing to the location of the pcap if the search query is successful.
    • Packet Objects: Retrieve the packet objects that match the query. At the end of search query a URL is generated pointing to the location of the objects (images) if the search query is successful.
    • Replay: Identify the Delivery interface in the field that appears, where the replayed packets are forwarded.
    • FlowAnalysis: Select the flow analysis type (HTTP, HTTP Request, DNS, Hosts, IPv4, IPv6, TCP, TCP Flow Health, UDP, RTP Streams, SIP Correlate, SIP Health).
  • Time/Date Format: Identify the time range for the matching packets either as an absolute value or relative to a specific time, including the present.
  • Source Info: Match a specific source IP address / MAC Address / CIDR address.
  • Bi-directional: Enabling this will query bi-directional traffic.
  • Destination Info: Match a specific destination IP address / MAC Address / CIDR address.
  • IP Protocol: Match the selected IP protocol.
  • Community ID: Flow hashing.
Additional Parameters
  • VLAN: Match the VLAN ID.
  • Outer VLAN: Match the outer VLAN ID when more than one VLAN ID exists.
  • Inner/Middle VLAN: Match the inner VLAN ID of two VLAN IDs or the middle VLAN ID of three VLAN IDs.
  • Innermost VLAN: Match innermost VLAN ID of three VLAN IDs.
  • Filter Interfaces: Match packets received at the specified DANZ Monitoring Fabric filter interfaces.
  • Policy Names: Match packets selected by the specified DANZ Monitoring Fabric policies.
  • Max Size: Set the maximum size of the query results in bytes.
  • Max Packets: Limits the number of packets returned by the query to this set value.
  • MetaWatch Device ID: Matches on device ID / serial number found in the trailer of the packet stamped by the MetaWatch Switch.
  • MetaWatch Port ID: Matches on application port ID found in the trailer of the packet stamped by the MetaWatch Switch.
  • Packet Recorders: Query a particular DMF Recorder Node. Default is deselected/none, and the query is sent to all packet recorders configured on the DANZ Monitoring Fabric.
  • Dedup: Enable/Disable Dedup.
  • Query Preview: After expanding, this section provides the Stenographer syntax that is used in the selected query. You can cut and paste the Stenographer query and include it in a REST API request to the DMF Recorder Node.

Using Recorder with Analytics

For interactive analysis, any set of packets exceeding 1 GB becomes unwieldy. To reduce the number of packets to a manageable size, complete the following steps:

  1. Use the Summary query to determine the number of packets captured by the Recorder. Apply filters until the packet set is a manageable size (less than 1 GB).
  2. Search over the metadata received from all sources and analyze to retrieve a limited and useful set of packets based on source address, destination address, timeframe, and other filtering attributes.
  3. Submit the Stenographer query, which is used by DMF Recorder Node automatically composed by Arista Analytics.

    You can perform flow analysis without downloading the packets from Recorder. Select specific rows to show Throughput, RTT, Out of order, Re-transmissions. This analysis is done on various types of packet varieties like HTTP, HTTTP request, DNS, Hosts, IPv4, IPv6, TCP, TCPFlow Health, UDP, RTP Streams, SIP Correlate, and SIP Streams. Then sort and search as required, and save to CSV for later analysis. You can search over a given duration of time for the IP address by exact match or prefix match.

    Replay lets you direct large packet sets to an archive for later analysis; this frees up the Recorder to capture a new packet set.

    You can also use DMF Recorder Node for identifying the applications on your network that are encrypting packets. Use a Recorder Detail query to see the applications with encrypted packets.

    For information about installing and setting up the DMF Recorder Node, refer to the DANZ Monitoring Fabric Deployment Guide. For details about using the Recorder from the DANZ Monitoring Fabric controller GUI or CLI, refer to the DANZ Monitoring Fabric User Guide.

Analyzing SIP and RTP for DMF Analytics

This feature describes how Session Initiation Protocol (SIP) packets are parsed in a DANZ Monitoring Fabric (DMF) Analytics Node deployment and presented in a dashboard to allow the retrieval of data packets conveying voice traffic (RTP) from the DMF Recorder Node (RN). DMF accomplishes this by showing logical call information such as the call ID, phone number, and username. After retrieving the SIP record, the associated IP addresses are used to retrieve packets from the RN and opened in Wireshark for analysis.

The dashboard is located in the dashboard mode of Kibana named SIP.
Figure 2. SIP Dashboard

DMF Preconditions

The feature requires a physical connection from the DMF Delivery Switch to the 10G Analytics Node (AN) Collector interface.
  • Policy configured to filter for SIP traffic (UDP port 5060) such that low-rate traffic (< 1Gbps) is delivered to AN via collector interface with a filter on the Layer 4 port number or UDF.
  • LAG to send SIP Control Packets to 1, 3, and 5 AN Nodes with symmetric hashing enabled and without hot-spotting.
  • Recorder Node to receive SIP and Control packets recorded with standard key fields.

Configuration

Configure SIP using the broker_address, timestamp-field and field_group to enable the feature. Refer to Field Details for more information on broker_address.

Figure 3. Edit-topic indexer

Limitations

This feature is supported in AN DMF 8.5.0 release onwards.
  • There is no toggle switch to enable or disable this feature.