DMF Recorder Node REST API

The REST server is available over HTTPS on the default port (443) using either of the two authentication methods supported:
  • HTTP basic: The client presents a valid username and password for the Controller connected with the Recorder. The DANZ Monitoring Fabric (DMF) Recorder Node verifies at the DMF Controller if the provided username and password are valid and have sufficient privileges to use the Recorder REST API.
  • Authentication tokens: The DMF Recorder Node REST API accepts revocable authentication tokens as an alternative to HTTP basic. Valid authentication tokens are configured in the Controller and pushed down to the DMF Recorder Node using a gentable. Any client with a valid authentication token can query the DMF Recorder Node REST API without real-time consultation with the Controller.

Some APIs accept a Stenographer query string as input or return a Stenographer query string as output. A Stenographer query string is a BPF-like syntax for defining the scope of a query. Packets that match this scope are included in the query result or operation. For details about the Stenographer query syntax supported by the DMF Recorder Node, refer to the Stenographer Reference for DMF Recorder Node. The DMF Recorder Node provides a REST API so clients can look up packets and metadata. The REST server runs securely (HTTPS) on TCP port 443.

Authentication

Clients must either authenticate using a valid DANZ Monitoring Fabric (DMF) Controller username and password over HTTP Basic or with an authentication token configured on the DMF controller specifically for DMF Recorder Node REST API authentication.

Basic HTTP Authentication

Use a valid DANZ Monitoring Fabric (DMF) Controller username and password to authenticate with a DMF Recorder Node over its REST API. The recorder node delegates authentication to the DMF Controller. If the username and password provided are valid, the recorder node authorizes the user for the invoked recorder node REST endpoint.

In the following example, the query uses the HTTP Basic method of authentication:
$ curl https://1.2.3.4/query/window -u admin:12345 -k | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 79 100 79 0 0 143 0 --:--:-- --:--:-- --:--:-- 143
{
"begin": "2019-01-23 15:15:23 +0000 UTC",
"end": "2019-02-04 17:39:52 +0000 UTC"
}

In this example, the recorder node IP address is 1.2.3.4. The username on the DMF Controller is admin, and the password is 12345.

Authentication with an Authentication Token

An authentication token is primarily designed for third-party applications and automation scripts where creating an account or storing a username and password is undesirable. This method can also allow a DANZ Monitoring Fabric (DMF) Recorder Node access if the management network connection to the Controller is disrupted.

To create an authentication token, log in to the DMF Controller associated with the recorder node, then complete the following steps:

  1. Change to config mode on the active DMF Controller.
    controller-1# configure
    controller-1(config)#
  2. Define the authentication token using a unique name.
    controller-1(config)# recorder-node auth token my-token
    Auth : my-token
    Token : the-secret-token
    Note: This name does not need to be secret. This example uses the name my-token.
The Controller generates a secret token for you (in this example, the-secret-token). Treat this token as private. Anyone who presents it to the DMF Recorder Node can use the DMF Recorder Node REST APIs.
Note: Only the non-reversible hash of this token is stored on the DMF Recorder Node and Controller. There is no way to recover the token if it is lost. (See below for how to revoke the token if it is lost or compromised.)
The Controller stores the token hash and the name assigned, viewable by entering the show running-config recorder-node command, as in the following example:
controller-1(config)# show running-config recorder-node auth token
! recorder-node
recorder-node auth token my-token $2a$12$pXm62tl5rMD8c4vSrzU6X.DTjeoBmRUwZvTkvNXatsZ8TFb4PxanC
If the token is lost or compromised, remove it from the Controller, and the Controller will fail any attempt to authenticate to the recorder using the token.
controller-1(config)# no recorder-node auth token my-token
controller-1(config)# show running-config recorder-node auth token
controller-1(config)#
The following example shows a query using the authentication token method. The authentication token is defined in the HTTP request as the value of the cookie header.
$ curl https://1.2.3.4/query/inventory/window --header "Cookie:plaintext-secret-auth-token" -k |
python
-m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 79 100 79 0 0 83 0 --:--:-- --:--:-- --:--:-- 83
{
"begin": "2019-01-23 15:15:23 +0000 UTC",
"end": "2019-02-04 17:50:46 +0000 UTC"
}

In this example, the DMF Recorder Node IP address is 1.2.3.4. The authentication token has already been generated on the DMF controller associated with the recorder node and is included in the cookie header as plaintext-secret-auth-token.

You must include the plaintext authentication token, not the token hash, saved in the controller running configuration. If you do not know the plaintext token, revoke access for the token and generate a new one. Note the plaintext value displayed after generating the new token.

DMF Recorder Node API Headers

The supported REST API HTTP header entries are listed in the following table.

Table 1. DANZ Monitoring Fabric (DMF) Recorder Node REST API HTTP Headers
Header Value Type Description
Steno-Limit-Bytes:value integer max number of bytes to accept in a packet query response
Steno-Limit-Packets:value integer max number of packets to accept in a packet query response
Cookie:value string auth token to use in lieu of HTTP basic auth

DMF Recorder Node REST APIs

The supported DANZ Monitoring Fabric (DMF) Recorder Node REST APIs are listed below.

Ready

/ready

  • Description: Is the DANZ Monitoring Fabric (DMF) Recorder Node able to accept queries? Return payload indicates progress towards start up completion.
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type:
  • Return Payload:
    {
    "current-value": <int>,
    "max-value": <int>,
    "percent-complete": <float>
    }
  • Return Status Code:
    • 200, ready
    • 503, not ready

Query Window

/query/window

  • Description: Get timestamp of oldest and newest packet available for query.
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "begin" : <RFC-3339>,
    "end" : <RFC-3339>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Size

/query/size

  • Description: Get count and aggregate size of packets matching provided filter.
  • HTTP Method: POST
  • Request Payload: Stenographer query string
  • Return MIME Type: application/json
  • Return Payload:
    {
    "packet-count" : <int>,
    "aggregate-size" : <int>
    }
  • Return Status Code:
    • 200, success 400, input error
    • 500, internal error
    • 503, not ready

Query Application

/query/application

  • Description: Perform DPI on packets matching provided filter. DPI is performed using nDPI.
  • HTTP Method: POST
  • Request Payload: Stenographer query string
  • Return MIME Type: application/json
  • Return Payload: Defined by nDPI
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Packet

/query/packet

  • Description: Download pcap of packets matching provided filter.
  • HTTP Method: POST
  • Request Payload: Stenographer query string
  • Return MIME Type: application/vnd.tcpdump.pcap
  • Return Payload: .pcap file
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Analysis Filter Stenographer Analysis Type

/query/analysis[filter="<stenographer-query-string>"][type ="<analysis-type>"]

  • Description: Perform an analysis on the packets matching the stenographer query string. Supported values for analysis-type are:
    • analysis_http_tree
    • analysis_http_stat
    • analysis_http_req_tree
    • analysis_http_srv_tree
    • analysis_dns_tree
    • analysis_hosts
    • analysis_conv_ipv4
    • analysis_conv_ipv6
    • analysis_conv_tcp
    • analysis_conv_udp
    • analysis_rtp_streams
    • analysis_sip_stat
    • analysis_conv_sip
    • analysis_tcp_packets
    • analysis_tcp_flow_health
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload: Determined by the analysis type selected.
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Replay Request Stenographer Real-time Boolean

/query/replay/request[filter="<stenographer-query-string>"][real-time="<boolean>"]

  • Description: Asynchronously request packets matching filter be replayed into the monitoring fabric. Replay is performed using tcp replay.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "id" : <int>,
    "message": <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Replay Request Filter Stenographer Integer

/query/replay/request[filter="<stenographer-query-string>"][speed-mbps="<int>"]

  • Description: Asynchronously request packets matching filter be replayed into the monitoring fabric. Replay is performed using tcp replay.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "id" : <int>,
    "message": <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Query Replay Done

/query/replay/done

  • Description: Check the status of a replay matching the provided ID. Message contains replay result from tcp replay.
  • HTTP Method: POST
  • Request Payload: Replay ID
  • Return MIME Type: application/json
  • Return Payload:
    {
    "id" : <int>,
    "done" : <boolean>,
    "message": <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 404, replay ID unknown
    • 406, replay not done
    • 500, internal error
    • 503, not ready

Erase Packet Filter Stenographer Query String

/erase/packet[filter="<stenographer-query-string>"]

  • Description: Erase packets matching the provided filter. Note that any packet not matching the filter but in the same packet file of a packet matching the filter will also be deleted.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "bytes-erased" : <int>,
    "message" : <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Event Update Trigger Boolean Name Integer

/event/update[trigger="<boolean>"][name="<string>"][pre-buffer-minutes="<int>"]

  • Description: Trigger or terminate the named event. Set pre-buffer-minutes to 0 to use the available pre-buffer.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "message" : <string>,
    "event-queued" : <boolean>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Abort Query

/abort/query

  • Description: Terminate a particular query defined by the provided Stenographer query string.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "message" : <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Abort All Query

/abort-all/query

  • Description: Terminate all running queries.
  • HTTP Method: POST
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "message" : <string>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Queries

/queries

  • Description: Determine the currently running queries, enumerated by the Stenographer query string of the query.
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "queries" : [
    <stenographer-query-string>, ...
    ]
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Status Query

/status/query

  • Description: Determine how far a given query has progressed. This can be used to estimate the time remaining to run the query.
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "query" : <stenographer-query-string>, "current-value" : <int>,
    "max-value" : <int>, "percent-complete" : <float>
    }
  • Return Status Code:
    • 200, success
    • 400, input error
    • 500, internal error
    • 503, not ready

Status All

/status/all

  • Description: Determine how far all queries have progressed. This can be used to estimate the time remaining to run the queries.
  • HTTP Method: GET
  • Request Payload:
  • Return MIME Type: application/json
  • Return Payload:
    {
    "queries" : [
    {
    "query" : <stenographer-query-string>,
    "current-value" : <int>,
    "max-value" : <int>,
    "percent-complete" : <float>
    },
    ...
    ]
    }
    • Return Status Code:
      • 200, success
      • 400, input error
      • 500, internal error
      • 503, not ready