Management and Control Plane Security

This chapter describes options for increasing the security of management access to the DMF Controller node.

 

Management Plane Security

The management plane network is used by the administrator, whether locally or remotely, to reach the Controller management interfaces. DANZ Monitoring Fabric (DMF) uses standard, well-known cryptographic technology, such as RSA and AES. Still, system administrators must choose strong passwords and change them frequently, according to well-established security best practices.

All services the Controller uses are enabled by default except for SNMP, which is disabled by default. Refer to the Protocol Access Required to the DMF Controller section to block or permit specific protocols to the management interface.

For example, the control plane is the network between the Controllers and the switches to carry OpenFlow control traffic. The following are general requirements and recommendations for deployment:

  • The controller must be on the same Layer 2 network as the switches—physically isolated data, control, and management plane networks.
  • The only devices on the control plane are switches and Controllers.
  • Make the control plane network not routed or minimally IP access restricted via its egress router.
  • Physically secure the management and data plane networks (for example, locks on the cage doors).

Many of the Zero-Touch Networking (ZTF) protocols (DHCP, such as ONIE, controller discovery, and image download) and the OpenFlow protocol are not authenticated. They are subject to spoofing in an untrusted network. The following are best practices regarding securing the control plane within the switched fabric.

  • The control plane network is “Layer 2 trusted,” meaning the attacker cannot spoof Layer 2 messages on the control network. In practice, this means the control plane network should be an isolated VLAN, ideally containing only the Controller and switches.
  • Harden the switch management interface against Layer 3 attacks (all services are authenticated, unnecessary services are turned off, and so forth).
  • The network should not be reachable by Layer 3 protocols. If Layer 3 access is required, the administrator should maintain a Layer 3 allowlist of hosts that can access the control network, for example, using an ACL on the edge router.

Importing the Controller Private Key and Certificate

This section describes how to import a private key and a certificate to the Controller after copying it to the Controller using the copy command.

To import a private key to the Controller, enter the private-key command in the config-controller submode: 
[no] private-key <controller-key-name>

Replace controller-key-name with the name of the private key. Use the no version of the command to remove the private-key.

To import the Controller certificate, use the certificate command in config-controller submode. 
[no] certificate <name>

Replace the name with the name assigned to the Controller certificate. Use the no version of the command to remove the certificate.

Import the private key and certificate to the Controller using the copy command.

Using Certificates Signed by a CA for GUI Access to the Controller

By default, SSL is enabled on the Controller using a self-signed certificate. Complete the following steps to install a certificate signed by a public or private CA.

Procedure

  1. Generate the Certificate Signing Request (CSR) and the private key for the Controller.
    Perform this operation on any workstation that supports OpenSSL. The following example shows the operation performed on a Linux workstation. 
    root@Ubuntu-12:~/openssl-ca/admin# openssl req -newkey rsa:2048 -nodes -keyout controller.
key -new -out controller.csr
Generating a 2048 bit RSA private key
.......................................+++
...............................+++
writing new private key to 'controller.key'
-----
You are about to be asked to enter information that will be incorporated into your
certificate request.
What you are about to enter is what is called a Distinguished Name or a DN. There are
quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:California
Locality Name (eg, city) []:Santa Clara
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Arista Networks Organizational
Unit Name (eg, section) []:Engineering
Common Name (e.g. server FQDN or YOUR name) []:DMF Secure Certificate
Email Address []:This email address is being protected from spambots. You need JavaScript enabled to view it.
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:anet1234
An optional company name []:Arista
root@Ubuntu-12:~/openssl-ca/admin#
root@Ubuntu-12:~/openssl-ca/admin# ls -ltr
total 8
-rw-r--r-- 1 root root 1708 Feb 7 15:39 controller.key
-rw-r--r-- 1 root root 1184 Feb 7 15:39 controller.csr
root@Ubuntu-12:~/openssl-ca/admin#
  2. Submit the CSR to the CA and get the certificate signed.

    Submit the CSR to the trusted CA for browsers used to access the DMF GUI. For organizations using GUI based CAs, copy the contents of the CSR to the CA for signature.

    The following example shows the operation performed on a Linux workstation. 
    root@Ubuntu-12:~/openssl-ca# openssl ca -config openssl-ca.cnf -policy signing_policy -
extensions signing_req -out admin/controller.pem -infiles admin/controller.csr
Using configuration from openssl-ca.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName :Printable:'US'
stateOrProvinceName :ASN.1 12:'California'
localityName :ASN.1 12:'Santa Clara'
organizationName :ASN.1 12:'Arista Networks'
organizationalUnitName:ASN.1 12:'Engineering'
commonName :ASN.1 12:'DMF Secure Certificate'
Certificate is to be certified until Nov 3 23:41:17 2020 GMT (1000 days) Sign the
certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new
entries
Data Base Updated
root@Ubuntu-12:~/openssl-ca#
root@Ubuntu-12:~/openssl-ca/admin# ls -ltr
total 16
-rw-r--r-- 1 root root 1708 Feb 7 15:39 controller.key
-rw-r--r-- 1 root root 1184 Feb 7 15:39 controller.csr
-rw-r--r-- 1 root root 5882 Feb 7 15:41 controller.pem
root@Ubuntu-12:~/openssl-ca/admin#
  3. Copy the signed certificate to the Controller: 
    controller-1# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it..3:/root/openssl-ca/admin/controller.pem cert://
This email address is being protected from spambots. You need JavaScript enabled to view it..3's password:
controller.pem
5.74KB - 00:00
controller-1# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it..3:/root/openssl-ca/admin/controller.key private-key:/
/controller- private.key
This email address is being protected from spambots. You need JavaScript enabled to view it..3's password:
controller.key
1.67KB - 00:00
controller-1#
  4. Verify that the certificate was copied correctly: 
    controller-1# show secure
<SNIP>
~~~~~~~~~ Cert ~~~~~~~~~
# Name
-|----------------------|
1 DMF Secure Certificate 2 QA CA
3 ovsclient
~~~~~~~~~~~ Csr ~~~~~~~~~~~
#Name
- |------------------------ |
112358.controller.cluster
232591.controller.cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Private Keys~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NameAlgorithmValue 
---------------------- |--------- |-----------------------------------------------------------------------------------------------|
controller-private.keysha256 DB:6D:C1:01:E2:CD:71:C4:AA:54:FA:6F:3F:80:4E:C7:25:4C:A9:2A:CA:7F:F5:44:CF:37:3C:C7:67:93:19:BB
ovsclient sha256 EB:88:0C:9D:EE:37:AA:BA:1A:6E:7B:F9:6E:7F:89:45:69:C4:7F:58:D3:18:D2:DC:49:16:2E:1D:2A:2B:94:89
controller-1#
  5. Apply the certificate and private key. 
    controller-1(config-controller)# certificate DMF\Secure\Certificate
controller-1(config-controller)# private-key controller-private.key
  6. Display the Controller security configuration. 
    controller-1(config-controller)# show this
! controller
controller
certificate 'DMF Secure Certificate'
cluster-name DMF_Cluster
private-key controller-private.key
access-control
!
access-list api
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list gui
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list ssh
1 permit from ::/0
2 permit from 0.0.0.0/0
controller-1(config-controller)#
  7. Access the DMF GUI using a browser and display the certificate.
  8. After connecting to the Controller, click the padlock icon to the left of the location field to display information about the certificate.

Replacing the Certificate

Please use the following steps to replace the Controller's certificate.

Scenario 1: Using the same CSR used to sign the current certificate.

Obtain a newly signed certificate from CA using the same CSR and copy it to the Controller using the following command:

# copy new certificate from the source cert://
For example: 
# copy scp://This email address is being protected from spambots. You need JavaScript enabled to view it./root/openssl-ca/certificate.pem cert:// This email address is being protected from spambots. You need JavaScript enabled to view it. password certificate.pem 
6.49KB - 00:00 
#
No other action is needed as the current certificate will be overwritten when copying the new one.

Scenario 2: Does not have the same CSR for the current certificate.

  1. Generate a new CSR and the private key.
  2. Sign the CSR to get the new certificate.
  3. Import/copy the certificate to the Controller. The current certificate will be overwritten if the Common Name matches the new one.
  4. Import/copy the new private key to the Controller. The private key will be overwritten if the file name is the same as the old one. In that case, there is no need for any config changes.
Assuming the CN and the private key dest file names are different than the original ones, remove the old cert and private key and install a new cert and private key.
To remove the old certificate and private key, use the following commands: 
C1(config)# controller
C1(config-controller)# no certificate certificate name
C1(config-controller)# no private-key private-key name
C1(config-controller)#
To configure the new certificate and private key use the following commands: 
C1(config)# controller
C1(config-controller)# certificate new certificate name
C1(config-controller)# private-key new private-key name
C1(config-controller)#

Managing the Controller HTTP and SSH Ciphers, Protocols, and Data Integrity Algorithms

Use the crypto command to enter the config-crypto submode to configure settings for HTTP and SSH. Use the http and ssh commands in the config-crypto submode to configure the ciphers and protocols. Configure the list of enabled ciphers, protocols, or algorithms by appending to the list.

Use the no version of this command with any keyword to remove the specific cipher, protocol, or algorithm. Use the no version of the command without a keyword to restore the list to the default value. Use the CLI help feature to identify the supported ciphers, protocols, or data integrity (MAC) algorithms.

Configuring HTTP Ciphers

Enter the following commands to configure the ciphers for HTTP: 
controller-1(config)# crypto
controller-1(config-crypto)# http
controller-1(config-crypto-http)# cipher <index> <cipher-name>
Note: When you configure a set of ciphers instead of using the default set, please make sure to configure at least one or all the three ciphers mentioned below in addition to your choice. ECDHE-RSA-CHACHA20-POLY1305, ECDHE-RSA-AES128-GCM-SHA256, ECDHE-RSA-AES256-GCM-SHA384.
Example: 
controller-1(config)# crypto
controller-1(config-crypto)# http
controller-1(config-crypto-http)# cipher 1 <your choice of cipher-name>
controller-1(config-crypto-http)# cipher 2 <your choice of cipher-name>
controller-1(config-crypto-http)# cipher 3 <your choice of cipher-name>
controller-1(config-crypto-http)# cipher 21 ECDHE-RSA-CHACHA20-POLY1305
controller-1(config-crypto-http)# cipher 22 ECDHE-RSA-AES128-GCM-SHA256
controller-1(config-crypto-http)# cipher 23 ECDHE-RSA-AES256-GCM-SHA384

Configuring HTTP Protocols

Starting in the DANZ Monitoring Fabric 8.4 release, the TLSv1.3 HTTPS protocol is supported. DMF supports TLSv1.3 and TLSv1.2 by default, with the TLSv1.3 protocol preferred for TLS connections.

Enter the following commands to configure the protocols for HTTP: 
controller-1(config)# crypto
controller-1(config-crypto)# http
controller-1(config-crypto-http)# protocol <index> <protocol-name>

Configuring SSH Ciphers

Configured SSH ciphers and MAC algorithms on the Controller are pushed to the switches running Switch Light OS via ZTN. With this enhancement, users can also restrict the SSH ciphers and MAC algorithms on the switches.

Enter the following commands to configure the ciphers for SSH: 
controller-1(config)# crypto
controller-1(config-crypto)# ssh
controller-1(config-crypto-ssh)# cipher <index> <cipher-name>

Configuring SSH Data Integrity Algorithms

Enter the following command to configure data integrity (MAC) algorithms for SSH: 
controller-1(config)# crypto
controller-1(config-crypto)# ssh
controller-1(config-crypto-ssh)# mac <index> <mac-name>

Changes to Supported MACs/Ciphers/SSH Keys

Please note the below changes to supported MACs/Ciphers and SSH Keys in DANZ Monitoring Fabric starting with the DMF 8.0 release due to the upgrade of Ubuntu OS from 16.04 to 20.04. Arista Networks refined the list of supported ciphers to represent better what can be configured and those default enabled when TLSv1.3 protocol is enabled.
  • The default list of SSH MACs has changed: This email address is being protected from spambots. You need JavaScript enabled to view it. and hmac-ripemd160 have been removed from the default list of SSH MACs.
  • The new default list of SSH MACs is : 
    This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
hmac-sha2-512
hmac-sha2-256
This email address is being protected from spambots. You need JavaScript enabled to view it.
hmac-sha1
  • The following SSH MACs are obsolete and no longer supported: 
    hmac-ripemd160
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
  • The following SSH ciphers are obsolete and no longer supported: 
    arcfour
arcfour128
arcfour256
blowfish-cbc
cast128-cbc
  • Changes in SSH host keys: 
    ssh_host_dsa_key is obsolete and no longer supported.
ssh_host_ed25519_key is now available, along with ssh_host_ecdsa_key
and ssh_host_rsa_key that have been available since past releases.
  • Removed SSL ciphers: 
    CAMELLIA128-SHA;
CAMELLIA256-SHA;
DES-CBC3-SHA;
DH-DSS-AES128-GCM-SHA256;
DH-DSS-AES128-SHA;
DH-DSS-AES128-SHA256;
DH-DSS-AES256-GCM-SHA384;
DH-DSS-AES256-SHA;
DH-DSS-AES256-SHA256;
DH-DSS-CAMELLIA128-SHA;
DH-DSS-CAMELLIA256-SHA;
DH-DSS-DES-CBC3-SHA;
DH-DSS-SEED-SHA;
DH-RSA-AES128-GCM-SHA256;
DH-RSA-AES128-SHA;
DH-RSA-AES128-SHA256;
DH-RSA-AES256-GCM-SHA384;
DH-RSA-AES256-SHA;
DH-RSA-AES256-SHA256;
DH-RSA-CAMELLIA128-SHA;
DH-RSA-CAMELLIA256-SHA;
DH-RSA-DES-CBC3-SHA;
DH-RSA-SEED-SHA;
DHE-DSS-AES128-GCM-SHA256;
DHE-DSS-AES128-SHA;
DHE-DSS-AES128-SHA256;
DHE-DSS-AES256-GCM-SHA384;
DHE-DSS-AES256-SHA;
DHE-DSS-AES256-SHA256;
DHE-DSS-CAMELLIA128-SHA;
DHE-DSS-CAMELLIA256-SHA;
DHE-DSS-SEED-SHA;
DHE-RSA-CAMELLIA128-SHA;
DHE-RSA-CAMELLIA256-SHA;
DHE-RSA-SEED-SHA;
ECDH-ECDSA-AES128-GCM-SHA256;
ECDH-ECDSA-AES128-SHA;
ECDH-ECDSA-AES128-SHA256;
ECDH-ECDSA-AES256-GCM-SHA384;
ECDH-ECDSA-AES256-SHA;
ECDH-ECDSA-AES256-SHA384;
ECDH-ECDSA-DES-CBC3-SHA;
ECDH-ECDSA-RC4-SHA;
ECDH-RSA-AES128-GCM-SHA256;
ECDH-RSA-AES128-SHA;
ECDH-RSA-AES128-SHA256;
ECDH-RSA-AES256-GCM-SHA384;
ECDH-RSA-AES256-SHA;
ECDH-RSA-AES256-SHA384;
ECDH-RSA-DES-CBC3-SHA;
ECDH-RSA-RC4-SHA;
ECDHE-ECDSA-DES-CBC3-SHA;
ECDHE-ECDSA-RC4-SHA;
ECDHE-RSA-DES-CBC3-SHA;
ECDHE-RSA-RC4-SHA;
EDH-DSS-DES-CBC3-SHA;
EDH-RSA-DES-CBC3-SHA;
PSK-3DES-EDE-CBC-SHA;
PSK-RC4-SHA;
RC4-MD5;
RC4-SHA;
SEED-SHA;
SRP-3DES-EDE-CBC-SHA;
SRP-DSS-3DES-EDE-CBC-SHA;
SRP-DSS-AES-128-CBC-SHA;
SRP-DSS-AES-256-CBC-SHA;
SRP-RSA-3DES-EDE-CBC-SHA;
DHE-PSK-AES128-CBC-SHA;
DHE-PSK-AES128-CBC-SHA256;
DHE-PSK-AES128-GCM-SHA256;
DHE-PSK-AES256-CBC-SHA;
DHE-PSK-AES256-CBC-SHA384;
DHE-PSK-AES256-GCM-SHA384;
DHE-PSK-CHACHA20-POLY1305;
DHE-RSA-CHACHA20-POLY1305;
ECDHE-PSK-AES128-CBC-SHA;
ECDHE-PSK-AES128-CBC-SHA256;
ECDHE-PSK-AES256-CBC-SHA;
ECDHE-PSK-AES256-CBC-SHA384;
ECDHE-PSK-CHACHA20-POLY1305;
ECDHE-RSA-CHACHA20-POLY1305;
PSK-AES128-CBC-SHA256;
PSK-AES128-GCM-SHA256;
PSK-AES256-CBC-SHA384;
PSK-AES256-GCM-SHA384;
PSK-CHACHA20-POLY1305;
RSA-PSK-AES128-CBC-SHA;
RSA-PSK-AES128-CBC-SHA256;
RSA-PSK-AES128-GCM-SHA256;
RSA-PSK-AES256-CBC-SHA;
RSA-PSK-AES256-CBC-SHA384;
RSA-PSK-AES256-GCM-SHA384;
RSA-PSK-CHACHA20-POLY1305
  • Added SSL Ciphers: 
    ECDHE-ECDSA-CHACHA20-POLY1305;
  • Conditionally Enabled Ciphers:
    Note: Enabled by default for TLSv1.3, the SSL ciphers below cannot be configured using crypto/http/ciphers configuration.
    TLS_AES_256_GCM_SHA384;
TLS_CHACHA20_POLY1305_SHA256;
TLS_AES_128_GCM_SHA256

Inherit MAC and Cipher Configuration

This feature provides the ability to mirror the SSH/HTTPS cryptographic configuration of the DMF Controller to the managed appliances (i.e., service nodes and recorder nodes) and the SSH cryptographic configuration of the Controller to the EOS switches.

Using the CLI to Configure SSH and HTTPS

The configuration that a managed appliance or EOS switch receives is intended for the Controller itself. Configuring a cipher or message authentication code (MAC) on the Controller will automatically be reflected onto a managed appliance or EOS switch.

SSH and HTTPS Cryptographic Configuration Syntax

(config)# crypto
(config-crypto)# ssh
(config-crypto-ssh)# cipher number algorithm
(config-crypto-ssh)# mac number algorithm
(config-crypto-ssh)# http
(config-crypto-http)# cipher number algorithm
(config-crypto-http)# protocol number algorithm

The following is a configuration example using common algorithms.

(config)# crypto
(config-crypto)# ssh
(config-crypto-ssh)# cipher 1 3des-cbc
(config-crypto-ssh)# mac 1 hmac-md5
(config-crypto-ssh)# http
(config-crypto-http)# cipher 1 AES128-GCM-SHA256
(config-crypto-http)# cipher 2 ECDHE-RSA-CHACHA20-POLY1305
(config-crypto-http)# protocol 2 SSLv2

Verify the Cryptographic Configuration

Check the cryptographic configuration of the Controller using the show running-config command, as shown in the example below, and verify the settings in the crypto section.

# show running-config
.
.
.
! crypto
crypto
!
ssh
cipher 1 3des-cbc
mac 1 hmac-md5
.
.
.

All ciphers/protocols/MACs of the HTTPS/SSH cryptographic configuration supported on the Controller are supported on the managed appliances, with one caveat listed in the Limitations section below. Check the HTTPS/SSH cryptographic configuration by reviewing the running-config of a managed appliance, as shown below for a Recorder Node.

# show recorder-node device rn1 running-config
.
.
.
! crypto
crypto
!
ssh
cipher 1 3des-cbc
mac 1 hmac-md5
.
.
.
Note: EOS does not support all SSH ciphers and MACs that the Controller does.
The following SSH MAC algorithms supported by the Controller are not supported by EOS:
  1. This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-MD5 in “encrypt-then-mac” mode)
  2. This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-MD5 in “encrypt-then-mac” mode)
  3. This email address is being protected from spambots. You need JavaScript enabled to view it. (HMAC-SHA1 in “encrypt-then-mac” mode)
  4. This email address is being protected from spambots. You need JavaScript enabled to view it. (message authentication code based on universal hashing (UMAC) in “encrypt-then-mac” mode)
  5. This email address is being protected from spambots. You need JavaScript enabled to view it. (UMAC)

The following SSH cipher algorithm supported by the Controller are not supported by EOS:

  1. This email address is being protected from spambots. You need JavaScript enabled to view it. (Rijndael in CBC mode)

This difference can be seen when reviewing the running-config of the Controller and the ZTN-generated running-config of an EOS switch:

# show running-config
.
.
.
.
! crypto
crypto
!
ssh
cipher 1 This email address is being protected from spambots. You need JavaScript enabled to view it.
cipher 2 3des-cbc
mac 1 This email address is being protected from spambots. You need JavaScript enabled to view it.
mac 2 This email address is being protected from spambots. You need JavaScript enabled to view it.
.
.
.
.


# show switch switch-name running-config
.
.
.
cipher 3des-cbc
mac This email address is being protected from spambots. You need JavaScript enabled to view it.
.
.
.

Only the ciphers/MACs that are supported get added to the running-config of the EOS switch. To review the disallowed MACs/ciphers when generating the running-config of the switch, use the following show command:

# show fabric warnings feature-unsupported-on-device
# NameWarning
-|-----|---------------------------------------------------------------------|
1 core1 This email address is being protected from spambots. You need JavaScript enabled to view it. is not a supported cipher on EOS switches
2 core1 This email address is being protected from spambots. You need JavaScript enabled to view it. is not a supported mac on EOS switches

Syslog Messages

No syslog messages are generated when the DMF Controller’s cryptographic configuration is mirrored to the managed appliances and EOS switches.

Limitations

  1. There are limitations to the HTTPS configuration (some options may cause ZTN protocol/communication and controller-to-controller communication failures). The following HTTPS protocol versions are required to be used to avoid communication failures:
    1. TLSv1.2
    2. TLSv1.3
  2. It is not apparent when a cipher/MAC is not reflected onto an EOS switch (due to it being unsupported). To ascertain this scenario,check the Controller's running-config and the switch's ZTN-generated running-config and compare them (alternatively, check the “show fabric warnings” command output to review any generated warnings).
  3. An ECDSA based cryptographic cipher configuration inherited by the managed appliances will cause a failure in communication with the Controller.

Protocol Access Required to the DMF Controller

Management Plane Access

You can control access to the DMF Controller for specific protocols, and in the case of SSH, you can allow access only from specific IP addresses or subnetworks. The following table summarizes the TCP/UDP protocol ports that DMF uses. The CLI access-list option column shows the options for the ports that can be enabled or disabled using the CLI access-list command (config-controller-access submode). The ports listed are open by default on the Controller, except for SNMP, which is disabled by default.

These ports must also be open on any device, such as a router or firewall, that connects the management console or application to the DMF Controller.
Protocol Port Application

CLI access-list option

Match criteria
HTTP TCP 80 GUI auto-redirect    
HTTPS TCP 443 GUI remote access gui, applicable to Controller, Service Node, Recorder Node, Analytics Node

Default any, configurable
HTTPS TCP 8443 REST API api, applicable to Controller, Service Node, Recorder Node, Analytics Node

Default any, configurable
ICMP/ICMPv6 ICMP/ICMPv6     selected ICMP types
ICMP/ICMPv6 ICMP/ICMPv6     selected ICMP types
SNMP UDP 161, 162 SNMP, applicable to Controller, Service Node, Analytics Node snmp

Default none, configurable
SSH TCP 22 CLI remote access ssh, applicable to Controller, Service Node, Recorder Node, Analytics Node

Default any, configurable
syslog UDP 514      
vce-api UDP 7443 vCenter integration vce-api Enabled by default
Note: Be careful when configuring firewall rules for the SSH protocol, which is permitted from all subnetworks by default. The option exists to restrict SSH access to one or more specific subnetworks. However, this denies access from all other subnetworks. If no connectivity from a specified subnetwork is available, accessing the Controller is only through the local console.

Control Plane Access for DMF Controller

The following ports must be open between the DMF Controller and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if any DMF nodes or fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Controller.
Table 1. DMF Controller
Protocol Port Direction Application In Flows Out Flows
TCP 22 Both Direction SSH Customer

Switches,

managed appliances
TCP 49 Out TACACS+  

Customer

TACACS+ server
TCP 53 Out DNS   Customer DNS server
UDP 53 Out DNS   Customer DNS
UDP 67 Out DHCP   Customer DHCP server
UDP 68 In DHCP Customer DHCP server  
TCP 80 In ZTN ONIE DMF Switches  
UDP 123 Both Direction NTP

Switches, Service Node,

Recorder Node, Analytics Node

 Customer NTP server
UDP 161 In SNMP Customer  
UDP 162 Out SNMP Traps   Customer
TCP 443 In GUI Customer  
UDP 514 Out Syslog   Customer Syslog server
UDP 1813 Out RADIUS  

Default RADIUS accounting port
UDP 5353 In ZTN MDNS

Switches, Service Node,

Recorder Node

  
TCP 6379 Out Controller Stats   Analytics Nodes
TCP 6642 Both Direction Cluster Sync Controller HA Controller HA
TCP 6653 In OpenFlow

Switches, Recorder Node,

Service Node

  
TCP 7443 In VCE API vCenter API  
TCP 8443 Both Direction

Floodlight

REST API

 Customer, Recorder Node

Recorder Node, Service Node
TCP 8843 In ZTN

Switches, Service Node,

Recorder Node

  
TCP 9379 Out

AN Replicated Redis

   Analytics Node
To enable SNMP access to the Controller or to restrict access to the Controller for the REST API, web-based GUI access, or SSH applications, complete the following steps.

Procedure

  1. Enter the controller command from config mode to enter config-controller submode. 
    controller-1(config)# controller
controller-1(config-controller)#
  2. Enter the access-control command from config-controller submode. 
    controller-1(config-controller)# access-control
controller-1(config-controller-access)#
  3. Enter the access-list command from config-controller-access submode followed by the protocol for which you want to configure a rule.
    The protocols for which you can configure rules include the following:
    • api: Enter the config-controller-access-list submode for REST/API access to the Controller.
    • gui: Enter the config-controller-access-list submode for web-based GUI access to the Controller.
    • ns-api: Enter the config-controller-access-list submode to manage NS-API access to the Controller, for example, from OpenStack or vCenter.
    • ssh: Enter the config-controller-access-list submode for SSH access to the Controller.
    • snmp: Enter the config-controller-access-list submode for SNMP access to the Controller.

    By default, the access list for all services except for SNMP is 0.0.0.0/0, which allows access from any IPv4 subnetwork and ::/0, which allows access from any IPv6 subnetwork. For SNMP, the access list is empty, meaning access is not permitted unless specifically enabled. SNMP UDP port 161 is blocked on Controllers and fabric switches by default. You must configure an SNMP access-list using the controller CLI to communicate using SNMP.

    For example, the following command enters config-controller-access-list submode for the SSH protocol: 
    controller-1(config-controller-access)# access-list ssh
controller-1(config-controller-access-list)#
    When configuring the following access-list on the Controller, the ACL is pushed to all connected switches, and the SNMP ACL is applied to each switch ma1 management interface. 
    controller-1(config-controller-access)# access-list snmp
controller-1(config-controller-access-list)# 1 permit from ::/0
controller-1(config-controller-access-list)# 2 permit from 10.0.0.0/8
  4. Specify the subnetworks from which access is permitted for the specified protocol. Specify the subnetwork followed by a slash and the number of bits in the subnet mask.
    For example, the following commands allow access for the SSH protocol only from subnetwork 192.168.1.0: 
    controller-1(config-controller-access)# access-list
ssh controller-1(config-controller-access-list)# 10 permit from 192.168.1.0/24
  5. To view the current firewall configuration for the Controller, enter the show running-config command and see the access-control section.

Protocol Access Required to the DMF Controller - Sync

Sync has been added to the access-list. All traffic is permitted for IPv4 (0.0.0.0/0) and IPv6 (::/0) by default. If the Active standby Controller HA pair is deployed in different L3 subnets, permitting all traffic for sync service can be a security risk. For a secure connection between the active and standby Controller, the access-list for sync should only permit the active and standby Controller-management IP addresses.

Procedure

  1. Add a rule to permit active Controller management IP address. Do not overwrite the existing default rule for sync. Use a different rule number when adding a new rule for sync access-list. The example below shows how to configure Rule ID 3 (3 permit from 10.240.130.17/32). 
    DMF-CTL2(config)# show controller access-control access-list
# Access-listRuleActionSource
-- |----------- |---- |------ |--------- |
1api1 permit::/0
2api2 permit0.0.0.0/0
3gui1 permit::/0
4gui2 permit0.0.0.0/0
5ntp1 permit::/0
6ntp2 permit0.0.0.0/0
7snmp 1 permit0.0.0.0/0
8ssh1 permit::/0
9ssh2 permit0.0.0.0/0
10 sync 1 permit::/0
11 sync 2 permit0.0.0.0/0
12 vce-api1 permit::/0
13 vce-api2 permit0.0.0.0/0
DMF-CTL2(config-controller-access-list)# controller
DMF-CTL2(config-controller)# access-control
DMF-CTL2(config-controller-access)# access-list sync
DMF-CTL2(config-controller-access-list)# 3 permit from 10.240.130.17/32
DMF-CTL2(config-controller-access-list)#
  2. Add another rule to permit standby Controller management IP address. Do not overwrite the existing default rule for sync. Use a different rule number when adding a new rule for sync access-list. The example below shows how to configure Rule ID 4 (4 permit from 10.240.130.16/32). 
    DMF-CTL2(config-controller-access-list)# 4 permit from 10.240.130.16/32
DMF-CTL2(config-controller-access-list)# show controller access-control access-list
#Access-listRuleActionSource
-- |----------- |---- |------ |----------------|
1api1 permit::/0
2api2 permit0.0.0.0/0
3gui1 permit::/0
4gui2 permit0.0.0.0/0
5ntp1 permit::/0
6ntp2 permit0.0.0.0/0
7snmp 1 permit0.0.0.0/0
8ssh1 permit::/0
9ssh2 permit0.0.0.0/0
10 sync 1 permit::/0
11 sync 2 permit0.0.0.0/0
12 sync 3 permit10.240.130.17/32
13 sync 4 permit10.240.130.16/32
14 vce-api1 permit::/0
15 vce-api2 permit0.0.0.0/0
DMF-CTL2(config-controller-access-list)#
  3. Remove the default permit entry. 
    DMF-CTL2(config-controller-access-list)# no 1 permit
DMF-CTL2(config-controller-access-list)# no 2 permit
DMF-CTL2(config-controller-access-list)# show controller access-control access-list
#Access-listRuleActionSource
-- |----------- |---- |------ |---------------- |
1api1 permit::/0
2api2 permit0.0.0.0/0
3gui1 permit::/0
4gui2 permit0.0.0.0/0
5ntp1 permit::/0
6ntp2 permit0.0.0.0/0
7snmp 1 permit0.0.0.0/0
8ssh1 permit::/0
9ssh2 permit0.0.0.0/0
10 sync 3 permit10.240.130.17/32
11 sync 4 permit10.240.130.16/32
12 vce-api1 permit::/0
13 vce-api2 permit0.0.0.0/0
DMF-CTL2(config-controller-access-list)#
  4. Verify cluster state using show controller details and make sure cluster state is redundant. 
    DMF-CTL2(config)# show controller details
Cluster Name : DMF-7050
Cluster UID : a5de38214971de42aa7b51b96ac7345f4f228b20
Cluster Virtual IP : 10.240.130.18
Redundancy Status : redundant
Redundancy Description : Cluster is Redundant
Last Role Change Time : 2022-11-05 00:56:04.862000 UTC
Cluster Uptime : 2 months, 1 week
# IPHostname @ Node Id Domain Id State StatusUptime
-|-------------|--------|-|-------|---------|-------|---------|---------------|
1 10.240.130.17 DMF-CTL2 * 22049 1 activeconnected 2 weeks, 2 days
2 10.240.130.16 DMF-CTL1 27671 1 standby connected 2 weeks, 2 days
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Failover History ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# New Active Time completed NodeReasonDescription
-|----------|------------------------------|-----|---------------------|-------------------------------------------------------|
1 220492022-11-05 00:55:35.994000 UTC 22049 cluster-config-change Changed connection state: cluster configuration changed
DMF-CTL2(config)#

    Removing default rules before adding the new sync access-list rules can break Controller cluster communication. To recover from this, please refer to Controller Cluster Recovery.

Control Plane Access for DMF Switches

The following ports must be open for DMF Switches to communicate with DMF Controller, Analytics Node, and other services (eg. NTP, DHCP, etc.). No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if any DMF Controller and fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Controller.
Table 2. DMF Switch
Protocol Port Direction Application In Flows Out Flows
TCP 22 In SSH Customer, DMF Controller  
TCP/UDP 53 Out DNS   Customer DNS Server
UDP 67 Out DHCP   Customer DHCP Server
UDP 68 In DHCP Customer DHCP Server  
UDP 123 Out NTP   Customer NTP Server
UDP 161 In SNMP Customer  
UDP 162 Out SNMP Trap   Customer
UDP 514 Out Syslog   Customer Syslog Server
UDP 5353 Out ZTN MDNS   DMF Controller
UDP 6343 Out sFlow   Analytics Node
UDP 6380 Out Control Packets   Analytics Nodes
TCP 6653 Out OpenFlow   DMF Controller
TCP 8843 Out ZTN   DMF Controller

Control Plane Access for DMF Service Node

The following ports must be open for the DMF Service Node to communicate with the DMF Controller, Analytics Node, and other services (eg. NTP, DHCP, etc.). No further configuration is required if all devices are in the same Layer 2 network as the DMF Controller. However, if the DMF Controller and Service Nodes connect over a Layer 3 network, these ports must be open on any firewalls or routers.

Table 3. DMF Service Node
Protocol Port Direction Application In Flows Out Flows
TCP 22 In SSH Customer, DMF Controller  
TCP 49 Out TACACS+  

Customer TACACS+

Server
TCP/UDP 53 Out DNS   Customer DNS Server
UDP 67 Out DHCP   Customer DHCP Server
UDP 68 In DHCP Customer DHCP  
UDP 123 Out NTP   Customer NTP Server
UDP 161 In SNMP Customer  
UDP 162 Out SNMP Trap  

Customer SNMP Trap

Server
UDP 514 Out Syslog   Customer Syslog Server
UDP 1812 Out

Default RADIUS

Authentication port

   Customer RADIUS Server
UDP 1813 Out

Default RADIUS

Accounting port

   Customer RADIUS Server
UDP 5353 Out ZTN MDNS   DMF Controller
TCP 6653 Out OpenFlow   DMF Controller
TCP 8443 Both Direction

Floodlight REST

API

 DMF Controller DMF Controller
TCP 8843 Out ZTN   DMF Controller

Control Plane Access for DMF Recorder Node

The following ports must be open between the DMF Recorder Node and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the DMF Recorder Node. However, if the DMF Controller, Analytics Node, or fabric switches connect over a Layer 3 network, these ports must be open on any firewalls or routers that connect the devices to the DMF Recorder Node.

Table 4. DMF Recorder Node
Protocol Port Direction Application In Flows Out Flows
TCP 22 In SSH Customer, DMF Controller  
TCP 49 Out TACACS+  

Customer TACACS+

Server
TCP/UDP 53 Out DNS   Customer DNS Server
UDP 67 Out DHCP   Customer DHCP Server
UDP 68 In DHCP Customer DHCP Server  
UDP 123 Out NTP   Customer NTP Server
UDP 161 In SNMP Customer  
UDP 162 Out SNMP Trap  

Customer SNMP Trap

Server
TCP 443 In

Stenographer

Query API

 Customer, DMF Controller  
UDP 514 Out Syslog   Customer Syslog Server
UDP 1812 Out

Default RADIUS

Authentication port

   Customer RADIUS Server
UDP 1813 Out

Default RADIUS

Accounting port

   Customer RADIUS Server
TCP 2049 Both NFS   Customer NSF Server
UDP 2049 Both NFS   Customer NFS Server
UDP 5353 Out ZTN MDNS   DMF Controller
TCP 6653 Out OpenFlow   DMF Controller
TCP 8443 Both Direction

Floodlight REST

API

 DMF Controller DMF Controller
TCP 8843 Out ZTN   DMF Controller

Control Plane Access for Analytics Node

The following ports must be open between the Analytics Node and any connected devices. No further configuration is required if all devices are in the same Layer 2 network as the Analytics Node. However, if the Analytics Node connects over a Layer 3 network, these ports must be open on any firewall or router.

Table 5. DMF Analytics Node
Protocol Port Direction Application In Flows Out Flows
TCP 22 In SSH Customer  
TCP 25   SMTP  

Analytics Nodes to Mail

Server
TCP 49 Out TACACS+  

Customer TACACS+

Server
TCP/UDP 53 Out DNS   Customer DNS Server
UDP 67 Out DHCP   Customer DHCP Server
UDP 68 In DHCP Customer DHCP Server  
UDP 123 Out NTP   Customer NTP Server
UDP 161 In SNMP Customer  
UDP 161 In SNMP

from Analytics Nodes to

DMF switch

  
UDP 162 Out SNMP Trap   Customer
UDP 162 Out SNMP Trap  

from Analytics Nodes to

DMF switch
TCP 443 In GUI Customer  
TCP 467   SMTP   Analytics to Mail Server
UDP 514 Out Syslog   Customer Syslog Server
UDP 1812 Out

Default RADIUS

Authentication port

   Customer RADIUS Server
UDP 1813 Out

Default RADIUS

Accounting port

   Customer RADIUS Server
UDP 2055 In NetFlow v5

DMF Service Nodes and

Switches

  
UDP 4739 In

IPFIX & NetFlow

v9

DMF Service Nodes and

Switches

  
TCP 5043 Both Direction Active Directory

Customer Active Directory

Server

Customer Active Directory

Server
UDP 6343 In sFlow DMF Switches  
TCP 6379 Both Direction Controller Stats

Controller to Analytics

VIP

  
UDP 6380 In Control Packets DMF Switches  
TCP 6642 Both Direction

Analytics Cluster

sync

 HA controller HA controller
TCP 8443 Both Direction

Floodlight REST

API

 Customer Managed Appliances
TCP 9379 Both Direction Replicated Redis

DMF Controller to Analytics Node VIP

  
TCP 9379 Out

Analytics Node

Replicated Re- dis Server (for dpid.port -> Filter Name)

   Analytics Node