31 Jan 2020 by Simon Greaves
Having recently passed my NSX-T VCP, I thought I’d share the command line tools I used to practice in a lab to get to grips with NSX-T installation, configuration, managing and troubleshooting.
I know I will be revisiting this site for reference in future. I hope you all enjoy it!
I will cover a walkthrough of some of these commands in another post if it’s new to you and link it here.
Setup Troubleshooting
| Description | Commands |
|---|---|
| Query Management Cluster Status | get cluster status [verbose] |
| View the cluster configuration and which node is running which cluster component | get cluster config |
| Detach a NSX Manager node from Cluster: | detach node <node-id> |
| Retrieve NSX Manager Certificate Thumbprint | get certificate api thumbprint |
| Join NSX Manager to Cluster | join NSX-Manager-ip-address cluster-id <cluster-id> thumbprint <thumbprint> username <username> password <password> |
| Disable CLI timeout | set cli-timeout 0 |
| List the transport nodes registered with NSX Manager | get nodes |
| Query the Managers Connection Status | get managers |
| List all ESXi hosts to get the transport Node UUIDS | get transport-nodes status |
| List the transport Node Status | get transport-node <uuid> status |
| List the Transport Node vtep information | get transport-node <uuid> vtep |
| Lists the VIF UUID of a VM connected to Segment on a Transport Node | get transport-node <uuid> vifs |
| Set logging level on NSX Manager | set service manager logging-level debug |
| Configure a remote syslog server | set logging-server <hostname-or-ip-address[:port]> proto <protocol> level <level> |
| Read the Policy Manager log | get log-file policy.log <follow> |
| Read the syslog log | get log-file syslog <follow> |
| Setup packet capture | start capture interface <interface-name> [file <filename>] [count <packet-count>] [expression <expression>] |
| Enter root privileged Mode | st en |
Logical Switching
| Description | Commands |
|---|---|
| List all the Logical Switches | get logical-switches |
| List all the switch ports connected to the Segment | get logical-switch <uuid> ports |
| List information about a Segment | get logical-switch <vni-or-uuid> |
| List the ARP table of a Logical Switch | get logical-switch <vni-or-uuid> arp-table |
| List the MAC table of a Logical Switch | get logical-switch <vni-or-uuid> mac-table |
| List the statistics of a Logical Switch | get logical-switch <vni-or-uuid> stats |
| List the Transport Node table of a Segment | get logical-switch <vni-or-uuid> transport-node-table |
| List the VTEP table of a Segment | get logical-switch <vni-or-uuid> vtep |
| View the Logical Switch Port information | get logical-switch-port <uuid> |
| List the logical Switches statistics | get logical-switches stats |
Logical Routing
| Description | Commands |
|---|---|
| View the list of logical routers | get logical-router |
| View the information about a logical router | get logical-router <uuid> |
| View the list of logical router interfaces | get logical-router <uuid> interfaces |
| View the logical router interface information | get logical-router <uuid> interface <interface-id> |
| View the Routers on a logical router | get logical-router <uuid> route |
| List the NSX Edge nodes registered with NSX Manager and their associated controller | get transport-node status |
Firewall
| Description | Commands |
|---|---|
| View the Rule count of L2, L3 Firewall Rules | get firewall summary |
| List of firewall entities in the excluded-list | get firewall exclude-list |
| Firewall Section that is not created or deleted completely from the system | get firewall orphaned-section |
| Firewall rules published to CCP | get firewall published-entity |
| Firewall Status | get firewall status |
User Account Administration
| Description | Commands |
|---|---|
| Change local user password | Set user <username> [password <password> [old-password <old-password>] |
| Password length | Set auth-policy minimum-password-length <password-length> |
| UI and API authentication policies | |
set auth-policy api lockout-period <lockout-period> |
|
set auth-policy api lockout-reset-period <lockout-reset-period> |
|
set auth-policy api max-auth-failures <auth-failures> |
|
| Set CLI authentication policy | set auth-policy cli lockout-period lockout-period <lockout-period> |
Setup Troubleshooting
| Description | Commands |
|---|---|
| List the VIBs loaded on ESXi | esxcli software vib list | grep -e nsx -e vsip |
| List all the NSX-T modules currently loaded in the system | esxcli system module list | grep nsx |
| Check the User world agents (UWA): | |
nsx-mpa /etc/init.d/nsx-mpa status | start | stop | restart |
|
nsx-proxy /etc/init.d/nsx-proxy status | start | stop | restart |
|
nsx-opsagent /etc/init.d/nsx-opsagent status | start | stop | restart |
|
nsxa /etc/init.d/nsxastatus | start | stop | restart |
|
| Check UWA Connection: | |
| Port 1235 to Controllers | esxcli network ip connection list | grep 1235 |
| Port 5671 to NSX Manager | esxcli network ip connection list | grep 5671 |
| List Physical NICs/vmnic | esxcli network nic list |
| Physical NIC details | esxcli network nic get -n vmnic3 |
| List vmk NICs with IP addresses/MAC/MTU and so on (vmk10 is TEP, vmk50 is containers) | esxcli network ip interface ipv4 get |
| Details of each vmk NIC, including vDS information | esxcli network ip interface list |
| Details of netstack IP Stack created on ESXi | esxcli network ipinterface list --netstack=vxlan |
| Ping from a VXLAN TCP/IP Stack | vmkping ++netstack=vxlan <host-IP> -s <packet-size> |
| View routing table of VXLAN-dedicated TCP/IP stack | esxcli network ip route ipv4 list -N vxlan |
| View ARP table of VXLAN dedicated TCP/IP stack | esxcli network ip neighbor list -N vxlan |
| Setup syslog: | |
| 1. | esxcli network firewall ruleset set -r syslog -e true |
| 2. | esxcli system syslog config set --loghost=<hostname-or-ip-address[:port]> |
| 3. | esxcli system syslog reload |
Logical Switching
| Description | Commands |
|---|---|
| View all the logical switches | get logical-switches |
| View the Logical Switch information from ESXi host | get logical-switch <logical-switch-id> |
| View the ARP table of a logical switch | get logical-switch <logical-switch-id> arp-table |
| View the MAC table of a logical switch | get logical-switch <logical-switch-id> mac-table |
| View the Neighbor Discovery (ND) table of a logical switch | get logical-switch <logical-switch-id> nd-table |
| View the VTEP table of a logical Switch | get logical-switch <logical-switch-id> vtep-table |
| View the logical switch port status | get logical-switch-port status |
| View the MAC, ARP, VTEP tables from local or remote host using VNI | get logical-switch [local | remote] [mac-cache | arp-cache | vtep-cache] <vni> |
| Verify the Transport Node Tunnel Status | get host-switch <host-switch-name> tunnels |
| To view the Switch port ID from root mode | net-stats -l |
| To view the Switches configured on ESXi | esxcfg-vswitch -l |
| Performance monitoring tool on ESXi | esxtop |
| View the VTEP and VNI Configuration | net-vdl2 -l |
| To view the N-VDS Uplink Configuration | net-vdr -C -l |
| View the Logical Routers from ESXi | net-vdr -I -l |
| Verify VXLAN kernel module vdl2 is loaded | esxcli system module get -m nsxt-vdl2 |
| Setup packet capture | pktcap-uw [-o <filename.pcap>] |
| View captured packets | tcpdump-uw |
| Capture packets and display live output on screen (-dir 1 is outgoing traffic, -dir 0 is incoming traffic) | pktcap-uw --switchport <VM-Switch-Port-Number> [--dir <1 | 2>] | tcpdump-uw |
| Find VM Switch Port Number | esxtop <n> |
| Packet Capture Target Options | pktcap-uw |
| PreDVFilter | --capture PreDVFilter --dvfilter <dvfilter-Name> |
| PostDVFilter | --capture PostDVFilter --dvfilter <dvFilter-Name> |
| Source VM Switch Port | --switchport <VM-Switchport-Number> --dir 0 |
| Leaving the VNI Port | --switchport <VM-Switch-Port-Number> --dir 1 --vni=<Switch-VNI-Number> |
| Leaving vdrPort | --switchport <VM-Switch-Port> --dir 0 |
| Outgoing encapsulated overlay traffic | ---uplink <vmnic#> --dir 1 --overlay geneve |
| Incoming encapsulated | --uplink <vmnic#> --dir 0 --overlay geneve |
| Arriving at destination VM Switch Port | --switchport <VM-Switchport-Number> --dir 0 |
| PostDVFilter | --capture PostDVFilter --dvfilter <dvFilter-Name> |
| PreDVFilter | --capture PreDVFilter --dvfilter <dvfilter-Name> |
Logical Routing
| Description | Commands |
|---|---|
| View the Logical Router Forwarding table | get logical-router <UUID> forwarding |
| View the Logical Router Interface | get logical-router <UUID> interface |
| View the Logical Router Interfaces | get logical-router <UUID> interfaces |
| View the logical Router Neighbour | get logical-router <UUID> neighbor |
| View the logical Router Neighbours | get logical-router <UUID> neighbors |
Firewall
| Description | Commands |
|---|---|
| View the Firewall Status | get firewall status |
| View the Firewall Rules applied at the VIF with Address Sets | get firewall <vif_uuid> addrsets |
get firewall <vif_uuid> profile |
|
| List all the VMs dvFilter Names | summarize-dvfilter |
| List all the VMs dvFilter Names associated with a VM and limit the response to 16 lines | Summarize-dvfilter | grep -A16 <SERVER_NAME> |
| List the Firewall Rules applied on DvFilter | vsipioctl getrules -f <filtername> |
| View the Firewall Configuration for a given dvFilter name | vsipioctl getfwconfig -f <dvfilter-name> |
| View the DVSPort ID and MAC address associated with a VM | nsxdp-cli -c get ports | egrep -A1 "<VM_NAME|MAC" |
| View the number of packets dropped on a host switch | nsxdp-cli swsec get stats -dvs <Overlay_N-VDS> --dport <DVSPort-UUID> |
Setup Troubleshooting
| Description | Commands |
|---|---|
| List the VIBs loaded on Ubuntu KVM | sudo dpkg --list | grep nsx |
| List the VIBs loaded on RedHat KVM | rpm -qa | grep nsx |
| Check the User world agents (UWA): | |
nsx-mpa /etc/init.d/nsx-mpa status | start | stop | restart |
|
nsx-proxy /etc/init.d/nsx-proxy status | start | stop | restart |
|
nsx-opsagent /etc/init.d/nsx-opsagent status | start | stop | restart |
|
nsxa /etc/init.d/nsxa status | start | stop | restart |
|
| Check UWAs Connection: | |
| Port 1235 to Controllers | lsof-i -P -n | grep 1235 [or netstat -an | grep 1235] |
| Port 5671 to NSX Manager | lsof-i -P -n | grep 5671 [or netstat -an | grep 5671] |
| NIC Details | ifconfig [-a] |
lspci |
|
| Setup syslog: | |
| Login as root and create this file | /etc/rsyslog.d/40-vmware-remote-logging.conf |
| Add this line to the file | '*.*@syslog_server_ip:514;RFC5424fmt' |
| Restart syslog | Systemctl restart rsyslog |
Logical Switching
| Description | Commands |
|---|---|
| Query the NSX Controllers | get controllers |
| View all the logical switches | get logical-switches |
| View the Logical Switch information from ESXi host | get logical-switch <logical-switch-id> |
| View the ARP table of a logical switch | get logical-switch <logical-switch-id> arp-table |
| View the MAC table of a logical switch | get logical-switch <logical-switch-id> mac-table |
| View the VTEP table of a logical Switch | get logical-switch <logical-switch-id> vtep-table |
| View the ports on a logical Switch | get logical-switch <logical-switch-id> ports |
| View the MAC, ARP, VTEP tables from local or remote host using VNI | get logical-switch [local | remote] [mac-cache | arp-cache | vtep-cache] <vni> |
| Verify the OpenvSwitch Kernel Module | lsmod | grep openvswitch (from root mode) |
| Open vSwitch Configuration File | /etc/openvswitch/conf.db |
| Print the current version of openvswitch | ovs-vsctl –V |
| Prints a brief overview of the switch database configuration | ovs-vsctl show |
| Prints a list of configured bridges | ovs-vsctl list-br |
| Prints a list of ports on a specific bridge | ovs-vsctl list-ports <bridge> |
| OVSDB Log | ovsdb-tool show-log |
Logical Routing
| Description | Commands |
|---|---|
| View the Logical Router Forwarding table | get logical-router <UUID> forwarding |
| View the Logical Router Interface | get logical-router <UUID> interface |
| View the Logical Router Interfaces | get logical-router <UUID> interfaces |
| View the logical Router Neighbor | get logical-router <UUID> neighbor |
| View the logical Router Neighbors | get logical-router <UUID> neighbors |
Firewall
| Description | Commands |
|---|---|
| View app firewall virtual interfaces | ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/vif |
| View firewall rules with containing addrsets | ovs-appctl -t /var/run/openvswitch/nsxa-ctl dfw/rules <VIF_ID_NUMBER> |
Setup Troubleshooting
| Description | Commands |
|---|---|
| Verify SSH service status | get service ssh |
| Start the SSH service | start service ssh |
| Set the SSH service to autostart when the VM is powered on | set service ssh start-on-boot |
| Verify that the SSH service is running and Start on boot is set to True | get service ssh |
| View the Edge configuration | get configuration |
| Display the Edge node UUID | get node-uuid |
| View the Edge interfaces | get interfaces |
| Query the connection to the NSX Managers | get managers |
| View all the host switches information | get host-switches |
| View the tunnel port information | get tunnel-ports |
| Setup packet capture | set capture session <session-number> interface <port-uuid> direction <direction> |
| Remove captured session information | del capture session 1 |
| Enter root privileged Mode | st en |
Logical Routing
| Description | Commands |
|---|---|
| View the VTEPs | get vteps |
| View the logical routers | get logical-routers |
| View the logical router information | get logical-router <uuid> |
| View the logical router statistics | get logical-routers stats |
| View the logical router interfaces | get logical-router <uuid> interfaces |
| View the logical router neighbour | get logical-router <uuid> neighbor |
| View the logical router interfaces statistics | get logical-router interfaces stats |
| View the logical router bgp neighbour | get logical-router |
| To enter into the VRF construct | vrf <VRF-ID-of-Tier-0-SR> |
| View the bgp neighbor of a Tier-0 SR | (tier0_sr)> get bgp neighbor |
| (tier0_sr)> | get bgp neighbor summary |
| View the interfaces on a Tier-0 SR | (tier0_sr)> get interfaces |
| View the forwarding table (tier0_sr)> | get forwarding |
| View the Routers | (tier0_sr)> get route |
| View the BFG configuration | (tier0_sr)> get bfd-config |
| View BGP IPv4 route info | (tier0_sr)> get bgp ipv4 |
| View the Tier-1 or Tier-0 distributed router routing information | (tier[0 | 1]_dr)> get forwarding |
Firewall
| Description | Commands |
|---|---|
| View the list of Firewall Interfaces | get firewall interfaces |
| View the Firewall Ruleset and Rules | get firewall <interface_id> ruleset rule |
Load Balancer
| Description | Commands |
|---|---|
| Display load balancers configuration | get load-balancers |
| Display load balancer pool config | get load-balancer <lb-uuid> pools |
| Display load balancer virtual servers configuration | get load-balancer <lb-uuid> virtual-servers |
| Display specific load balancer virtual servers info | get load-balancer <lb-uuid> virtual-server Virtual_Server_ID |
| Display load balancer status | get load-balancer <lb-uuid> status |
| Display load balancer virtual servers stats | get load-balancer <lb-uuid> virtual-servers stats |
| Display load balancer stats | get load-balancer <lb-uuid> stats |
DHCP
| Description | Commands |
|---|---|
| Retrieve the DHCP server information | get dhcp servers |
| View the configured DHCP pools | get dhcp ip-pools |
| List the leased IP addresses | get dhcp leases |
VPN
| Description | Commands |
|---|---|
| Verify L2VPN session is active, identify the peers, and ensure that the tunnel status is up | get ipsecvpn session active |
| Verify that the sessions are up | get ipsecvpn session status |
| Check whether the ipsecvpn session is up between the local and remote peer | get ipsecvpn session summary |
| Get the l2vpn session, tunnel, and IPSEC session numbers, and check that the status is UI | get l2vpn sessions |
| Get statistical information of the local and remote peers, whether the status is UP, count of packets received, bytes received (RX), packets transmitted (TX), and packets dropped, malformed, or loops | get l2vpn session stats |
| Get the session configuration information | get l2vpn session config |
Tagged with: NSX-T Command Line
Keep the conversation going on Twitter!
Reply with Twitter