Network Configuration Details

These notes look at a system's network configuration with a focus on exploration and understanding. Some of the tools used to examine the configuration also offer the means to alter the configuration, but these notes do not consider changes beyond a few coincident examples.

Toolkit

The ip command suite (package iproute) provides tools for examining and configuring network objects in detail: interfaces, IP addresses, routes, neighbors, etc.; see examples below.

This suite obsoletes arp (with ip neighbor), ifconfig (with ip address), netstat (with ip route), ipmaddr (with ip maddr), and ip tunnel (with ip tunnel). The obsolete commands come from package net-tools.

You can use Tcpdump (package tcpdump) or Wireshark (packages wireshark-gnome and wireshark) to examine network traffic at the packet level. You describe what packets you wish to capture with expressions intelligible to libpcap (eponymous package); the man page is pcap-filter.

Devices & Drivers

Use lshw to list your system's network devices:

-> lshw -quiet -class network
  *-network
       description: Ethernet interface
       product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
       vendor: Realtek Semiconductor Co., Ltd.
       physical id: 0
       bus info: pci@0000:03:00.0
       logical name: p2p1
       version: 0c
       serial: 60:a4:4c:ce:63:8f
       size: 100Mbit/s
       capacity: 1Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress msix vpd bus_master cap_list ethernet …
       configuration: autonegotiation=on broadcast=yes driver=r8169 …
       resources: irq:44 ioport:e000(size=256) …

You can learn more about your interface's features and capabilities by running ethtool with option --show-features or --driver:

-> ethtool --show-features p2p1 | wc --lines
45
-> ethtool --driver p2p1 | wc --lines
9

Ethernet Connectivity

Use ethtool (package ethtool) on an Ethernet interface to review link status, autonegotiation status, speed, duplex mode, and more. Here's an example for a desktop whose interface p2p1 is connected to a switch:

-> ethtool p2p1
Settings for p2p1:
	Supported ports: [ TP MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Half 1000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 
	Link partner advertised pause frame use: Symmetric
	Link partner advertised auto-negotiation: Yes
	Speed: 100Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: pumbg
	Wake-on: g
	Current message level: 0x00000033 (51)
			       drv probe ifdown ifup
	Link detected: yes

Note that the report reflects both nodes on this Ethernet segment. The desktop and switch used autonegotiation to determine their mutually-best interchange capability of 100 Mb/s and full duplex. The negotiated speed falls short of the desktop's top speed of 1000 Mb/s because the partner switch peaks at 100 Mb/s.

ethtool provides a basic sanity check in diagnosing network problems since it reveals the connection status of the first link from a host machine to its LAN. It can also report packet statistics, including errors and collisions:

-> ethtool --statistics p2p1
NIC statistics:
     tx_packets: 23761
     rx_packets: 25672
     tx_errors: 0
     rx_errors: 0
     rx_missed: 0
     align_errors: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     unicast: 25608
     broadcast: 16
     multicast: 48
     tx_aborted: 0
     tx_underrun: 0

If your computer's switch normally broadcasts LLDP packets, you can observe these as a second diagnostic of connectivity between your computer and its switch. Their presence at the expected interval indicates a functioning segment. Their absence may indicate a problem with the switch.

Ethernet devices periodically emit electrical pulses when they are not otherwise sending or receiving data frames. When a device detects pulses during traffic dearth, it is reassured that its link partner is still alive. Depending on the context, a pulse may be called a Link Integrity Test (LIT) pulse, a Normal Link Pulse (NLP), or a Fast Link Pulse (FLP) burst. If a device does not receive a frame or detect a pulse in a specified wait period, it enters a fail state until a connection is reestablished. Tcpdump and Wireshark and suchlike cannot sniff these pulses.

Two network devices directly connected by an Ethernet cable can automatically determine what transmission rate and duplex mode to use. This autonegotiation chooses the greatest common capabilities supported by the pair of devices. For example, when a desktop's NIC capable of 1000 Mb/s at full duplex connects to a switch topping out at 100 Mb/s full duplex, they agree to transmit at 100 Mb/s and communicate over full duplex. They also agree to the type of pause frame to use for flow control. Gigabit devices negotiate additional terms of engagement. Autonegotiation happens transparently on connection provided that both devices offer it. A device with autonegotiation enabled can infer the hard-set line speed of a non-negotiating partner, a process called parallel detection, but it cannot infer the partner's duplex mode. Instead, it uses half duplex, which is Ethernet's default mode, and hopes its partner does as well.

Autonegotiation operates in the physical layer. Each device emits Fast Link Pulse (FLP) bursts that encode its capabilities in a link code word (LCW) or page (of 16 bits). Use ethtool to see the agreement of autonegotiation or inference of parallel detection. Packet sniffers are anosmic to negotiation pulses.

You can also use mii-tool (package net-tools) on a wired NIC to get a concise summary of link status:

-> mii-tool p2p1
p2p1: negotiated 100baseTx-FD flow-control, link ok

Its man page lists this command as obsolete and defers to ethtool, however. Similar utility mii-diag reports more detail. (Its man page does not mark it as obsolete.)

The Ethernet Configuration Test Protocol (ECTP) provides link-layer pinging and network discovery. It is also known as the LOOP or Loopback protocol (unrelated with the loopback interface, lo). ECTP is not an IEEE 802.x specification, however. It looks like only Cisco and DEC implemented ECTP, and its support in Linux is not clear (to me).

To list your system's network interfaces and to see their properties:

-> ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff

This host has a single Ethernet interface p2p1 (link/ether). p2p1 is enabled and connected to a cable or radio (flags UP and LOWER_UP), which means that p2p1 is willing and able to both transmit and receive packets (state UP). It also accepts broadcast and multicast packets from the LAN (flags BROADCAST and MULTICAST). The kernel fragments transmissions over p2p1 into packets with a maximum size of 1,500 bytes (mtu), and it queues outgoing packets using a first-in/first-out algorithm and a queue length of 1,000 packets (qdisc pfifo_fast, qlen, "qdisc" for "queue discipline"). p2p1 accepts unicast packets addressed to 60:a4:4c:ce:63:8f, its link-layer or MAC address, and it also accepts broadcast packets addressed to ff:ff:ff:ff:ff:ff.

The host also has the usual loopback device lo, which is operating (UP, LOWER_UP). Loopback does not queue packets (qdisc noqueue). Link-layer addresses do not apply nor does broadcasting (00:00:00:00:00:00).

The state reports the operational status of the interface, rather than the administrative setting reported with the interface's flags enclosed in angle brackets. Status UP indicates that the interface can operationally exchange packets with another node: The interface's transceiver is enabled, and it is connected to a live carrier. When an interface is disabled administratively, flags UP and LOWER_UP do not appear in the angle brackets:

-> ip link set p2p1 down
-> ip link show p2p1
2: p2p1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
    link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff
-> ip link set p2p1 up

The flag LOWER_UP connotes a layer in the network stack that is lower than the link layer; e.g. the carrier is up. When an interface is enabled but otherwise disconnected from its carrier, flag NO-CARRIER replaces flag LOWER_UP:

-> ip link show p2p1
2: p2p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
    link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff

More operational states are possible: dormant, testing, unknown, and not present. The possible administrative flags are up, down, and testing. For details, read about ifOperStatus and ifAdminStatus in RFC 1213 and RFC 2863.

You can see an interface's packet statistics by specifying option -stat:

-> ip -stat link show p2p1
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 60:a4:4c:ce:63:8f brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast   
    8780168    16407    0       0       0       0      
    TX: bytes  packets  errors  dropped carrier collsns 
    1965052    17761    0       0       0       0     

If there are errors, add a second -stat to see details.

IP Addresses

To see the internet addresses assigned to your system's interfaces:

-> ip -family inet address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 192.168.1.4/24 brd 192.168.1.255 scope global dynamic p2p1
       valid_lft 74853sec preferred_lft 74853sec

The report shows the IPv4 unicast and broadcast (if applicable) addresses associated with each interface (inet, brd). The scope of address 127.0.0.0 is limited to the host system because of the parochial nature of loopback (scope host). In contrast, address 192.168.1.4 on NIC p2p1 travels the LAN (scope global)

The dynamic flag and finite lifetimes (valid_lft, preferred_lft) on 192.168.1.4 reflect the address's assignment through DHCP. Compare this dynamic address with a static assignment:

-> ip address add dev p2p1 192.168.2.4/24 brd +
-> ip -family inet address show dev p2p1
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 192.168.1.4/24 brd 192.168.1.255 scope global dynamic p2p1
       valid_lft 69900sec preferred_lft 69900sec
    inet 192.168.2.4/24 brd 192.168.2.255 scope global p2p1
       valid_lft forever preferred_lft forever
-> ip address del dev p2p1 192.168.2.4/24

After the preferred lifetime lapses but before the valid lifetime lapses, a dynamic address becomes deprecated: Established connections continue to work, but new connections on the address are denied. Once the valid lifetime passes, the address expires for any use.

You can get IPv6 info, too:

-> ip -family inet6 address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 fe80::62a4:4cff:fece:638f/64 scope link 
       valid_lft forever preferred_lft forever

Use ipcalculator (package ipcalculator) to expound IP notation:

-> ipcalculator --nocolor 191.168.1.4/24
Address:   191.168.1.4          10111111.10101000.00000001. 00000100
Netmask:   255.255.255.0 = 24   11111111.11111111.11111111. 00000000
Wildcard:  0.0.0.255            00000000.00000000.00000000. 11111111
=>
Network:   191.168.1.0/24       10111111.10101000.00000001. 00000000
HostMin:   191.168.1.1          10111111.10101000.00000001. 00000001
HostMax:   191.168.1.254        10111111.10101000.00000001. 11111110
Broadcast: 191.168.1.255        10111111.10101000.00000001. 11111111
Hosts/Net: 254                   Class B

Or, use ipcalc (package initscripts):

-> ipcalc -hnmpb 192.168.1.4/24 
NETMASK=255.255.255.0
PREFIX=24
BROADCAST=192.168.1.255
NETWORK=192.168.1.0
HOSTNAME=desktop.home

Routing

Here's the main routing table of a desktop, named Avacado, with a single NIC, an Ethernet interface, assigned to IP address 192.168.1.4:

-> ip route show table main
default via 192.168.1.1 dev p2p1  proto static  metric 1024 
192.168.1.0/24 dev p2p1  proto kernel  scope link  src 192.168.1.4 

The first line describes the default route that Avacado uses to send packets to a host that is not a neighbor in Avacado's LAN. This default punts the packet to the LAN's gateway at 192.168.1.1 and lets the gateway figure out where next to route the packet. Avacado can reach the gateway because both computers are neighbors on the same LAN; that's where the second route above comes in.

The second route indicates that the subnet 192.168.1.0/24 is directly reachable via the LAN that Avacado itself belongs to. When Avacado wants to send an IP packet to Banana at 192.168.1.3, say, it encloses the IP packet in an Ethernet packet addressed to Banana's MAC address and then drops the packet on the LAN segment. Since Banana monitors the LAN's traffic, it notices and takes ownership of any such packet.

These two routes suffice to send every outgoing packet on its way.

Link-layer (e.g., Ethernet) packets incorporate the MAC addresses of both the source and destination stations. For example, when Avacado wants to send a packet to Banana, it finds the destination MAC address by first broadcasting an ARP request over the LAN. The ARP message asks the owner of IP address 192.168.1.3 to introduce itself. Its destination address is the special link-layer broadcast address (ff:ff:ff:ff:ff:ff) that all stations monitor, and its source address is Avacado's own MAC address. Banana heeds the request and remits its MAC address directly to Avacado. Now in possession of Banana's MAC address, Avacado can get down to business. Here's what that sequence looks like when Avacado (60:…:8f) pings Banana (c4:…:74):

-> tcpdump -q -e -n -t -i p2p1 'arp or icmp'

60:a4:4c:ce:63:8f > Broadcast, ARP, length 42: Request who-has 192.168.1.3 tell 192.168.1.4, length 28
c4:17:fe:75:f7:74 > 60:a4:4c:ce:63:8f, ARP, length 60: Reply 192.168.1.3 is-at c4:17:fe:75:f7:74, length 46
60:a4:4c:ce:63:8f > c4:17:fe:75:f7:74, IPv4, length 98: 192.168.1.4 > 192.168.1.3: ICMP echo request, id 2925, seq 1, length 64
c4:17:fe:75:f7:74 > 60:a4:4c:ce:63:8f, IPv4, length 98: 192.168.1.3 > 192.168.1.4: ICMP echo reply, id 2925, seq 1, length 64

Avacado can skip the ARP request if it already knows Banana's addresses:

-> ip neigh list 192.168.1.3
192.168.1.3 dev p2p1 lladdr c4:17:fe:75:f7:74 REACHABLE

Name Servers

To see what DNS servers your system specifies:

-> cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 208.67.222.123
nameserver 208.67.220.123

To see what server is active:

-> dig | grep SERVER
;; SERVER: 208.67.222.123#53(208.67.222.123)

Or, you can use tcpdump

-> tcpdump -q -n -t -i p2p1 port 53

IP 192.168.1.4.37112 > 208.67.222.123.domain: UDP, length 28
IP 208.67.222.123.domain > 192.168.1.4.37112: UDP, length 239

A LAN's gateway may be configured to cache host names once resolved rather than to send each query anew to an upstream name server. You can see the effect of this with dig by looking up a name you've not looked up before (or recently) and then repeating the command. The reported query times tell the tale:

-> dig fox.com | grep -P 'Query time|SERVER'
;; Query time: 29 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
-> dig fox.com | grep -P 'Query time|SERVER'
;; Query time: 2 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)

In this example, the supplicant computer consults the name server on its gateway at 192.168.1.1 (port 53). On the first request, the gateway asked a full-featured name server to resolve the name into an IP address. But on the next request, the gateway simply remitted the cached answer.

Dnsmasq

Tomato firmware for residential gateways uses Dnsmasq for DHCP and DNS.

Dnsmasq stores configuration settings in /etc/dnsmasq.conf:

pid-file=/var/run/dnsmasq.pid
interface=br0
resolv-file=/etc/resolv.dnsmasq
addn-hosts=/etc/hosts.dnsmasq
expand-hosts
min-port=4096
dhcp-range=192.168.1.100,192.168.1.149,255.255.255.0,1440m
dhcp-option=3,192.168.1.1
dhcp-lease-max=255
dhcp-authoritative
dhcp-host=00:0F:1F:73:94:F2,192.168.1.2,1440m
dhcp-host=60:A4:4C:CE:63:8F,192.168.1.4,1440m
dhcp-host=00:26:B9:A0:A2:CC,C4:17:FE:75:F7:74,192.168.1.3,1440m

Note the static assignments recorded by the multiple dhcp-host settings.

As a name server, Dnsmasq maintains a cache of familiar names and forwards unrecognized names to an upstream DNS server:

-> cat /etc/resolv.dnsmasq
nameserver 208.67.222.123
nameserver 208.67.220.123
nameserver 8.8.8.8

It also keeps its own list of local-host names for static DHCP assignments:

-> cat /etc/hosts.dnsmasq
192.168.1.1 gateway
192.168.1.2 old-desktop.home
192.168.1.4 desktop.home
192.168.1.3 gogo.home

Active DHCP leases live in /var/lib/misc/dnsmasq.leases; for example:

cat /var/lib/misc/dnsmasq.leases 
86400 c4:17:fe:75:f7:74 192.168.1.3 gogo 01:c4:17:fe:75:f7:74
85332 60:a4:4c:ce:63:8f 192.168.1.4 desktop 01:60:a4:4c:ce:63:8f

Assorted Reminders & Remarks

Well-known mappings between services and ports go in /etc/services:

-> grep -P 'boot(ps|pc)' /etc/services 
bootps          67/tcp                          # BOOTP server
bootps          67/udp
bootpc          68/tcp          dhcpc           # BOOTP client
bootpc          68/udp          dhcpc

The man pages is services.

A MAC address's first three bytes constitute an Organizationally Unique Identifier (OUI) assigned to a vendor. A vendor may have multiple OUIs.