7. First Steps in Network Monitoring and Performance

Learning baseline: Ubuntu Server 24.04 LTS, with an alternate Rocky Linux 9 path
Last Updated: September 15, 2026

Prerequisites: Complete lessons 1–6. The two VMs communicate through their lab NICs, and TCP 8000 on the server is allowed only from the lab client. If another lab server owns port 8000, identify it and stop it using that exercise's procedure.

Outcomes: Distinguish listeners, connections, queues, packets and traffic volumes, and turn “it is slow” into a testable question. Allow approximately 2–3 hours including practice.

Tools and observation points

Question	Tool	What it cannot establish
Which address and port is a program listening on?	`ss -ltnp`	Whether an external client can reach it
What are the connection states and queues?	`ss -tinm`	The cause of loss from one snapshot
Which packets crossed this NIC?	`tcpdump`	Activity on other, uncaptured NICs or namespaces
Which host pairs use bandwidth?	`iftop`	Application processing time or exact billed traffic
Where does an HTTP request spend time?	`curl -w`	Timing inside individual server functions

Older material may use netstat. This course uses ss for socket observation. netstat -lnt and ss -ltn answer similar questions, but their formats and fields are not identical. Missing netstat is not a reason to remove newer tools or reconfigure the system.

Check the required packages on the server. On Ubuntu:

bash

sudo apt update
sudo apt install iproute2 tcpdump python3

On Rocky:

bash

sudo dnf install iproute tcpdump python3

The client needs curl; install it using the package-manager instructions in lesson 1.

Generate a small HTTP workload

Server console A: Create one exercise file in an empty temporary directory.

bash

LAB_WEB=$(mktemp -d /tmp/net-web.XXXXXX)
printf 'network lesson\n' > "$LAB_WEB/index.html"
timeout 300 python3 -m http.server 8000 \
  --bind 192.0.2.20 --directory "$LAB_WEB"

This server exposes the selected directory and stops after five minutes. Exit status 124 means timeout reached its limit. This is not a production web server with TLS and user authentication. Do not put other files or symbolic links in the exercise directory.

Client: Send one request, then ten more at one-second intervals.

bash

curl --disable --noproxy '*' --connect-timeout 2 --max-time 5 \
  -i http://192.0.2.20:8000/
for n in $(seq 1 10); do
  curl --disable --noproxy '*' --connect-timeout 2 --max-time 5 \
    -sS -o /dev/null http://192.0.2.20:8000/
  sleep 1
done

Pass criteria: The first response is HTTP 200 with network lesson in its body. Subsequent requests finish without errors and appear in the server log. If they fail, return to the lesson 4 diagnosis sequence instead of increasing load.

Read socket states and queues

Server console B: Use another terminal.

bash

sudo ss -ltnp 'sport = :8000'
ss -tinm '( sport = :8000 or dport = :8000 )'

Find the 192.0.2.20:8000 listener in the first command. Repeat the second observation while requests run. Short connections can finish between snapshots; the absence of an ESTABLISHED row does not establish request failure. Correlate server logs and packet captures.

State or field	Interpretation
LISTEN `Recv-Q` / `Send-Q`	Connections awaiting acceptance / maximum accept backlog
ESTABLISHED `Recv-Q`	In-order bytes not yet read by the application
ESTABLISHED `Send-Q`	Unacknowledged stream bytes, including data not yet sent
`rtt`	TCP's round-trip estimate, not total HTTP duration
`cwnd`	Sender congestion window, usually shown in segments by `ss`
`skmem`	Socket memory accounting including overhead, not simply queued payload bytes

Zero queues are normal with these small requests. Building a large queue is not a success condition. The Linux TCP queue exercise uses a longer-lived connection and provides further interpretation.

Capture packets on a selected NIC

Server console B: Match the lab NIC against its MAC address recorded in the hypervisor.

bash

ip -br link

Substitute the verified name below. enp0s8 is an example, not an instruction to select your management NIC.

bash

LAB_IF=enp0s8
ip -br address show dev "$LAB_IF"
sudo timeout 15 tcpdump -ni "$LAB_IF" -c 20 \
  'host 192.0.2.10 and tcp port 8000'

Start the capture, then repeat the curl request from the client. -n avoids name lookups, -i selects the interface, and -c 20 limits the captured packet count. The 15-second limit may finish first.

Illustrative TCP flags include [S] for SYN, [S.] for SYN+ACK, [.] for ACK, and [F.] for FIN+ACK. Data and acknowledgments can be combined in different ways. Do not memorize a fixed rule such as “one HTTP request is exactly three packets”.

To save evidence, capture only your exercise traffic for a short period.

bash

LAB_CAPTURE=$(mktemp -d /tmp/net-capture.XXXXXX)
sudo timeout 15 tcpdump -Z "$(id -un)" -ni "$LAB_IF" -c 20 \
  -w "$LAB_CAPTURE/http.pcap" 'host 192.0.2.10 and tcp port 8000'
sudo tcpdump -nn -r "$LAB_CAPTURE/http.pcap"

Send the client request again while the write command runs. -Z "$(id -un)" drops to the current regular user after opening the capture device, allowing the file to be written in that user's temporary directory. A PCAP can contain addresses and real payloads. Use only the exercise string, not SSH credentials or real user traffic. The read command uses sudo to accommodate file-permission differences.

Interpretation: Identify request and response addresses and ports. If the capture is empty, first check its NIC, filter and time window. NIC offload can make host-observed packet sizes and checksums differ from the wire. A checksum warning in one capture does not establish a network fault.

Bandwidth and request timing

iftop is optional. Ubuntu can install it with sudo apt install iftop. If Rocky's configured repositories do not provide it, skip this exercise or first check the EPEL prerequisites in lesson 6. A failed sudo dnf install iftop is not a reason to run an arbitrary external installation script.

Server: In the terminal where the verified LAB_IF is set:

bash

sudo iftop -nNP -i "$LAB_IF" -f 'host 192.0.2.10'

Repeat client requests, read directional rates, and press q to exit. Small, short requests may appear close to zero. This is not a link-capacity or internet-throughput test.

Client: Observe HTTP timing separately.

bash

curl --disable --noproxy '*' --connect-timeout 2 --max-time 5 \
  -sS -o /dev/null \
  -w 'dns=%{time_namelookup} connect=%{time_connect} first_byte=%{time_starttransfer} total=%{time_total}\n' \
  http://192.0.2.20:8000/

Values are seconds elapsed from the start. This request uses an IP literal and does not measure DNS performance. Time to first byte includes earlier steps and server preparation; do not add all fields as though they were independent intervals.

Measurement	Meaning	Conditions to control
Latency	Duration of an operation or round trip	Request size, route, server load
Throughput	Amount delivered per unit time	Concurrency, direction, bytes versus bits, duration
Loss/retransmission	Observed missing packets or TCP recovery	Capture location/drops, route changes, time window
Resource use	CPU, memory and I/O activity or waiting	Process, VM allocation, other workload

1 byte = 8 bits. Do not equate 100 Mbit/s with 100 MB/s. For repeated measurements, record conditions and minimum/median/maximum values or another suitable distribution.

Kernel and I/O study comes next

These commands only observe:

bash

uname -r
sysctl net.ipv4.tcp_congestion_control
sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem
lsblk -o NAME,TYPE,SIZE

The buffer policy values returned by sysctl are not each socket's current usage. Before increasing them, distinguish socket buffers, windows and BDP. After identifying a disk with lsblk, reading /sys/block/DEVICE/queue/scheduler shows supported/selected block I/O schedulers; not every virtual device offers the same choices.

A block I/O scheduler handles storage requests, while a network qdisc handles network transmit queuing. Sharing the word “scheduler” does not make them interchangeable tuning controls. This lesson does not change sysctls, MTU, qdiscs or I/O schedulers. Advanced work requires a separate experiment with one changed variable, measurements and rollback.

Cleanup and completion

Stop the server in console A with Ctrl+C or its time limit. In that same terminal, remove only your exercise file and directory.

bash

rm -- "${LAB_WEB:?}/index.html"
rmdir -- "${LAB_WEB:?}"

In the capture terminal, remove the capture too.

bash

sudo rm -- "${LAB_CAPTURE:?}/http.pcap"
rmdir -- "${LAB_CAPTURE:?}"

${VARIABLE:?} stops the command when the variable is unset or empty. If a directory is not empty, investigate instead of using a broad recursive deletion.

Explain why a listener, a TCP connection and HTTP success are different evidence.
Identify both directions' addresses/ports in a PCAP and record the capture point.
Explain queue/bandwidth units and the DNS-measurement limit of an IP-literal request.
Distinguish observation from tuning that needs a separate controlled experiment.

References and next lesson

Previous: Firewalls and host security · Quiz · Next: Capstone

Traffic Management

Security

Observability

Resilience

Advanced

Comparison Guide

ArgoCD

Metrics

Logging

Tracing

Alerting

Observability End-to-End Labs

7. First Steps in Network Monitoring and Performance ​

Tools and observation points ​

Generate a small HTTP workload ​

Read socket states and queues ​

Capture packets on a selected NIC ​

Bandwidth and request timing ​

Kernel and I/O study comes next ​

Cleanup and completion ​

References and next lesson ​

7. First Steps in Network Monitoring and Performance

Tools and observation points

Generate a small HTTP workload

Read socket states and queues

Capture packets on a selected NIC

Bandwidth and request timing

Kernel and I/O study comes next

Cleanup and completion

References and next lesson