Running tcpdump inside Docker Swarm

Photo by Taylor Vick on Unsplash

Have you ever deployed a stack in Docker Swarm and wanted to run tcpdump to monitor packets that are passing between different network components? I have.

If you run the tcpdump command directly on the host you won’t monitor most of the packets that are moving between different services can containers. You need to somehow go inside the networks and runt it.

When you run a docker stack, you will have one ingress network and one overlay network. They each have their role. You can read about them here.

So how to run tcpdump in Docker Swarm and monitor the connection between different services and containers? You may need to run multiple tcpdumps. One for the ingress network and one for overlay network. For each of them, you need to use nsenter command and sh into a container. It might be easier to use another Docker container like nicolaka/netshoot. But first, find the ID for ingress and overlay network by listing networks in the host:

docker network lsujgie39kofp2          ingress-network          swarm
nmo87n20s1s4 overlay-network swarm

Then for running nicolaka/netshoot container, use this command:

docker run -it --rm -v /var/run/docker/netns:/var/run/docker/netns --privileged=true nicolaka/netshoot

You need to map /var/run/docker/netns in the container and use the privileged mode. Then you can see a list of networks in the /var/run/docker/netns directory (run a ls to see them). You must see many IDs but you will find these twos:

1-ujgie39kof
1-nmo87n20s1

Which their IDs are showing they are the overlay and ingress network. Let’s say we want to run tcpdump on the first one, so we use:

nsenter --net=/var/run/docker/netns/1-ujgie39kof sh

And now we are in the shell of “1-ujgie39kof”. Now you can run tcpdump here and monitor the packets which are passing between containers (running tcpdump on the host directly doesn’t show them).

Just remember, if you want to see all of the packets you need to run tcpdump -i any, to monitor every network interface. (there are so many of them, just run ifconfig too see them). Also, you need to do this for both ingress and overlay network to see communications inside Docker Swarm.

Everything else about tcpdump is normal here, you can also use -w option to save the dump to a pcap file and later retrieve it using docker cp command.

I hope this helps you, like it helped me. I also need to mention that I used this post on Docker forums to come up with this post. But I wrote this post to both have it documented somewhere and also I’m doing more interesting stuff using this which I’ll post about them soon.

--

--

--

Computer Programmer with passion for new stuff in the tech world. Mostly focused on backend design and architecture.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

POP vs IMAP | What is it, and what’s the difference?

Should You Draw It?

PatternFly-design — Going Github

The PatternPast visual logo. It reads “PatternFly vintage blog, established 2015.”

Deep dive into C++ STLs — vectors

Learning to Code — Part 5c: Method Overloading and Recursion

{ TCP/IP model of networking}

5 lessons I learned working in a remote scaled Scrum team

Мessy Route Handling in Golang and how to avoid it

Two gophers, one is trying to stop a running dog that creates havoc and messing up bunch of cables, that are connected to a weird looking machine.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vahid Mostofi

Vahid Mostofi

Computer Programmer with passion for new stuff in the tech world. Mostly focused on backend design and architecture.

More from Medium

Multiple K8S Cluster Management with Rancher + K3S (Lightweight k8s cluster for edge and…

How to collect NVIDIA GPU Metrics using Prometheus, Docker & Pushgateway

ElastAlert 2 for OpenShift 4.9

Async HTTP Requests with Aiohttp & Aiofiles