Using Wireshark to Diagnose AoIP Issues
Back in the old days, you could find any signal in a radio/TV station with a Telco Butt-set.
Now that everything is IP, the go-to tool is Wireshark. If you don't have it already, download here.
Simple Troubleshooting
Sometimes we can use the following easy method to learn what is going on with your audio issue. If you are having trouble with a particular audio source, give this method a try.
Load Wireshark onto your Navigator computer. Right-click on the source that you want to troubleshoot. Select "Monitor Source." Then select a set of speakers and click "Start Listening." When you select "Start Listening," you ensure that the source is transmitting and that your Navigator computer is receiving the stream. Make sure you can hear the audio.
Now you can start Wireshark. Don't apply any capture filters. Just select the NIC that is in the WNIP network and let it start capturing. As soon as you start to hear the pops/clicks/droputs/distortion that you are troubleshooting, you can stop the capture.
Save the capture file and send it to Wheatstone Tech Support via a cloud link (if you don't have Dropbox or similar, we can provide an upload link for you).
This method will capture the data from the source blade to your Navigator PC. But if it's clean at your Navigator PC but it's dirty at the console, we'll have to dive deeper.
Wireshark Deep Dive
For more in-depth troubleshooting, we may ask you to take your Wireshark journey a bit further. If we suspect flooding problems, IGMP or other multicast issues or just have no idea what’s going on, we may ask you to set up a span port or a “network tap” to help figure out what the problem is.
For this method, here are the things you will need.
You will either need a gigabit "dumb" switch (a cheap, unmanaged switch like a Netgear or D-Link that has no configurable properties) or you will need to be able to "mirror" a port on your managed switch. That's usually an IT Dept request. Note that newer "dumb" switches are not always as dumb as we would like. Some will automatically block multicast audio unless the stream has been subscribed to. This will hinder the capture of audio packets and will necessitate using a mirrored port instead (see below)
Some networks have their switches set up for security purposes to shut down a port if any (unauthorized) switch is connected to it, so if your facility is set up like that, work with your IT department to set up the mirrored port and skip using the "dumb" switch to grab a capture.
You will need a laptop or other computer that you can run Wireshark on. Wireshark can be downloaded for free from wireshark.org for Windows, Mac or Linux.
Below are the steps you will need to take.
If Using a GB Dumb Switch:
Unplug the PC, console, blade or other device that you are having trouble with from the network, whether it be a surface or PC or other device. Plug the dumb switch into the port you just vacated, then plug the device in question and the laptop with Wireshark loaded into the dumb switch. This ensures that your Wireshark will see the same traffic that the devices sees.
If Using a Mirrored Port:
• Plug the laptop with Wireshark loaded directly into the mirrored port. The mirroring of the port will ensure that your Wireshark sees the same traffic that the devices sees.
• Turn off Wifi on the laptop.
• Open Wireshark and select the NIC that is connected to your WNIP network. Make sure "Enable promiscuous mode" is checked.
• If troubleshooting a PC, start recording using the recording software of your choice. Make sure you are recording audio from one of your WNIP sources. If you are troubleshooting a playback issue, play back a known-good file rather than starting a recording. If troubleshooting another device such as a console, blade or other device, just make sure audio is routed into and out of the device.
• Without applying any capture filters, just start Wireshark in Capture mode. Let Wireshark capture packets until you hear the pop/click/dropout/distortion you are troubleshooting.
• Stop the capture and get the resulting capture file to Wheatstone tech support via a cloud link (we can provide a link for you). To the best of your ability, tell us approximately what time (your local time) the audio artifacts were heard.
Setting up a Long-Term Ring Buffer Capture
For problems that are difficult to capture, Wireshark can be run in a ring buffer configuration. This will allow the program to capture data over a long period of time without causing your computer to croak due to running out of disk space.
Wireshark can capture packets into files of a pre-determined size (number of KB, number of packets or number of seconds/minutes/hours). The program will name these files according to your specification and will keep only the last X number of files (also specified by the user). In this manner, when a problem occurs you can go to the Wireshark file list and find the one or two files that cover the time period in which the problem occurred and examine those files to learn what happened.
Make sure the computer is connected to a SPAN (mirrored) port or a dumb switch (as described above) and start Wireshark. Select the correct NIC and make sure "Enable promiscuous mode" is checked. Click to the Output tab and enter a descriptive name in the Filename box. Leave the Output format set to pcanpng and check the Create a new file automatically box. Next, select how you want Wireshark to determine when to create each new file. You can play with these settings to determine what's best but if you're confident you will learn of the problem within a few minutes, you might set it up for 15-minute-long files. If it might take hours to learn that a problem occurred, set it for 1-2 hours.
You will also need to decide how many files Wireshark should keep. If each file is going to be one hour long, and you think you'll need 24 hours to learn of the problem, you'll need 24 files. Check the Use a ring buffer box and tell it how many files to keep.
When setting up your ring buffer, keep in mind that the larger the packet capture file is, the harder it is to analzye so please don't set up a single file to hold many hours worth of data. In fact, it's much preferable to have dozens of smaller files rather than a few large ones. Select a file size no larger than 100 megabytes, or maybe go by number of packets instead. Set it for, say, 100,000 packets.
If you don't want to keep the Wireshark GUI running long-term on the computer you are using (maybe because you're also using it for other purposes), you can run Wireshark (actually, you'll use "dumpcap" to ditch the GUI) from the command line and set up your ring buffer there. Just type dumpcap -h at a command prompt and it will show you all the commandline options that are available. You'll probably use -D (get a list of ethernet interfaces available), -i (specify which interface to capture on), -b (set up ring buffer options), and -w (set the base filename to use for the capture).
The command:
dumpcap -i 5 -b duration:3600 -b files:4 -w mycapture.pcapng
will monitor ethernet interface #5 and will result in four files of one hour duration each, with the oldest one continually being deleted in favor of the newest one. The filenames will start with mycapture, followed by a file number, date/time stamp and the .pcapng extension. Here is an example:
mycapture_00012_20230214114846.pcapng
Once your problem occurs, it should be a simple matter to figure out which file or files will need to be examined in order to find the problem packets.
Related Articles
Using Wireshark to Find an Unknown IP Address
If you have a device that you need to connect to but don't have a clue what the IP address is, you can find it easily with a laptop and Wireshark. Wireshark can be downloaded at no cost and is easy to install. Assuming your laptop or the target ...Using Wireshark to Find an Unknown IP Address
If you have a device that you need to connect to but don't have a clue what the IP address is, you can find it easily with a laptop and Wireshark. Wireshark can be downloaded at no cost and is easy to install. Assuming your laptop or the target ...Optimizing Windows for AoIP
This comprehensive document explains how to prepare your Windows PC to work reliably with the WheatNet-IP Audio Driver. Download/View DocumentUsing WheatNet-IP Software LIOs to control VoxPro 7
This document explains how to configure bi-directional remote control functions on WheatNet-IP consoles and the WheatNet-IP audio driver. We'll start in Navigator. In our example, we have configured VoxPro to record and play back from WNIP Audio ...How To Set Up Telos VX with WheatNet-IP Using AES67
Prerequisites You will need: A Precision Timing Protocol (PTP) v2 clock source on your WNIP Local Area Network. Both the blades and the VoIP phone hybrid need to sync to this clock, and they do not generate it. There are numerous PTP clock generators ...