Note that the information below applies not only to PCs, PACs, and PLCs, but to pretty much any industry-standard compliant network device.

There are a ton of great network analysis tools out there today that you can pay huge sums of money for. But a go-to industry standard, and my tool of choice for everyday network troubleshooting, is a free piece of software called Wireshark.

Before it was Wireshark and before it ran on Windows, a humble open-source packet analyzer named Ethereal was all the rage with network geeks like myself. It took a bit of Linux know-how to get Ethereal up and running in those days—not to mention the additional driver files that had to be loaded, like WinPcap.

I used it for all kinds of troubleshooting. It was a great tool for identifying where your network problem lies on the network stack.

For example, you could use it to figure out if you're running into a firewall problem. Or use it to determine whether or not a device's TCP/IP stack is RFC compliant.

But even better, you could use it to view network sessions between two devices, to figure out what’s actually going on at a packet level. And that can be handy, especially when you’re working with different types of automation software and hardware designed by different vendors. You could even use it to look at Ethernet/IP traffic between devices.

Fast forward to today, and the Ethereal open-source project has settled a pesky trademark issue, renamed the packet analyzer software to Wireshark, and built a version that runs on Windows with an unattended installation of the software and its accompanying drivers.

You can download Wireshark to help you troubleshoot all of your PLC- and PAC-related networking problems at https://www.wireshark.org/.

Here are a few tips for using Wireshark to debug your automation network and devices.

The SYN/ACK how do you do

The beauty of industry standards is that they constrain hardware and software vendors into using the same rules for product development, which makes it a lot easier to get devices connecting and working together. For example, if a product conforms to industry standards, the way that piece of hardware initiates or responds to network connections must conform to a given set of rules.

In the TCP/IP protocol, this is known as the SYN/ACK handshake. It’s an exchange of specific types of network packets between two devices to open a TCP network connection and exchange data.



In the screenshot above you can see the 3-step SYN/ACK exchange, sometimes called a handshake, take place. This is the standard connection process for client/server connections in networking.

SYN - The device initiating the connection (client), in this case my PC, sends a SYN TCP packet to the destination device (server). SYN is short for synchronization. Note the sequence number of the packet is 0 because it’s the first packet of the connection.

SYN/ACK - The device I’m trying to connect to, a server out on the Internet somewhere, responds to the SYN packet with a SYN/ACK packet. The ACK is short for acknowledgement. Basically the receiving device is acknowledging my PC's connection attempt.

ACK - The third packet is an ACK packet from my PC.

With these steps completed, the connection is up and running and the devices can exchange information. In this case I’m using a basic telnet connection, because telnet is easy to work with for showing examples. In practice, though, I would always use a secure connection over SSH from a terminal program like Putty. Putty is another great and free software tool. Check out more network security tips in our security blog series.
Regardless of what kinds of devices are being used, this 3-way handshake must always take place to initiate a session between two network devices over TCP.

So what are some things to look for in the handshake to help pinpoint a problem?

The server might respond with a reset packet like this.



The reset packet is a way for the server to basically reject the connection request. Usually a reset packet indicates that the TCP port you’re trying to connect to isn’t open on the server. This can be a result of software misconfiguration or a firewall blocking certain ports.

So make sure you’ve configured the software on the server correctly to listen for connections on the port your client is trying to connect to. And make sure you don’t have something like the built-in Windows firewall configured to block ports.

Reset packets are also used during a session to tear down the connection. If you’re on a really unreliable network connection and the packets get way out of order, or you’re dropping way more packets than the network stack can handle, sometimes a device will send a reset packet to literally reset the connection.

In this case both devices have to go through the SYN/ACK process again. Note that the TCP/IP protocol is designed to put packets back in order, but all network stacks have their limits.

Sniffing network connections

Sometimes you need to actually look at the dataexchanged between two devices to figure out if the information they’re exchanging is in the right format or conforming to the underlying data protocol each device uses.

For example, let's say that the TCP connection comes up correctly, but the software you’re using to communicate with your PAC or PLC fails to open a successful connection. How would we troubleshoot that?

Wireshark has a handy tool built in for this. If you find a single packet in the network trace that’s part of the session you’re troubleshooting, you can right-click on the packet and select "follow TCP stream." That filters out all of the packets except the ones you’re looking for and also pops up a window that lets you look at the data exchanged between the devices during the session.

For example, I opened a telnet connection to a device and typed in "this is a test."


You would use this tool to help troubleshoot data exchanges. For example, maybe your device is expecting a carriage return, and instead your software is sending a carriage return line feed. You can toggle over to the Hex Dump view to see non-printable characters.

Those are a few tips on working with Wireshark to help troubleshoot your industrial devices and networks. For more industrial Ethernet troubleshooting and industrial cyber security info, subscribe to the Opto 22 blog.