OptoPartner Enginuity shares how their engineers ensure reliability and minimize downtime with custom scalable redundancy.

When your operations rely on consistent performance and uptime, you can't afford to be caught off guard by control system failures.

But nothing lasts forever—failures will occur. A robust redundancy system ensures your processes continue running without costly downtime. 

This blog post, written by Daniel Baugh of Enginuity Inc., an IIoT-certified OptoPartner, explores how you can implement effective redundancy solutions using groov products from Opto 22.

"Everybody has a plan until they get punched in the mouth" – Former Heavy Weight Champion, Mike Tyson.

These profound words of wisdom from the boxing icon demonstrate the need for a backup plan. No one wants to get punched in the mouth. But things happen, and when they inevitably do, you need to be prepared.

Even the most comprehensive and secure systems are subject to failure. Being unprepared, should things go awry, can quite literally break an organization. If your business relies on consistent product and/or communication delivery, it is imperative that redundancy systems are put into place.

Integrating redundancy into Programmable Logic Controller (PLC) systems is crucial for several reasons, primarily to enhance reliability, availability, and operational efficiency. These reasons directly translate into increased safety and profitability for sectors such as manufacturing, critical industries, process industries, utilities, and infrastructure.

Two is one, and one is none

“The idea was to create a system where you would end up having more fault tolerance if one of your PLCs dies. You could still run. You could still operate. You could still keep the system online and buy yourself time to swap out whatever is defective. Without it, the whole system goes down, and then it's an urgent matter versus a less urgent matter that has to be addressed.” – Enginuity Automation Engineer, Adam Desjardins

Opto 22's groov EPIC is a versatile industrial controller that allows you to collect, process, view, and exchange data where it’s produced. Additionally, it provides the ability to create bespoke redundancy solutions, particularly when combined with CODESYS software, which supports various industrial protocols—EtherNet/IP was used for our first project, but the code is easily adaptable to various protocols supported by CODESYS's IEC 61131-3 programming platform.

Enginuity’s Automation team offers clients a tailored PLC redundancy solution that efficiently and cost-effectively ensures system reliability while also allowing for greater control and flexibility in implementation. Furthermore, it may be seamlessly integrated into existing operations and provides scalability.

Methodology

Automation & Robotics Engineer Adam Desjardins humbly describes the executed methodology: “Technically, we did nothing novel other than applying first principles, like not allowing for single points of failure." 

He and co-engineer Seinan Khan implemented the following 6 mandates in their strategy:

1. Hardware Redundancy: Implement Hot Standby* failover and duplicate controllers and circuits that can take over if the primary components fail. This implementation involves designing the system with redundant boards or circuits that can be switched on when needed. This approach is common in high-reliability embedded systems and helps maintain operation during failures or upgrades.

*Hot Standby refers to a redundancy setup where a secondary system runs in parallel with the primary system, constantly synchronized and ready to take over immediately in case of a primary system failure. The minimal lag and quick switchover time are key features of this setup.

This reduction in lag and switchover in critical applications reduces urgency when components fail, improving uptime and reducing “burn” on your support resources.

2. Continuous Monitoring: Develop a system to continuously monitor critical signals and components to detect failures early. Embedded code checks signal integrity and system health, allowing for timely switchover to the backup controller when necessary.
3. Syncing Data: Continuous data sharing between both PLCs and the Redundancy Management Controller (RMC) ensures that they are optimally synced and ready for a failover event.
4. Failover: Ensures that if one node fails, the other can handle the load without significant performance degradation. This is crucial for maintaining service reliability and availability.
5. Scalable Architecture: Design a system architecture that allows for scalability and flexibility, including using a modular approach where additional controllers can be easily added either as sets or to a set.
6. Connectivity: Although various protocols such as Modbus, ProfiNet, and CanBus are available, the communication protocol used between Enginuity’s groov EPIC controllers and the RMC is Ethernet. Ethernet allows the team to achieve a switch-over time of only 100 milliseconds!
   
   

Outcome

The outcome was a system that can recover from faults faster and seemingly uninterrupted, as well as communicate externally if a failure should occur. The client has time to repair, upgrade, or fix the system without sacrificing precious operational efficiency. 

Although some operators may find the cost and complexity of integrating redundancy somewhat daunting, the necessity is becoming widely understood. We are living in a digital age, and as we enjoy the fruits of automated systems, we must avoid exposing our operations to failure—and if we do happen to get “punched in the mouth,” there’s a secondary PLC that has got our back.

-Daniel Baugh
Content Creation Specialist
Enginuity Inc.
www.enginuityinc.ca
dbaugh@enginuityinc.ca