Sensors - July 2003 - Securing Your Process Control Networks

Interconnecting business networks with modern process networks has opened the door for hackers and viruses to enter the production environment. If left untended, this doorway can lead to failure of process networks of unauthorized modification of the production systems that are attached to them.

To maximize the uptime and efficiency of process networks, and to reduce liability in the case of process failure, security procedures should be in place to eliminate undesirable access to production systems. This security should take the form of separated networks where access to the process systems is strictly limited by routers and firewalls. Users and applications on the production networks are limited to those specifically required for the process—no email, no games, no Internet browsing. This philosophy may require a parallel installation of multiple networks to the same location. For example, control rooms will likely require a business network for email and business applications and a protected process network for control system devices. This article focuses on protecting process systems from external network problems.

Interconnect Your Networks
In most industrial plants, business users want production data from the plant floor and plant personnel need access to business applications. Often this takes the form of historical process information flowing from the control system to the business users and/or business system file and email access for control room personnel. Regardless of the reason, it involves a network connection between the process and business sides of the company. Because many control systems use Ethernet networking as a critical component of their system architecture, problems on the business network can be passed on to the process network through this business-process data link, and this can seriously impact production.

Problems that can befall a process network can be divided into two general categories: accidental and deliberate. Accidental problems are typically caused by cabling or configuration errors, or by faulty network devices. Deliberate problems are those caused by individuals with malicious intent, such as disgruntled employees, network hackers, or viral attack.

In industrial environments, we’ve found that accidental errors outnumber deliberate attacks, but the proliferation of viruses and the increase in PC-based control systems is leading to an increase in deliberate system intrusions.

Accidental and systemic issues on a process control network are often one of three types:

Noise or Bad Packets. A faulty network card or bad cabling can flood the network with garbage packets, causing communication slowdowns and disconnects.

IP Address Duplication. This common issue can be hard to diagnose and will result in the intermittent loss of connections and data.

Broadcast Storms. While broadcast packets are a normal part of any network, in large quantities they can stop normal network operations because all network devices must devote CPU resources to interpreting each packet. Simply plugging a patch cable into the wrong port can cause a broadcast storm that terminates all network communications.

Deliberate events on a process control network may fall into one of the following categories:

Viral Infection. Worms and trojans are generally designed to proliferate via mainstream mechanisms popularized by Microsoft Windows, Internet Explorer, and Outlook email on Intel-type computers. The chance of infection has increased in the production environment because these technologies are being used more frequently in process control due to low cost and the desire for open systems interoperability. Viruses can populate a computer via the network or directly from a floppy disk or homemade CD-ROM, so security policies must include both network protection (firewalls) and physical protection (lock up that server).

External Intrusion. Hacking of process control networks has been rare to date. However, as more mills connect to either the Internet or the corporate wide-area network (WAN), the incidence of unauthorized incursion is increasing. Typically a hacker will gain entrance to the mill business network and attempt to locate host computers he or she can invade. Servers (both Windows-based and multi-user varieties, such as Unix and Linux) are favorite targets for network penetration. Internal Web servers are a weak point in many industrial installations. They are deployed as an effective data distribution tool, but because they may not be directly exposed to the Internet, they are rarely maintained with the latest security patches and updates. Also, http is often allowed through firewalls to enable browser functionality, making a poorly configured Web server an inviting target.

Internal Intrusion. Unauthorized probing of the network and devices by employees is increasing dramatically because people are becoming more computer literate and the devices are increasingly PC-based. A common problem is a computer-savvy employee inadvertently changing the configuration of a device, causing process disruption. It is worth noting that passwords provide only limited protection against hacking because most process control groups use easy-to-remember (and easy-to-guess) passwords on their systems and don’t change them regularly.

Process Security Policy
While most companies have (should have?) a written IT security policy to define who gets network access to what, many don’t include the production environment in this document. It is obvious that most employees should not be granted access to the accounting databases. In the same way, most employees should not be granted access to the process control systems—a point often missed in manufacturing facilities. A process security policy defines user types based on function and location and specifies what sort of computer, network, and application access is required for each user. The policies should be broad-based directives, such as “PCs and users from the accounting area will not have access to the production networks” or “Floppy disc drives will be disabled for nonadministrative users on all process control PCs.” Specific exceptions are dealt with on a case-by-case basis and should not be part of the security policy.

Separate Your Networks
The security policy should mandate separate network segments for devices with different operational roles. From an integrity and access

Figure 1. Move critical production equipment onto high-security networks away from the rest of the business. Minimizing access, especially from the business networks, can maximize production uptime.

viewpoint, a mill’s networks can be divided into at least three levels of importance to ensure that broadcasts, network problems, and/or intruders are not passed through to the critical process equipment. Use routers to divide the network into smaller broadcast domains (subnets) and firewalls to provide access management based on how critical the attached devices are to the process. Limiting access to critical networks and locating most users on business networks can maximize production uptime. In Figure 1 users and servers that need to communicate with both production and business networks would be located on the process network. There is no communication directly from the business network to the production networks. Access between the process network and the production networks may be further limited to support only those protocols and hosts specifically required for configuration and data-gathering functions.

Production Network. These networks (e.g., DCS and PLC networks) are critical to mill production and must be functional at all times. They should contain a minimum of devices, which should be protected from unauthorized access. Common PCs and printers should not be attached to the critical networks unless specified by the manufacturer of the process equipment. At the router, network access to the critical networks should be limited to only those specific nodes that may require access, such as engineering configuration stations or plant information system data collection nodes. PCs on this network should be physically isolated and have limited floppy drive and CD-ROM access.

General Process Network. This type of network is used for communications by process personnel but is not critical to the operation of the mill. The process network includes PCs and printers and is used for common business and network applications, such as MS Office and email. The process network is often used to provide control room access to network applications, such as ISO and Workplace Hazardous Materials Information System information, or access to trends and archived data on a plant information system. A key feature of the process network is that its nodes may have limited access to the critical production networks. Data servers for the plant information system would be on the process network collecting data from the critical networks and serving to the business network. This network should be expected to suffer from viral infection without transmitting it to the production network.

Business Network. This network is usually serviced by office IT staff, so process personnel typically have little say in what happens in this area. Access to the Internet is likely and viral infections will occur. The office network should be denied access to the production networks, but may be allowed to connect to servers on the general process network. Most disruptions come from the office network, so it is important to isolate this one from the rest.

Network Topology
The structure shown in Figure 2 is typical of many industrial manufacturing installations today.

Figure 2. Many facilities have upgraded their networks to include a separate process control network. Unfortunately, this often doesn’t include any access control or firewall functionality. At best, this sort of installation provides a false sense of security. To ensure that hackers and viruses are unable to locate and attack critical process control equipment, a third level of network is recommended. This secure production network benefits from access controls on the router, separating it from the general process network.

While the plant has installed a router between the business and process networks to protect against broadcast storms, there is still no security in place to limit traffic across the router. Users on the process network still get their email and likely have connections (network drives/ shares) to business servers, both of which almost guarantee viral infection. Users on the business network and perhaps the Internet can scan the process control devices, looking for vulnerabilities.

Figure 3 illustrates a structure whereby only dedicated process control devices exist on the production network.

Figure 3. Process control equipment vendors often prefer to see their equipment on isolated Ethernet networks. A single router with firewall functionality capable of stateful packet inspection can provide several isolated and secure critical production networks. This sort of router blocks inbound access (unless specifically allowed) and inspects outbound traffic to increase security with a minimum of management.

The security policy should prohibit network shares, email, network printers, Internet access, and any computers that are not specifically provided by the process control system vendor. In many cases, it is advisable to provide individual networks for different equipment types, such as a DCS network, a PLC network, and a utility network for variable frequency drives and intelligent meters.

In conclusion, production network security is an ongoing project in which the security policy disallows all access unless there is a specific reason to allow it. The design, implementation, and management requirements for process networks have higher initial associated costs than an equivalent business IT network because increased training and outside support are required. The payback is reduced production downtime caused by network or computer interruption.

A Hypothetical Infection

Let’s examine the entry of a virus into a hypothetical plant. A business associate sends employee X an email that includes a virus. This virus begins to replicate itself onto other machines on the network: It emails itself to everyone in employee X’s address book; it scans the local network in an attempt to connect to http servers; and it tries to copy itself to other machines using any local Windows network shares. In two hours, 11 other computers in the area also have the virus. All of the infected computers continue to scan for new hosts, so network traffic rises to 400 times normal levels, and few real messages are getting through. The business server has the CPU running at 100% and is unable to service any other computers.

Now, in scenario A, the process network is connected through a router to the business network, but there is no firewall or security in place. The PLC configuration computer on the process network stores configurations on a network share connected to the business server. The business server infects the PLC computer through this connection and, within an hour, five other process PCs are infected. The process network is rendered useless and several PLCs fail due to the extreme traffic loads. The operator interface computers are unable to communicate with their controllers, so the operators are effectively blind and production stops until the virus is purged from the system 14 hours later.

Alternatively, in scenario B, there is a separate high-integrity production network for the process control systems. Although the PLC configuration computer on the process network may still be infected and the process network may fail, production is unaffected because the production network firewall doesn’t allow the PLC computer to connect to anything other than the PLCs. The PLCs are not infected because they don’t run an operating system susceptible to viruses, and the operator workstations are unaffected because they can’t be accessed from the configuration computer through the firewall. The mill does not suffer any downtime.

Tips for the Process Security Policy

Although there are many examples of security policies for IT business networks, there are few documents available to help you write a security policy for your production environment. Here are some suggestions and examples for a process network security policy.

Deny Everything. The basis for secure access is that any communication not expressly permitted is prohibited—turn all services and protocols off to start. A security policy should start by denying all access to all network resources and then grant access on a case-by-case basis.

Passwords. Operating systems provide such features as password aging and password policy enforcement. Good password policy dictates a minimum of eight characters and a mix of letters and numbers in user passwords and forces each user to change the password every 90 days. The operating system won’t accept a password that does not meet these rules. Good password procedures are as follows:
DON’T use your login name in any form (e.g., as is, reversed, capitalized, and doubled).

DON’T use your first, middle, or last name in any form or use your spouse’s or child’s name.

DON’T use other information easily obtained about you. This includes license plate numbers, telephone numbers, social security numbers, the make of your automobile, and the name of the street you live on.

DON’T use a password of all digits, or all the same letter.

DON’T use a word contained in English or foreign language dictionaries, spelling lists, or other lists of words.

DON’T use a password shorter than six characters.

DO use a password with mixed-case alphabetics.

DO use a password with nonalphabetic characters (digits or punctuation).

DO use a password that is easy to remember, so you don’t have to write it down.

A good username/password policy helps to verify the identify of the user. When using resources or sending messages in a large private network, not to mention the Internet, ascertaining this identity is of the utmost importance.

Logging. Save and analyze the logs of servers and routers to determine normal usage and to help identify unauthorized events. You will likely need a syslog server and RMON trap host to receive this information.

Phone Lines. Modems and phone lines present a significant threat of unauthorized access to the production network.

Don’t allow modems on users’ computers—provide a modem server that is protected and managed.

Remote users typically save the phone number, username, and password for the connection in both paper documentation and as part of a saved RAS session. This information is easily copied for misuse.

Use leased line (point-to-point) communication for SCADA systems where possible to ensure that you know who is at the other end. Where dial-up lines are required, use callback functionality to ensure that the other end of the connection is where you expect it to be.

Always use full username/password access for modem connections, either through Windows RAS functionality or by purchasing a modem with built-in authentication.

Network Shares. Avoid disc sharing or network shares to/from the production network unless absolutely necessary.

Removable Discs. Disable floppy drives and CD-ROM readers for non-administrative users on the production network. Either edit NT profiles
on operator workstations to accomplish this or physically disable the removable device.

Physical Access. Lock up all servers, routers, switches, patch panels, and horizontal distribution points. Any network device that can be physically accessed can be breached.

Virus Scanners. Ensure that all PCs on the general process network have a virus scanner running and that the virus identification files are up to date. If possible, automate the update functionality and check for updates every couple of days. If your process control vendor allows it, run a virus scanner on your production network PCs.

Email. Do not allow email on the production network. Period. Never. Email is allowed on the general process network.

Internet Access. Do not allow Internet access from the production network. Period. Never. Internet access is allowed from the general process network.

Unknown Sources. NEVER open any files or macros attached to an email from an unknown, suspicious, or untrustworthy source. Delete these attachments immediately, then double-delete them by emptying the trash/recycle bin/deleted items folder. Delete spam, chain, and other junk email without forwarding. Never download files from unknown or suspicious sources. Always scan a floppy diskette from any source for viruses before using it.

Network Scanning. The process security policy should expressly prohibit any network monitoring, traffic capture, port scanning, or security scanning unless these duties are within the scope of an employee’s regular duties.

Unacceptable Usage. The process security policy should expressly prohibit effecting security breaches or disrupting network communication. Security breaches include, but are not limited to, circumventing user authentication or security of any host, network, or account; accessing data of which the employee is not an intended recipient; or logging into a server or account that the employee is not expressly authorized to access. For the purposes of this section, disruption includes, but is not limited to, network sniffing, pinged floods, packet spoofing, denial of service, and forged routing information for malicious purposes.

Single Access Point. The firewall device must be the only access point between the production network and the rest of the mill’s networks and/or the Internet. Any form of cross-connection that bypasses the firewall device should be strictly prohibited.

Patches. Apply current applicable security patches/hot-fixes for any applications on process servers. Administrative owner groups must have processes in place to stay current on appropriate patches and security issues by monitoring security sources and vendor security information.

Accounts. Don’t use a privileged account on any process devices when a nonprivileged account will do.

Unneeded Services on Servers.õDisable services and applications not serving business or process requirements. Pay special attention to network applications on servers such as, but not limited to, telnet, ftp, rhost, rexec, rsh, Web services, finger, gopher, lat, and snmp.

Unneeded Services on Routers. Closely examine routers to ensure they are secure. Keep user accounts to a minimum and change passwords regularly. Keep the enable password on the router in a secure encrypted form.

Disallow the Following:
IP directed broadcasts
Proxy ARP
Incoming packets at the router sourced with invalid addresses such as RFC1918 address
Incoming packets sourced on the production network
TCP small services
UDP small services
All source routing
CDP on Cisco routers
All Web services running on the router

Use corporate standardized SNMP community strings – not “public” and “private.”

Access rules are to be added as production network needs arise.

Each router should have a banner statement, such as the following:
“UNAUTHORIZED ACCESS TO THIS NETWORK DEVICE IS PROHIBITED. You must have explicit permission to access or configure this device. All activities performed on this device may be logged, and violations of this policy may result in disciplinary action and may be reported to law enforcement. There is no right to privacy on this device.”