Production Line Data Collection (MD&M Panel Discussion)
Updated: Feb 6, 2020
“Planning for Now: Immediate Need & Future Investment in Industrial IoT.”
If you missed the MD&M Conference in Anaheim this year, a summary of the Industrial IoT panel discussion is provided. This article highlights the questions and answers from the panel discussion: “Planning for Now: Immediate Need & Future Investment in Industrial IoT.” Design News, UBM was the moderator. The panel members represented analytics, integration, end-to-end security, and sensors. Panelist 1 (GE), Panelist 2 (RRAMAC Connected Systems), Keary Donovan (elliTek, Inc.), Panelist 4 (Balluff Inc.).
The panelists discussed Smart Manufacturing requirements regarding data storage, analytics, and visibility. Industrial IoT motivations, concerns, barriers to entry, advice, standards, legacy equipment, workforce, and data crack were all topics of discussion.
What’s Hot Right Now in Industrial IoT
Panel Host: As I’m out on the conference road, I go to a lot of sessions on Industrial IoT and IoT, and I hear a lot of promise about the impact of connected stuff. What I’d like to know from our panelists is, what IoT products and services they see that are gaining the most traction among their clients?
Panelist 1: Our customers are asking about outcomes and analytics. Analytics is in high demand. Second is investment whether it is sensors or IoT platform.
Panelist 2: One of the things I’ve seen in recent years is the secure connectivity piece has grown. Companies have gotten smarter on the edge nodes, and it is enabling the analytics platform.
Keary: Since you guys covered the analytics and security, I’ll add the cost considerations. elliTek manufactures an MES Gateway Appliance. A lot of our interactions are about how our end-users can use what they already have -- use their legacy systems, use the infrastructure that’s already in place to get the data from a new MES package or new analytics software, and how that will be secure. We have ways of doing that. As an example, medical device manufacturers. Within our industry, if you can reach the edge of your network without a PC, that means that you now, as a company, can avoid FDA software validation, because it’s a black box situation. Suddenly you’re talking about cost savings for the manufacturing process managers rather than how much a Cisco network costs because ethernet is needed down where the fieldbus is. That’s where we see a lot of traction - production managers making use of the infrastructure that they already have to become more efficient and distributing their machine data more broadly.
Panelist 4: At Balluff, we sell sensors, industrial networking, and RFID identification products for traceability. If you have to collect little bits of information from a bunch of prox switches, IO-Link is a more cost-effective solution for people, and it’s just been exploding over the last couple of years in implementation for these kinds of applications.
Motivations & Roadblocks for Investing in Industrial IoT
Panel Host: One of the questions I have now is why are people investing in Industrial IoT? What motivates these investments? What roadblocks are they running into? How are they calculating the ROI?
Panelist 4: Everyone talks about OEE and trying to reduce their downtime, eliminate problems. If we can apply analytics to predict when a machine will go down, that’s a major advantage. Things where you can add analytics and do predictive maintenance calculations, that brings huge value.
Keary: I’ll stick with the efficiency subject because that’s what we see too. The issue that we see surrounding it is the people in charge of efficiency, aren’t the programmers. Manufacturing process engineers are trying to get data for their KPI’s, and they’re not able to navigate controls engineering’s or IT’s concerns. So empowering those people with devices that they have control of themselves allows them to implement production processes that improve efficiency. When they’re able to control the data exchange, they are able to use log reports to identify which group needs to fix problems when they occur. A big issue is ownership. Who owns the MES? Who owns the database? Is it IT? Is it engineering? Is it production? There’s this great need to improve efficiency to be competitive in a global environment, and production needs that empowerment to have their own way to get the data without relying on calling on an integrator every time they want a PLC tag (or register) for instance. It’s these kinds of organizational issues that are road blocks to realizing the investments that are being made -- efficiency is the reason “why” that we encounter, but overcoming the OT-IT conflict is the roadblock for production managers.
Panelist 2: It’s return on investment. It’s got to hit the financial bottom line - some reduction in cost – operating expense that sort of thing – or an increase in productivity.
Panelist 1: I would add one larger theme that we have seen in GE, which is changing the business models. Changing the business model through IIoT is one of the key things.
Advice for IoT Champions
Panel Host: Behind every IoT initiative is a champion who goes to the organization and makes a case for why this is important and makes a case for the benefits it will offer to the organization. What piece of advice would you give to this champion to sell this thing to their company?
Panelist 1: In this specific stage of the market it is critical for this champion to do some POC first before choosing any specific system or vendor or platform. Second is to create the business case. Start with the value. From this value, think about the IoT solution, not vice versa.
Panelist 2: I say follow the money. So, yes, it’s value. You want your product champion or your project champion to be somebody who has a vested interest in the botPanelist 2 line.
Keary: I agree with the cost per transaction. Ultimately a manager or executive is going to say how much are we making and how much can we make. Where we see our champions have their idea accepted -- whatever they’ve chosen to push through their organization, is with proof. They must prove it. They get a concept and then what our challenge is, at least from our communications standpoint, is everybody says they can do IoT. So… IoT is this great big subject that everybody can do, but nobody can see who’s doing it, so they can take a working example to their boss. In this industry, there’s a great place where production processes are proven. Many of our end-users in medical device manufacturing prove concepts in their lab. They have a workbench; they have a person assigned to proving the process before it advances to validation. That process is mirrored down on the production floor. And, proving to them that IT will sign off on the way they’re exchanging machine data; and, proving that the controls engineers down on the machines will sign off on the solution is something that we help our champions do in the lab. The proof is seeing the data move from some part of the industrial controls systems, mostly PLCs, to a database where different parts of the organization can get to it securely.
Panelist 4: I think, as with anything, you have to find where the pain in the organization is so great that management is willing to spend money. Typically asking an operator or a line supervisor, you’re going to find that immediate pain that has direct ties to the goals of the organization.
Handling Objections from IT
Panel Host: You’ve got the champion. The champion has made the case to the business owner or the folks in the C-Suite. Now, you have another hurdle, and that is what do you say when the IT dept states they will never allow any production information to be accessed over the Internet? We’ll start with Panelist 4 and work our way back. I know Keary touched on this a little bit but go ahead and kick this off, Panelist 4.
Panelist 4: Go find that guy’s boss. If you have a pain that’s tied to the goals of the organization and IT guy says production data is not allowed to leave the production floor, you’re talking to the wrong person.
Keary: Our panel host knows that this is what we address with our appliance. Not only does IT not want production data on their network, but production also doesn’t want IT’s PCs next to their machines either. Our philosophy is that those two networks need to be separated versus converged. That infographic that we had up actually shows using an appliance rather than a PC in production; this is how to isolate those networks from each other. If you can translate the protocols of industrial control systems across an appliance and keep that fieldbus or OT network separate from the IT dept, we’re back to the ownership issue that I hinted about earlier. Now everybody can stick to their expertise and production can own the data and get the security checkbox checked off by IT, and a checkbox checked off by OT who won’t have to change their legacy solutions, because what they’re presenting to the people that are in charge of security for their organization can sign off because it’s a closed box. It’s not a general-purpose PC that brings those vulnerabilities down to the production floor, and that’s the concern of IT for security.
Panelist 2: Two things I will not say to the IT dept are I’m going to come in and break your rules [by putting in a firewall] and ask you to do work on my behalf [manage all the users for this]. Once they realize that’s not the case and that it can be done securely, yes it’s got to be financially driven, but you’ve essentially neutralized IT at that point.
Panelist 1: GE has developed an on-premise solution for Predicts. Now we’ve learned from the market that many customers Panelist 4 not share anything, so we have developed our on-premise solution. We will show you how those predictive models work with this data and that becomes the proof.
Industrial IoT Standards – Are There Any?
An attendee’s question: It seems to me that the focus should be a standard. So, is there any standard(s) because of disparities?
Panelist 4: There are a number of associations and organizations trying to build data communications standards. Some industrial protocols that are trying to address this as well. The biggest issue for me goes back to that individual prox sensor. There are so many devices that have little bits of information that we’re only interested in three or four bits of information out of a single device, but if you start doing the math on that device gives data two hundred times a second. Four bits of data doesn’t sound like very much until you start thinking about what to do with that or how often do I poll for that data. There some standards that are there, but some universal, all-encompassing standard, I haven’t seen one addressed yet.
Keary: What we’ve seen with the standards is that unfortunately, with the standards that exist, they are like OPC, for instance, they’re these client-server relationships. They’re not native to the devices you’re talking about, so they are added after the fact. Our philosophy for our end users is, they find devices that are speaking in the native protocols of the device to which they are talking. If you can have an interface that is speaking in a protocol, you don’t necessarily need to buy the software contract with the PLC manufacturer because you’re using their drivers on your appliance or gateway. If you have an edge network appliance like ours or you have other peripherals that are connecting to your devices, you should ask their companies about what their drivers are. This concept is catching on in a few other places, so ask your suppliers what drivers do you have, what protocols do you speak. Now you’re speaking natively to your device and then being a gateway to something else like a database to which you’re trying to communicate.
Panelist 2: What he said. The problem with OPC UA it’s not been around as long. People have factories full of stuff that’s not compatible. The edge node is how we usually address that.
Panelist 1: I agree with Panelist 2 in that with the maturity of the market the more standards there will be and maybe some of those standards will become more popular. It is important, but I don’t see that as a roadblock for any IoT communications.
Keary: I’d like to add one more thing to Panelist 4’s point that one sensor with that one bit of information… it’s something that will help you relative to standards, or answer your questions about how do I talk to these things? Something that will help all of you is looking for event-based communication instead of polling. You don’t want to poll a machine network because it’s looking all the time, and it’s going to get all the data, and then when it gets to where you want it to get, you’re dumping all the data you didn’t need for that one bit of information.
Panelist 4: And then comparing if it changes.
Keary: Yeah, that’s right, so, when we’re talking about the edge and those types of devices and sensors. Guess what, the part of IoT that’s not new is that sensor information is going to a machine’s controls right now. It’s called a PLC. All that information is there; it’s event-based. It’s in bit form and that size data, in that real-time type of communication, that’s dealing with deterministic networks and those types of things, is available to you through your PLCs. It just needs to be translated into a database format that can be used for the analytics that companies are seeking. While you’re doing your research, remember event-based and drivers. It will help you navigate through the sales jobs that you might receive.
Connecting Legacy Equipment
Panel Host: Those people who implement an IoT system are not going to be walking into a brand-new plant, what hurdles, difficulties, and solutions have you guys seen in connecting existing legacy equipment?
Panelist 2: The ROI equation is not manageable if you have to rip and replace controllers and spend engineering time doing that. It’s the edge node with the right protocol drivers to deal with the legacy equipment and then push it from there, event-based as Keary said, out to a server whether that’s on-premise or cloud hosted. Either way, the edge node can manage that communication to legacy equipment.
Panelist 1: What we have seen from the data analytics perspective is that customers were not often storing historical data from machines, from PLC controls. [In those cases] it doesn’t make sense at all to connect all these machines to the cloud, so that’s the most difficult from our perspective. All the other things you can resolve them somehow.
Panelist 4: We actually have that data. It’s in the guy who stands in the plant. What I think is a challenge that we start taking that data out of that guy and out of the machines and collecting the data. I agree the biggest challenge is the historian. My advice to people is to find a partner.
Keary: Relative to medical device manufacturing, it’s not just legacy systems, it’s validation. Once a process is in place, it can’t be changed even if you wanted to because it’s so cost prohibitive. The problem that we solve for our customers trying to exchange data with their machines is removing PC-based issues like Windows security updates from their production line with our MES gateway appliance. As an appliance, its embedded firmware doesn’t interfere with machine operation and is considered a “black box” by the FDA. Because the appliance doesn’t control the machine, software validation isn’t necessary for communication with the machine’s devices. If you can find an appliance that will talk to that machine’s controls and devices in its native protocols like ours, you’ll be able to communicate with legacy systems without disrupting compliance. In medical device manufacturing that’s a really -- really big deal because process validation is so expensive you typically can’t touch what’s already been approved.
Hooked on the Data Crack
Panel Host: You're putting in the IoT system to get your efficiencies to do your condition monitoring. Then what role do analytics and data science play? Is that an add-on? Is that the reason to do it? What are you guys seeing?
Panelist 1: There are different views on that in the market, and very often the companies would invest first in creating the IoT infrastructure of the company to get ready, to get prepared for some analytics. We are, and I am personally a strong believer, in the opposite approach. I would use data science right away and then from here try to develop the strategy and tactics what you want to do in the internet of things.
Keary: We see analytics start with requests for a runtime bit. Can you show me it’s just running? Is what you set up, is that acceptable to the OT and IT part of your company? Is it acceptable to production, whatever you chose for that one bit? It starts with runtime, but now can you tell me when it’s down. And then everything you chose… now we have hardware or firmware or software in place and the addiction starts. We all joke about “I got that one bit”, and when everyone’s happy with the way we got that one bit, then the dashboards start filling up because they’re hooked on the DATA CRACK. Because once you get the one, they’re [management] always asking, “how do I get another one.” “If you can give me that”, when the manager comes in the room, “that means you can get me that?”. Then everyone is able, because you have the infrastructure in place, they’re able to say, “yes we can -- yes we’d like to get that for you.”
Panelist 4: Hooked on the data crack. I’m stealing that for our future presentation, and I’ll give you credit. I just read this in an article the other day that in manufacturing, we generate four times more data than the US government generates. You have to have much more of a strategy about what you’re going to do with your data and the purpose of your data and have a plan rather than just let’s build our lake and then go fishing. You’re never going to catch anything.
Panelist 2: A couple of things here. You absolutely should plan ahead for the analytics, but at the same time, data crack. You got to get started somewhere, and it has to be profitable. Figure out what data you want through the next phase of the project and start grabbing that now while you’re getting a return on investment from phase one.
Impact of Industrial IoT on the Workforce
Panel Host: We have time for one last quick question, and that is, do your clients have the workforce to implement this once the IoT system is decided on? Does that have an impact on the workers who implement it?
Keary: Ours do because we designed the interface to be for them. They don’t need a specialized programmer to interface with our point-and-click application. And, anytime we’re helping people with whatever they use, our philosophy is to empower production. So, as an anecdote, we’ve been in meetings where… so augmented reality, right. It’s the coolest presentation you can see when they get up and start showing augmented information on everything you’re looking at. You can see the sensors. You can see the temperature of everything. But if you sit in those meetings… where is all that data going to go and where is all the information going to reside? And the programmers start saying “oh you can program it in anything, you can program it in Java, you can program Python, you can program it in any of these programs so that means anybody could do it.” And we leave those meetings, and the production guy pulls us by the elbow and says do I have to worry about this? And they don’t because they have our appliance, and it’s easy to convince them because they’re used to being able to control the data without being a programmer. That’s in our world - connecting the edge to this other side where C+ programmers are a dime a dozen and assume everybody is a programmer too.
Panelist 4: I think if you pick the right partner. Having that realization that maybe the application you’re doing right now doesn’t necessarily require it, but long term we are all going to require data scientists and software specialists in our organization, or if you’re a large organization we’re all going to have to have these skill sets in our organizations.
Panelist 2: The skillset question can be a big barrier because people will go this is something we can do someday. Because many companies don’t have the skillset internally or if they have the skillset they don’t have the time. What we recommend is a turnkey solution, so we will come in and build a reusable model.
Panelist 1: The answer to our host’s question depends on the size of the company. If it’s a larger manufacturer then they already have several data scientists within their organization, they have some software developers. If it’s a smaller company, of course, they don’t. Then the way to go is actually to start with out-of-the-box IoT solutions that you could benefit from right away and then see how you can build something on top or adjacent to this solution.