Solved: Where to store large arrays of REAL data?

maj9bj · ‎08-25-2022

For our parts we measure some values during assembly cycle. Result of this measurement are 3 arrays [1..1200] of REAL values (eg. current, voltage, resistance). Our customer wants to save this data for every part that means 3600 values for every assebmly cycle.

My question is how and were to store these data?

I tried to add these arrays to PartProcessed > SendBody > Array and send it to MES. I only had to change send buffer size and library constant in TwinCAT and it works and all 3600 values are send.

Then I was talking with colleague from MES team what is next step and he told me this.

The values cannot be processed in MES, bacuse in Array section he expects only arrays of standard ResultDataXXX structures. I do not want to use array of ResultRealData, because I do not need other information like limits, unit typ, process name and with usage of this structure the size of date will be much large.
MES database should not be used for storing of these big amount of data for every produced part.

Is there any other solution how to deal with that request?

nexidator · ‎08-29-2022

This is the answer I got from a colleague from the MES department:

The way of connecting the data also depends on your later usage in MES. For using this kind data you have different possibilities in MES.

Mostly QualityData is the module which will be filled with this kind of data in order to fulfill traceability
But you can also use our module Process Quality. This module is optimized for curve data. Please have also a look at this, too.
Or you can use a condition monitoring. For this we have 2 modules (CMControl and Condition Monitoring)

There are different ways to process such amount of data in MES:

Via XML message
- A “simple list” as shown in the original message
  This can be used for MES QualityData. For other modules it’s hard you convert the data into a usable format, since the shown way (below) is only available within QualityData.
- As mentioned from an MES application engineer we need to have the ProcessReal-Arrays to save data, but we have some “helpers” to convert them. So we have 2 choices here
  - To use the native array and on MES side we have to add some setting to generate a parameter name for those values. With this they will be named Station.MesTestingArray[1] … Station.MesTestingArray[3600]
    The array has to be defined like this in OSS (MES configuration tool)
    
    And used like this to store to QualityData like usual measurements. The red marked values have to be filled as shown, the blue marked checkbox in same cases. Please give a try for blue option which works better in your case.
- You can convert them to a ProcessRealArray on station side, using only name, value and datatype (optional). You can skip other not necessary content like limits here.
  - This is the recommended way, since you are achieve the highest flexibility in MES to use the data in different modules
    You can use it for QualityData, Process Quality and both ConditionMonitoring modules
    <arrays>
      <array>
        <array name="Station.MesTestingArray">
          <item name="Station.MesTestingArray[1]" value"0" dataType="4">
          <item name="Station.MesTestingArray[2]" value"0" dataType="4">
          <item name="Station.MesTestingArray[3]" value"0" dataType="4">
          <item name="Station.MesTestingArray[4]" value"0" dataType="4">
          ...
        </array>
      </array>
    </arrays>
- It is also possible to save the data as a 2D curve.
  - This can be used in QualityData only.
  - There is a defined structure for this. But there is no “standard” visualization in the web for it. To get this visualized you need a customized view, which needs to be implemented especially to your system. Please check if this can be done, before using this option.
    <structArrays>
      <curve2D>
        <structs>
          <head version="1" name="myCurve" result="0"/>
        </structs>
        <structArrays>
          <array name="axis">
            <structDef>
              <item name="name" dataType="8"/>
              <item name="axisType" dataType="8"/>
              <item name="unit" dataType="8"/>
            </structDef>
            <values>
              <item name="time" axixType="X" unit="s"/>
              <item name="pressure" axisType="Y" unit="bar"/>
            </values>
          </array>
          <array name="characteristicDiagram">
            <structDef>
              <item name="X" dataType="4"/>
              <item name="Y" dataType="4"/>
              <item name="lLY" dataType="4"/>
              <item name="ulY" dataType="4"/>
            </structDef>
            <values>
              <item X="5.0" Y="3.0" llY="2.8" uLY="3.3"/>
              <item X="6.0" Y="2.9" llY="2.9" uLY="3.4"/>
              <item X="7.0" Y="3.1" llY="3.0" uLY="3.5"/>
              <item X="8.0" Y="3.2" llY="3.1" uLY="3.6"/>
              <item X="9.0" Y="3.4" llY="3.2" uLY="3.7"/>
            </values>
          </array>
        </structArrays>
      </curve2D>
    </structArrays>
As already mentioned you can save them as binary file (e.g. *.csv)
- This can be used in QualityData only.
- They can be moved with CyclicUpdate and added to QualtiyData as file as some kind of “attachment” to the identifier.
  This binary file will also be included to the archiving process
PPMP
- Optimized protocol for curves and usage in ProcessQuality and ConditionMonitoring, no other MES modules could use this at the moment
  https://de.wikipedia.org/wiki/Production_Performance_Management_Protocol

maj9bj · ‎08-30-2022

@nexidator thank you so much for your detailed reply! 👍

I will try one of these solutions and later I'll put here some feedback for other users.

Where to store large arrays of REAL data?

Where to store large arrays of REAL data?

Still looking for something?