Parse value with a (at compile time) unknown structured DataType

[Nick135]

The UA Client has a DataTypeManager to pass through the extracted ExtensionObject from the DataValue.

Is there something simular in this Lib?

[Nick135]

Structure shown by UAExpert:

Read the data works, but then I have only the Byte[] from the ExtensionObject.Body

How parse or get the underlying node identifier?

[awcullen]

Great question.
In my work, I have not needed a method to decode an unknown structure from a ua server. All the custom types for my PLC's are known ahead of time, and I create a custom TypeLibrary for handling these types.

As UaExpert shows, the necessary information exists to decode the custom, unknown structure. In the current best practice, custom structures are documented by reading an attribute of the DataType named DataTypeDefinition. DataTypeDefinition is an object of type StructureDefinition and contains the structure's DefaultEncodingID and list of fields.

I could imagine a method that would decode the unknown ExtensionObject ( a NodeId and byte array) to a dictionary that stores the fields of the structure.

[Nick135]

On the Siemens Website there is a example with a simple unknown sturct. They use the Opc.Ua.Core of the OPC Foundation.
In this lib you can read all DataTypeDefinition as XML per Browse very fast.

I´m not sure how to make this with this lib. I test this code, but this is very slow.

Console.WriteLine("Step - Browse WinderDat.");
Console.WriteLine("+ \"WinderDat\".\"windingPar\".\"coil\"");

BrowseRequest browseReq = new BrowseRequest
{
	NodesToBrowse = new BrowseDescription[] {
		new BrowseDescription {
			NodeId = NodeId.Parse("ns=3;s=\"WinderDat\".\"windingPar\".\"coil\""),
			BrowseDirection = BrowseDirection.Forward,
			ReferenceTypeId = NodeId.Parse(ReferenceTypeIds.HierarchicalReferences),
			NodeClassMask = (uint)NodeClass.Variable | (uint)NodeClass.Object | (uint)NodeClass.Method,
			IncludeSubtypes = true,
			ResultMask = (uint)BrowseResultMask.All
		}
	},
};

await BrowseToConsole(browseReq);
async Task BrowseToConsole(BrowseRequest browseRequest, string indent = "")
{
	try
	{
		string indentOffset = "  ";
		indent += indentOffset;
		BrowseResponse browseResponse = await channel.BrowseAsync(browseRequest);
		foreach (var rd in browseResponse.Results[0].References ?? new ReferenceDescription[0])
		{
			Console.WriteLine("{0}+ {1}: {2}, {3}, {4}, {5}", indent, rd.NodeId.NodeId.Identifier.ToString().Split('.').Last(), rd.DisplayName, rd.BrowseName, rd.NodeClass, rd.NodeId);
			var bReq = new BrowseRequest
			{
				NodesToBrowse = new BrowseDescription[] { new BrowseDescription { NodeId = ExpandedNodeId.ToNodeId(rd.NodeId, channel.NamespaceUris), BrowseDirection = BrowseDirection.Forward, ReferenceTypeId = NodeId.Parse(ReferenceTypeIds.HierarchicalReferences), NodeClassMask = (uint)NodeClass.Variable | (uint)NodeClass.Object | (uint)NodeClass.Method, IncludeSubtypes = true, ResultMask = (uint)BrowseResultMask.All } },
			};
			await BrowseToConsole(bReq, indent);
		}
	}
	catch (Exception ex)
	{
		if (System.Diagnostics.Debugger.IsAttached)
			System.Diagnostics.Debugger.Break();
	}
}
Console.WriteLine("BrowseToConsole");

[quinmars]

The Siemens OPC server gives you different options to access the PLC data. You are mixing some concepts here. Suppose you have a global variable with the name Global, which is a structured type with the fields A and B. To get those values you could read the value (note: it's singular) of the Global variable, i.e. ReadAsync with the (one) node id of the Global variable. Here you need to know how to decode the whole structure. That is what UaExpert is doing in your screenshot and where Andrew showed you a possible starting point. The second option is that the Siemens OPC UA server also exposes the fields A and B (of the Global instance) as single variables. Both accessible by their individual node ID. Futhermore their node IDs can be browsed by browsing the variable Global. That is what your code example does. Both ways are viable. The question is what you want to achieve. What is actual intention?

[Nick135]

I need to write a complex struct to the plc in minimum time. The struct should not be completely fixed (Hard coded on the Application side).

So my plan is to read the complete struct, Parse it, override the needed Nodes and write the complete struct as ByteString back to the PLC.

This example works very fast, but I only get the ByteString (like in the Siemens Example).
In the Siemens Example they parse the byteArray per UDT Deaclaration (read per Browse from the PLC)

var readRequest = new ReadRequest
{
    // set the NodesToRead to an array of ReadValueIds.
    NodesToRead = new[] {
        // construct a ReadValueId from a NodeId and AttributeId.
        new ReadValueId
        {
            NodeId=NodeId.Parse("ns=3;s=\"WinderDat\".\"windingPar\".\"coil\""),
            AttributeId= AttributeIds.Value
        }
    }
};

// send the ReadRequest to the server.
var readResult = await channel.ReadAsync(readRequest);
var coil = readResult.Results[0].GetValueOrDefault<ExtensionObject>();

[Nick135]

I can imagine it like this:

//Read struct
var coil= ReadStruct("ns=3;s=\"WinderDat\".\"windingPar\".\"coil\");

//Change Values
coil["[0]"]["layer"]["[0]"]["enable"] = true;
coil["[0]"]["layer"]["[0]"]["winding"]["[0]"]["wireBreakForce"] = 123.567;

//Write struct
WriteStruct(coil);

[quinmars]

I see, so you have a deeply structured variable (WinderDat.windingPar.coil) where you do not know the full datatype definition upfront. I imagine that the definition could be changed very dynamical. At least that's the case with my collegues, when they are at the customer fab. But you want to change some known fields of the structure variable. If that's the case, than the fastest way is to write those values in a single write request. Something like that:

var writeRequest = new WriteRequest
{
    NodesToWrite = new[]
    {
        new WriteValue { AttributeId = AttributeIds.Value, NodeId = NodeId.Parse("ns=3;s=\"WinderDat\".\"windingPar\".\"coil[0]\".\"layer[0]\".\"enable\""), Value =  true},
        new WriteValue { AttributeId = AttributeIds.Value, NodeId = NodeId.Parse("ns=3;s=\"WinderDat\".\"windingPar\".\"coil[0]\".\"layer[0]\".\"winding[0]\".\wireBreakForce"\""), Value =  123.56700},
    }
};

As a rule of thumb, every service request is very expensive, independent of its content. Hence, to speed your communcation up, try to put as many as possible jobs into one service request (here the WriteValue entries). Even if you list there hundreds of variables and values, it is probably faster, than to query first the structure of the variable, reading the structured variable and writing back the structured variable.

[Nick135]

In your example, is it important to know the data type of the Value?
I can set "1" to a byte, int16, int32, fload, double,...
When I read the Struct first I know the declaration data type from the PLC.

new WriteValue { AttributeId = AttributeIds.Value, NodeId = NodeId.Parse("ns=3;s=\"WinderDat\".\"windingPar\".\"coil[0]\".\"layer[0]\".\"enable\""), Value=new DataValue(true)},

[quinmars]

Maybe you can describe your scenario a little bit more. I'm still just guessing what you intent to do. Apparently, I'm not good at guessing.

[Nick135]

About the project:
The PLC must execute a specified movement, which is calculated externally using Matlab. The result of the calculation must then be sent to the controller. And that in a reasonable amount of time.
Since OPC UA is becoming more and more popular, we decided to use this standard for transmission.
The UaExpert needs 500ms for the transmission of the nested structure from a total of approx. 5300 fields.

Since the Matlab program and the PLC program are written and maintained by different people, the structure should only be roughly defined, but it should be possible to expand it in the PLC at any time. Also in Matlab you shouldn't have to pay attention to the exact data type. The main thing is that it has been chosen large enough by the PLC programmer.

Hence the approach to read the structure once in order to determine the current structure and the variable types. This should happen in the C # connector. The structure is then filled with the current values ​​and then transferred back to the controller via the connector. The C # Connector does the TypeCast.

Hope this make it a little more clear and my english is not too bad. ;-)

[awcullen]

Thanks for this information. After reading the array of "typeCoilPar" structures, you wish to convert the ExtensionObject to a Dictionary<String, Object>. You would then alter the dictionary values and write the result back to the PLC. At this time, we do not have a method to do this. but I can see that the required information is available in the DataTypeDefinition.

Since you describe that you wish to compute a result in Matlab and write it to the PLC in minimum time, I like @quinmars suggestion to address the fields directly with NodeIDs that you discover after first connecting.

Another approach is to have the PLC engineer use TIA Portal and export the UA server nodeset file.

Then see @quinmars tool UaTypeGenerator that can extract the datatypes and produce a library of types as a .cs file that you include in your project.

Then you would be able to code:

var coil = readResult.Results[0].GetValueOrDefault<typeCoilPar[]>();

[quinmars]

Using the UaTypeGenerator is propably easiest and most elegant option, but it requires that PLC and the C# program are in sync all the time. I can understand that @Nick135 cannot guarantee that. With that amount of data and short target write times, I would probably also use the initial approach, i.e., query the data type definition once, and use a dictionary than.

Let me scatch how this could look like:

class FieldDefinition
{
    public string Name { get; }
    public VariantType Type { get; }
    public int[] ArrayDimensions { get; }
    // The constructor is missing here
}

[BinaryEncodingId(/*...*/)]
public class Coil : Structure
{
    private FieldDefinition[] _fieldDefinitions;

    public static async Task ReadDatatypeDefinitionAsync(IRequestChannel channel)
    {
          // TBD
        _filedDefinitions = null;
    }

    public Dictionary<string, object> Fields { get; } = new Dictionary<string, object>();

    public override void Encode(IEncoder encoder)
    {
        foreach (var d in _fieldDefinitions)
        {
            var val = Fields[d.Name];
            switch (d.Type)
            {
                case VariantType.Int16:
                    encoder.WriteInt16(null, Convert.ToInt16(val));
                    break;
                case VariantType.Int32:
                    encoder.WriteInt32(null, Convert.ToInt32(val));
                    break;
                default:
                    throw new Exception("Unsupported type");
            }
        }
    }

    public override void Decode(IEncoder encoder)
    {
        foreach (var d in _fieldDefinitions)
        {
            switch (d.Type)
            {
                case VariantType.Int16:
                    Fields[d.Name] = decoder.ReadInt16(null);
                    break;
                case VariantType.Int32:
                    Fields[d.Name] = decoder.ReadInt32(null);
                    break;
                default:
                    throw new Exception("Unsupported type");
            }
        }
    }        
}

Of course this is a very rough scatch. It's missing some essentials parts, like the creation of the field definition array. It only supports two integer types, no arrays, no nested types. Arrays, nested types and arrays of structured types could decay into single fields. So you would access them through the base value: coil.Fields["layer[0].winding[0].wireBreakForce"] = 12.3. That makes the encoding and decoding easier, maybe the field definition array creation a little bit more involved. Much to do, but it could be a starting point for you.

[awcullen]

@quinmars , we should continue refining this approach with some test data on a test server. The UaCPPServer has a variable "ns=3;s=ControllerConfigurations" which has a structure having arrays of other structures.

@Nick135, consider refactoring the problem to your advantage. Create two new structures that exactly represent the inputs and outputs of your Matlab function. Have the PLC engineer create the i/o datablocks in the PLC. The PLC engineer would then be responsible for adapting the results of your Matlab i/o datablocks to the WinderDat block. The engineer would be free to continue altering the WinderDat structure without worry that your solution code would stop working.

[Nick135]

@awcullen,

1. The struct contains actually round about 5300 Values. It´s not good to hold this struct double in the plc, it´s Siemens so memory is rare ;-) also the cost of move data from struct to struct.
2. Three different disciplines are working on the project. So when the PLC programmer adds a Variable, change a data type or expand a array, I don´t wanna change the connector (C#)

@quinmars, My first idea was to create a Node for each variable (like XmlNodes). But maybe your approach with a simple and one-dimensional dictionary is more easy to implement.

How can I browse all declarated UDTs from the PLC? My browse example was very slow and hat a lot of loops.
In the Siemens Example with OPC.UA.Client they only need three browse calls to get a complete XML document of the defined UDTs. (I´m not sure if there are all or only used UDTs, but i need only 1.5 seconds to have all required info to parse the ByteString)

[awcullen]

In modern UA servers, you may find the structures documented in two locations. The original location is to find documents called 'DataTypeDictionary' stored in a location under DataTypes... OPC Binary

The node 'BA' is an example of a 'DataTypeDictionary'. It is variable of type ByteString, and if you read the value and convert the bytes to a string, you will find it is an XML document that documents the data types 'ControllerConfiguration', etc

The second location is to read the DataTypeDefinition attribute of the data type itself.

[quinmars]

@Nick135 , I guess that you get the data from Matlab in some form of a list (even if it is a tree, it can be easily transformed to a list). Mapping a list to another list is much simpler than to map two trees. Mapping a list to a dictionary is even more simple. That's why I think a flat dictionary is in your case the prefered solution. Of course a tree structure is doable as well, but keep in mind that you than need many type checks to see if a field is an 1d-array to a single value type, an 2d-array to a single value type, a structured value, an 1d-array of a structured type, etc.

A brief excursion. For the UaTypeGenerator I could see an similar approach, to support later additions to the contract. There the access to the dictionary would happen via the typed properties. Hence I would there use indeed a tree based structure.

Back to topic. Accessing the data type definition with the known data type could become slow if you have many nested data types. I could imagine that it is faster to query all nodes that have a "NodeClass" of "Datatype" in advance. I haven't used the query service set yet though. On the other hand, those server access will only happen on start up and have no effect on the cycle time.

we should continue refining this approach with some test data on a test server. The UaCPPServer has a variable "ns=3;s=ControllerConfigurations" which has a structure having arrays of other structures.

@awcullen Good idea. Would we add an example to opc-ua-samples?