Information extraction

[andy-yoo]

Hi, A total Visit newbie here.
I have a very simple task: read an unstructured 3D mesh in silo file and extract information on each zone (I don't know the exact term Visit community uses) in the mesh including

1. Zone ID (it doesn't have to be explicit as long as there is a way to derive that)
2. Its bounding box information. That is Xmin, Xmax, Ymin, Ymax, Zmin, Zmax.
   I prefer to get this information in an editor-friendly format like ASCII.

I have followed Visit tutorial and exported the silo to a VTK file, but the content was very cryptic.
Visit exports the file in VTK 5.1 format, but it appears that there is no documentation about this latest format and I am scratching my head at this point.

Could you please help me pointing me to right direction?
Thank you so much for your help in advance.

[markcmiller86]

@andy-yoo can you say what "scale" of mesh this is in terms of zone count? I mean is it a few hundred zones or a bazillion zones? Next, for item 2, do you want the spatial bounding box of each zone or just of the whole mesh?

[andy-yoo]

First, Thank you so much speedy response. Wow, I expected to see any response tomorrow morning. :)

The mesh has about 8 million zones.
I need spatial bounding information of each zone. I found the bounding info for entire mesh.

Thanks again.

[markcmiller86]

Ok, I am assuming because the mesh is unstructured, each zone does not have faces that are orthogonal to the cartesian axes. Otherwise, the spatial bounds of each zone would, in that highly simplified case, be determined by the nodal coordinates of the 8 nodes of each zone. But, now that I read my own writing here, I just wanna check that you are not using "bounds" in item 2 as a proxy for "give me the nodal coordinates of all the nodes comprising each zone".

[andy-yoo]

I am not very familiar with this mesh as it is one of test meshes we use for our own testing.
Although it does not appear that each zone has 8 nodes (we refer them as subzones), let me ask someone to clarify first instead of giving you false information.

My guess would be "No, it is not the nodal coordinates of the nodes comprising each zone".

To make it clear, will I see information on such nodes, if exist, in the VTK file that I exported?

Thanks.

[markcmiller86]

Well, right, in general, an unstructured mesh can have hexahedra (8 noded things) and/or tetrahedra (4 noded things), wedges, pyramids or (in wild cases) arbitrary polyhedra.

Yes, if you happened to export to VTK, you would see in that VTK file, all the nodal coordinate data (output in the POINT DATA portion of the file).

So, I think the only way to do what you want to do is going to be to use the python command-line interface (CLI) to VisIt and/or use VisIt python expressions.

The challenge I have with using just python CLI (and not using python expressions), is that you would wind up having to iterate over your "millions of zones", plotting just the one zone of interest, querying it for spatial extents. I suspect that iterating over all zones would take hours. Now, you could accelerate that by parallelizing it by running multple instances and each instance iterates over some of the zones in the mesh.

Now that I think about it, what is probably best is to take the VTK data and read it into python using either VTK/python tools or some straight python if you are willing to code up reading the VTK data in. Then, you can just iterate over each zone there, examing the zone's nodes and computing there the spatial bounds of each zone. Have you considered that already?

[cyrush]

You can export to simple CSV files using the Blueprint Plugin.

But with that path, you get the centroid of each zone -- not the bounding box of each zone.

You could also export mesh as yaml files via Blueprint Plugin, but anything that preserves the mesh topology is going to be complex b/c the topology itself is complex.

[andy-yoo]

First, it appears that each zone in the mesh has nodes (subzones?) associated with.
From this information I think I can deduce the bounding box information for each zone by the approach you suggested, which is to write a python script to parse the VTK file.
The problem and somewhat weird thing is that from what I see in the VTK file each zone has 9 nodes, not 8.
I don't believe the number of nodes is actually 9, as some points for some zones are duplicate.
Not sure if you are familiar with VTK 5.1 format, but this is weird.
Have you seen something like this?

Exporting to Blueprint CSV file can be an option, but from what you described, it will be impossible to extract boundary information, it seems.
Again, I really appreciate your help.

[JustinPrivitera]

VisIt beta on LC at /usr/gapps/visit/beta/bin/visit has yaml option for Blueprint export.

[JustinPrivitera]

I'm sorry; that's not true. My mistake.

[andy-yoo]

I managed to export the mesh to Blueprint HDF5 format, which I managed to convert to a YAML (as well as JSON) file. I think I know how to interpret the file, but I would like to confirm with you. Wrong information is worse than no information.

Basically, there are 64M integer values in the list associated with "connectivity" key. I interpret this as, the first 8 values are indices of points comprising the first zone, the next 8 values are point indices for the second zone, etc. This means there are 8M zones in the mesh.

I located three key-value pairs in "coordsets" section of the file, labeled with "x", "y", and "z" respectively. Associated with these keys are lists of length 8120601. I interpret this as: there are 8120601 points used in the mesh and (x[0], y[0], z[0]) is the coordinate of the first point, and (x[1], y[1], z[1]) is the coordinate of the second point, etc. And I can get boundary of each zone from this information.

Is my interpretation correct?

[JustinPrivitera]

It would be better to use the Conduit Python API to digest these files and get at the data.

Have a look at the examples here: https://llnl-conduit.readthedocs.io/en/latest/blueprint_mesh.html#tris

I am guessing you are dealing with an unstructured mesh. The topologies/toponame/elements/shape entry should inform you as to how many points there are in each cell. The number of points for each name is described here: https://llnl-conduit.readthedocs.io/en/latest/blueprint_mesh.html#topologies You can use that information to walk the cells. The entries in connectivity point to <x,y,z> coordinates, like you describe.

[JustinPrivitera]

Try this to get an overview of your data:

/usr/gapps/visit/bin/visit -nowin -cli -s look_at_blueprint.py <your_blueprint_data_file.root>

where look_at_blueprint.py is the following script:

import conduit
import sys

mesh = conduit.Node()
opts = conduit.Node()

if len(sys.argv) < 2:
	print("Provide a filename.")
	exit()

filename = sys.argv[1]

do_yaml = False
provide_meshname = False
meshname = ""

for arg in sys.argv[1:]:
	if arg == "yaml":
		do_yaml = True
	if "mesh_name=" in arg:
		provide_meshname = True
		meshname = arg[10:]
	if "meshname=" in arg:
		provide_meshname = True
		meshname = arg[9:]

if provide_meshname:
	opts["mesh_name"] = meshname
	conduit.relay.io.blueprint.load_mesh(mesh, filename, opts)
else:
	conduit.relay.io.blueprint.load_mesh(mesh, filename)

verify_info = conduit.Node()
if not conduit.blueprint.mesh.verify(mesh, verify_info):
	print("does not verify")
	print(verify_info)

if do_yaml:
	print(mesh.to_yaml())
else:
	print(mesh)

# conduit.relay.io.blueprint.save_mesh(mesh, "name", "hdf5")

run

/usr/gapps/visit/bin/visit -nowin -cli -s look_at_blueprint.py <your_blueprint_data_file.root> yaml

if you want to see everything in your file in the terminal... I doubt you do because you have millions of zones.

[andy-yoo]

I will take it back.
The mesh does not have any subzones.
However, 9 nodes per zone still remains as mystery.

[cyrush]

Are you parsing the offsets in the cell section?

It is strange there are 9 entries. There isn't much documentation about the new VTK ASCII formats
(in fact, we are reverting to export older style files in future VisIt release due to this)

https://discourse.paraview.org/t/specification-of-vtk-datafile-version-5-1/5127

It may be that the offsets per zone cover the entire range.

If you export to blueprint yaml, you will have 8 indices per hex.

[andy-yoo]

I managed to export the mesh to Blueprint HDF5 format, which I managed to convert to a YAML (as well as JSON) file. I think I know how to interpret the file, but I would like to confirm with you. Wrong information is worse than no information.

Basically, there are 64M integer values in the list associated with "connectivity" key. I interpret this as, the first 8 values are indices of points comprising the first zone, the next 8 values are point indices for the second zone, etc. This means there are 8M zones in the mesh.

I located three key-value pairs in "coordsets" section of the file, labeled with "x", "y", and "z" respectively. Associated with these keys are lists of length 8120601. I interpret this as: there are 8120601 points used in the mesh and (x[0], y[0], z[0]) is the coordinate of the first point, and (x[1], y[1], z[1]) is the coordinate of the second point, etc. And I can get boundary of each zone from this information.

Is my interpretation correct?