Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data layout #8

Open
apatlpo opened this issue Jan 26, 2018 · 2 comments
Open

data layout #8

apatlpo opened this issue Jan 26, 2018 · 2 comments

Comments

@apatlpo
Copy link
Contributor

apatlpo commented Jan 26, 2018

Hi,

I'm trying to understand the layout of data among MPI procs.
Could anybody please take a look at the following pics in order to see if I got it right?

@mikaem
Copy link
Member

mikaem commented Jan 26, 2018

Hi
The array layouts are actually possible to configure when creating the TensorProductSpace. The keyword argument axes can be used to determine the order of which the different axes are transformed, and this determines also the decomposition. Furthermore, slab=True can be used to decompose in a 1- instead of 2-dimensional process grid. The default is to use the pencil method, though, which is what you are drawing.
From what I can tell the 2D pics are correct with default settings. I'm also pretty sure the 3D plots are correct. In case you're not aware, you can double check by printing the parts of the global array that belongs on each cpu as
print(comm.Get_rank(), T.local_slice(spectral=False))
print(comm.Get_rank(), T.local_slice(spectral=True))

@apatlpo
Copy link
Contributor Author

apatlpo commented Jan 26, 2018

Ok, thanks, I also found local_wavenumbers and local_mesh to be useful in order to inspect the layout:

In 2D:

X = T.local_mesh(True)
K = T.local_wavenumbers()
print(comm.Get_rank(), ' spectral kx=', K[0][[0, -1]], ', ky =', K[1][0, [0, -1]])  
print(comm.Get_rank(), ' physical x=', X[0][[0, -1], 0], ', y =', X[1][0, [0, -1]]) 

And in 3D:

X = T.local_mesh(True)
K = T.local_wavenumbers()
print(comm.Get_rank(), ' spectral kx=',K[0][[0,-1],0,0],', ky =',K[1][0,[0,-1],0],' kz =',K[2][0,0,[0,-1]]) 
print(comm.Get_rank(), ' physical x=',X[0][[0,-1],0,0],', y =',X[1][0,[0,-1],0],' z =',X[2][0,0,[0,-1]])     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants