Skip to content

Latest commit

 

History

History
845 lines (739 loc) · 52.3 KB

examples.rst

File metadata and controls

845 lines (739 loc) · 52.3 KB
Raw

Block, Block

A (5 X 9) array, with a Block, Block ('b' X 'b') distribution over a (3 X 1) process grid.

images/plot_block_block_3x1.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_block_block_3x1_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 3,
  'size': 5,
  'start': 0,
  'stop': 2},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 9,
  'start': 0,
  'stop': 9})

In process (1, 0):

>>> distbuffer['buffer']
array([[ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 3,
  'size': 5,
  'start': 2,
  'stop': 4},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 9,
  'start': 0,
  'stop': 9})

In process (2, 0):

>>> distbuffer['buffer']
array([[ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 2,
  'proc_grid_size': 3,
  'size': 5,
  'start': 4,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 9,
  'start': 0,
  'stop': 9})

Block, Block

A (5 X 9) array, with a Block, Block ('b' X 'b') distribution over a (1 X 3) process grid.

images/plot_block_block_1x3.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_block_block_1x3_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   1.,   2.],
       [  9.,  10.,  11.],
       [ 18.,  19.,  20.],
       [ 27.,  28.,  29.],
       [ 36.,  37.,  38.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 5,
  'start': 0,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 3,
  'size': 9,
  'start': 0,
  'stop': 3})

In process (0, 1):

>>> distbuffer['buffer']
array([[  3.,   4.,   5.],
       [ 12.,  13.,  14.],
       [ 21.,  22.,  23.],
       [ 30.,  31.,  32.],
       [ 39.,  40.,  41.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 5,
  'start': 0,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 3,
  'size': 9,
  'start': 3,
  'stop': 6})

In process (0, 2):

>>> distbuffer['buffer']
array([[  6.,   7.,   8.],
       [ 15.,  16.,  17.],
       [ 24.,  25.,  26.],
       [ 33.,  34.,  35.],
       [ 42.,  43.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 1,
  'size': 5,
  'start': 0,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 2,
  'proc_grid_size': 3,
  'size': 9,
  'start': 6,
  'stop': 9})

Block, Block

A (5 X 9) array, with a Block, Block ('b' X 'b') distribution over a (2 X 2) process grid.

images/plot_block_block_2x2.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_block_block_2x2_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   1.,   2.,   3.,   4.],
       [  9.,  10.,  11.,  12.,  13.],
       [ 18.,  19.,  20.,  21.,  22.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 3},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5})

In process (0, 1):

>>> distbuffer['buffer']
array([[  5.,   6.,   7.,   8.],
       [ 14.,  15.,  16.,  17.],
       [ 23.,  24.,  25.,  26.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 3},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9})

In process (1, 0):

>>> distbuffer['buffer']
array([[ 27.,  28.,  29.,  30.,  31.],
       [ 36.,  37.,  38.,  39.,  40.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 3,
  'stop': 5},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 32.,  33.,  34.,  35.],
       [ 41.,  42.,  43.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 3,
  'stop': 5},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9})

Block, Cyclic

A (5 X 9) array, with a Block, Cyclic ('b' X 'c') distribution over a (2 X 2) process grid.

images/plot_block_cyclic.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_block_cyclic_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   2.,   4.,   6.,   8.],
       [  9.,  11.,  13.,  15.,  17.],
       [ 18.,  20.,  22.,  24.,  26.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 3},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (0, 1):

>>> distbuffer['buffer']
array([[  1.,   3.,   5.,   7.],
       [ 10.,  12.,  14.,  16.],
       [ 19.,  21.,  23.,  25.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 3},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 1})

In process (1, 0):

>>> distbuffer['buffer']
array([[ 27.,  29.,  31.,  33.,  35.],
       [ 36.,  38.,  40.,  42.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 3,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 28.,  30.,  32.,  34.],
       [ 37.,  39.,  41.,  43.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 3,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 1})

Cyclic, Cyclic

A (5 X 9) array, with a Cyclic, Cyclic ('c' X 'c') distribution over a (2 X 2) process grid.

images/plot_cyclic_cyclic.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_cyclic_cyclic_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   2.,   4.,   6.,   8.],
       [ 18.,  20.,  22.,  24.,  26.],
       [ 36.,  38.,  40.,  42.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (0, 1):

>>> distbuffer['buffer']
array([[  1.,   3.,   5.,   7.],
       [ 19.,  21.,  23.,  25.],
       [ 37.,  39.,  41.,  43.]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 1})

In process (1, 0):

>>> distbuffer['buffer']
array([[  9.,  11.,  13.,  15.,  17.],
       [ 27.,  29.,  31.,  33.,  35.]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 10.,  12.,  14.,  16.],
       [ 28.,  30.,  32.,  34.]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 1})

Irregular-Block, Irregular-Block

A (5 X 9) array, with an Irregular-Block, Irregular-Block ('b' X 'b') distribution over a (2 X 2) process grid.

images/plot_irregularblock_irregularblock.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_irregularblock_irregularblock_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[ 0.,  1.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 1},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 2})

In process (0, 1):

>>> distbuffer['buffer']
array([[ 2.,  3.,  4.,  5.,  6.,  7.,  8.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0,
  'stop': 1},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 2,
  'stop': 9})

In process (1, 0):

>>> distbuffer['buffer']
array([[  9.,  10.],
       [ 18.,  19.],
       [ 27.,  28.],
       [ 36.,  37.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 2})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 38.,  39.,  40.,  41.,  42.,  43.,  44.]])
>>> distbuffer['dim_data']
({'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1,
  'stop': 5},
 {'dist_type': 'b',
  'padding': [0, 0],
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 2,
  'stop': 9})

Block-Cyclic, Block-Cyclic

A (5 X 9) array, with a Block-Cyclic, Block-Cyclic ('c' X 'c') distribution over a (2 X 2) process grid.

images/plot_blockcyclic_blockcyclic.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_blockcyclic_blockcyclic_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[  0.,   1.,   4.,   5.,   8.],
       [  9.,  10.,  13.,  14.,  17.],
       [ 36.,  37.,  40.,  41.,  44.]])
>>> distbuffer['dim_data']
({'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (0, 1):

>>> distbuffer['buffer']
array([[  2.,   3.,   6.,   7.],
       [ 11.,  12.,  15.,  16.],
       [ 38.,  39.,  42.,  43.]])
>>> distbuffer['dim_data']
({'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 2})

In process (1, 0):

>>> distbuffer['buffer']
array([[ 18.,  19.,  22.,  23.,  26.],
       [ 27.,  28.,  31.,  32.,  35.]])
>>> distbuffer['dim_data']
({'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 2},
 {'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 20.,  21.,  24.,  25.],
       [ 29.,  30.,  33.,  34.]])
>>> distbuffer['dim_data']
({'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 2},
 {'block_size': 2,
  'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 2})

Unstructured, Unstructured

A (5 X 9) array, with an Unstructured, Unstructured ('u' X 'u') distribution over a (2 X 2) process grid.

images/plot_unstruct_unstruct.png

The full (undistributed) array:

>>> full_array
array([[  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.],
       [  9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.],
       [ 18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.],
       [ 27.,  28.,  29.,  30.,  31.,  32.,  33.,  34.,  35.],
       [ 36.,  37.,  38.,  39.,  40.,  41.,  42.,  43.,  44.]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

images/plot_unstruct_unstruct_local.png

In process (0, 0):

>>> distbuffer['buffer']
array([[ 29.,  30.,  34.,  28.],
       [  2.,   3.,   7.,   1.]])
>>> distbuffer['dim_data']
({'dist_type': 'u',
  'indices': array([3, 0]),
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5},
 {'dist_type': 'u',
  'indices': array([2, 3, 7, 1]),
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9})

In process (0, 1):

>>> distbuffer['buffer']
array([[ 33.,  32.,  35.,  27.,  31.],
       [  6.,   5.,   8.,   0.,   4.]])
>>> distbuffer['dim_data']
({'dist_type': 'u',
  'indices': array([3, 0]),
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5},
 {'dist_type': 'u',
  'indices': array([6, 5, 8, 0, 4]),
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9})

In process (1, 0):

>>> distbuffer['buffer']
array([[ 38.,  39.,  43.,  37.],
       [ 20.,  21.,  25.,  19.],
       [ 11.,  12.,  16.,  10.]])
>>> distbuffer['dim_data']
({'dist_type': 'u',
  'indices': array([4, 2, 1]),
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5},
 {'dist_type': 'u',
  'indices': array([2, 3, 7, 1]),
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9})

In process (1, 1):

>>> distbuffer['buffer']
array([[ 42.,  41.,  44.,  36.,  40.],
       [ 24.,  23.,  26.,  18.,  22.],
       [ 15.,  14.,  17.,   9.,  13.]])
>>> distbuffer['dim_data']
({'dist_type': 'u',
  'indices': array([4, 2, 1]),
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5},
 {'dist_type': 'u',
  'indices': array([6, 5, 8, 0, 4]),
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9})

Cyclic, Block, Cyclic

A (5 X 9 X 3) array, with a Cyclic, Block, Cyclic ('c' X 'b' X 'c') distribution over a (2 X 2 X 2) process grid.

The full (undistributed) array:

>>> full_array
array([[[   0.,    1.,    2.],
        [   3.,    4.,    5.],
        [   6.,    7.,    8.],
        [   9.,   10.,   11.],
        [  12.,   13.,   14.],
        [  15.,   16.,   17.],
        [  18.,   19.,   20.],
        [  21.,   22.,   23.],
        [  24.,   25.,   26.]],
       [[  27.,   28.,   29.],
        [  30.,   31.,   32.],
        [  33.,   34.,   35.],
        [  36.,   37.,   38.],
        [  39.,   40.,   41.],
        [  42.,   43.,   44.],
        [  45.,   46.,   47.],
        [  48.,   49.,   50.],
        [  51.,   52.,   53.]],
       [[  54.,   55.,   56.],
        [  57.,   58.,   59.],
        [  60.,   61.,   62.],
        [  63.,   64.,   65.],
        [  66.,   67.,   68.],
        [  69.,   70.,   71.],
        [  72.,   73.,   74.],
        [  75.,   76.,   77.],
        [  78.,   79.,   80.]],
       [[  81.,   82.,   83.],
        [  84.,   85.,   86.],
        [  87.,   88.,   89.],
        [  90.,   91.,   92.],
        [  93.,   94.,   95.],
        [  96.,   97.,   98.],
        [  99.,  100.,  101.],
        [ 102.,  103.,  104.],
        [ 105.,  106.,  107.]],
       [[ 108.,  109.,  110.],
        [ 111.,  112.,  113.],
        [ 114.,  115.,  116.],
        [ 117.,  118.,  119.],
        [ 120.,  121.,  122.],
        [ 123.,  124.,  125.],
        [ 126.,  127.,  128.],
        [ 129.,  130.,  131.],
        [ 132.,  133.,  134.]]])

In all processes, we have:

>>> distbuffer = local_array.__distarray__()
>>> distbuffer.keys()
['buffer', 'dim_data', '__version__']
>>> distbuffer['__version__']
'0.10.0'

The local arrays, on each separate engine:

In process (0, 0, 0):

>>> distbuffer['buffer']
array([[[   0.,    2.],
        [   3.,    5.],
        [   6.,    8.],
        [   9.,   11.],
        [  12.,   14.]],
       [[  54.,   56.],
        [  57.,   59.],
        [  60.,   62.],
        [  63.,   65.],
        [  66.,   68.]],
       [[ 108.,  110.],
        [ 111.,  113.],
        [ 114.,  116.],
        [ 117.,  119.],
        [ 120.,  122.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 3,
  'start': 0})

In process (0, 0, 1):

>>> distbuffer['buffer']
array([[[   1.],
        [   4.],
        [   7.],
        [  10.],
        [  13.]],
       [[  55.],
        [  58.],
        [  61.],
        [  64.],
        [  67.]],
       [[ 109.],
        [ 112.],
        [ 115.],
        [ 118.],
        [ 121.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 3,
  'start': 1})

In process (0, 1, 0):

>>> distbuffer['buffer']
array([[[  15.,   17.],
        [  18.,   20.],
        [  21.,   23.],
        [  24.,   26.]],
       [[  69.,   71.],
        [  72.,   74.],
        [  75.,   77.],
        [  78.,   80.]],
       [[ 123.,  125.],
        [ 126.,  128.],
        [ 129.,  131.],
        [ 132.,  134.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 3,
  'start': 0})

In process (0, 1, 1):

>>> distbuffer['buffer']
array([[[  16.],
        [  19.],
        [  22.],
        [  25.]],
       [[  70.],
        [  73.],
        [  76.],
        [  79.]],
       [[ 124.],
        [ 127.],
        [ 130.],
        [ 133.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 5,
  'start': 0},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 3,
  'start': 1})

In process (1, 0, 0):

>>> distbuffer['buffer']
array([[[ 27.,  29.],
        [ 30.,  32.],
        [ 33.,  35.],
        [ 36.,  38.],
        [ 39.,  41.]],
       [[ 81.,  83.],
        [ 84.,  86.],
        [ 87.,  89.],
        [ 90.,  92.],
        [ 93.,  95.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 3,
  'start': 0})

In process (1, 0, 1):

>>> distbuffer['buffer']
array([[[ 28.],
        [ 31.],
        [ 34.],
        [ 37.],
        [ 40.]],
       [[ 82.],
        [ 85.],
        [ 88.],
        [ 91.],
        [ 94.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'b',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 9,
  'start': 0,
  'stop': 5},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 3,
  'start': 1})

In process (1, 1, 0):

>>> distbuffer['buffer']
array([[[  42.,   44.],
        [  45.,   47.],
        [  48.,   50.],
        [  51.,   53.]],
       [[  96.,   98.],
        [  99.,  101.],
        [ 102.,  104.],
        [ 105.,  107.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9},
 {'dist_type': 'c',
  'proc_grid_rank': 0,
  'proc_grid_size': 2,
  'size': 3,
  'start': 0})

In process (1, 1, 1):

>>> distbuffer['buffer']
array([[[  43.],
        [  46.],
        [  49.],
        [  52.]],
       [[  97.],
        [ 100.],
        [ 103.],
        [ 106.]]])
>>> distbuffer['dim_data']
({'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 5,
  'start': 1},
 {'dist_type': 'b',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 9,
  'start': 5,
  'stop': 9},
 {'dist_type': 'c',
  'proc_grid_rank': 1,
  'proc_grid_size': 2,
  'size': 3,
  'start': 1})