From 0a7b70aec6abc14de74e1d97921f41a9f742d30c Mon Sep 17 00:00:00 2001 From: Alistair Miles Date: Wed, 8 May 2019 17:34:01 +0100 Subject: [PATCH 1/2] initial drafting on memory layouts section --- docs/protocol/core/v3.0.rst | 43 ++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/protocol/core/v3.0.rst b/docs/protocol/core/v3.0.rst index e5c6c8d2..fe5aceec 100644 --- a/docs/protocol/core/v3.0.rst +++ b/docs/protocol/core/v3.0.rst @@ -112,7 +112,48 @@ TODO define regular chunk grids, including how to form a key for each chunk in a Memory layouts -------------- -TODO define "C" and "F" memory layouts +An array has a memory layout, which defines the way that the binary +values of the array elements are organised within each chunk to form a +contiguous sequence of bytes. This contiguous binary representation of +a chunk is then the input to the array's chunk encoding pipeline, +described in later sections. Typically, when reading data, an +implementation will load this binary representation into a contiguous +memory buffer to allow direct access to array elements without having +to copy data. + +The core protocol defines two types of contiguous memory +layout. However, protocol extensions may define other memory +layouts. Note that there may be an interdependency between memory +layouts and data types, such that certain memory layouts may only be +applicable to arrays with certain data types. + +C contiguous memory layout +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In this memory layout, the binary values of the array elements are +organised into a sequence such that the last dimension of the array is +the fastest changing dimension, also known as "row-major" order. This +layout is only applicable to arrays with fixed size data types. + +For example, for a two-dimensional array with chunk shape (dx, dy), +the binary values for a given chunk are taken from chunk elements in +the order (0, 0), (0, 1), (0, 2), ..., (dx - 1, dy - 3), (dx - 1, dy - +2), (dx - 1, dy - 1). + +F contiguous memory layout +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In this memory layout, the binary values of the array elements are +organised into a sequence such that the first dimension of the array +is the fastest changing dimension, also known as "column-major" or +"Fortran" order. This layout is only applicable to arrays with fixed +size data types. + +For example, for a two-dimensional array with chunk shape (dx, dy), +the binary values for a given chunk are taken from chunk elements in +the order (0, 0), (1, 0), (2, 0), ..., (dx - 3, dy - 1), (dx - 2, dy - +1), (dx - 1, dy - 1). + Codec interface --------------- From 34566d19ea5af6684de89ea8794599b0b1099be9 Mon Sep 17 00:00:00 2001 From: Alistair Miles Date: Mon, 20 May 2019 22:09:26 +0100 Subject: [PATCH 2/2] strike "Fortran" --- docs/protocol/core/v3.0.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/protocol/core/v3.0.rst b/docs/protocol/core/v3.0.rst index fe5aceec..601ceddd 100644 --- a/docs/protocol/core/v3.0.rst +++ b/docs/protocol/core/v3.0.rst @@ -145,9 +145,9 @@ F contiguous memory layout In this memory layout, the binary values of the array elements are organised into a sequence such that the first dimension of the array -is the fastest changing dimension, also known as "column-major" or -"Fortran" order. This layout is only applicable to arrays with fixed -size data types. +is the fastest changing dimension, also known as "column-major" +order. This layout is only applicable to arrays with fixed size data +types. For example, for a two-dimensional array with chunk shape (dx, dy), the binary values for a given chunk are taken from chunk elements in