From 6cc2e21a2d3382ec5a72bc50aada5168d711d5a9 Mon Sep 17 00:00:00 2001 From: Jeremy Maitin-Shepard Date: Wed, 1 Jun 2022 14:44:55 -0700 Subject: [PATCH] Remove the object data type extension. While this extension provides compatibility with zarr v2, and there are certainly applications where it makes sense to store arbitrary Python objects, the current extension has a number of issues: - As it is currently specified, it describes the in-memory Python object array representation, but surely it is not intended that the zarr array would actually store Python object memory addresses. - No encoding is specified (e.g. pickle or json). - Pickle is problematic since it is Python-specific and inherently unsafe to use to read untrusted data. --- docs/extensions/data-types/object/v1.0.rst | 105 --------------------- 1 file changed, 105 deletions(-) delete mode 100644 docs/extensions/data-types/object/v1.0.rst diff --git a/docs/extensions/data-types/object/v1.0.rst b/docs/extensions/data-types/object/v1.0.rst deleted file mode 100644 index 17a848ac..00000000 --- a/docs/extensions/data-types/object/v1.0.rst +++ /dev/null @@ -1,105 +0,0 @@ -=================================== - Object data types (version 1.0) -=================================== ------------------------------ - Editor's draft 2 March 2022 ------------------------------ - -Specification URI: - http://purl.org/zarr/spec/protocol/extensions/object-dtypes/1.0 -Issue tracking: - `GitHub issues `_ -Suggest an edit for this spec: - `GitHub editor `_ - -Copyright 2022 `Zarr core development -team `_ (@@TODO -list institutions?). This work is licensed under a `Creative Commons -Attribution 3.0 Unported -License `_. - ----- - - -Abstract -======== - -This specification is a Zarr protocol extension defining a data type where each -element is an arbitrary Python object. - - -Status of this document -======================= - -This document is a **Work in Progress**. It may be updated, replaced -or obsoleted by other documents at any time. It is inappapropriate to -cite this document as other than work in progress. - -Comments, questions or contributions to this document are very -welcome. Comments and questions should be raised via `GitHub issues -`_. When -raising an issue, please add the label "object-dtypes-v1.0". - -This document was produced by the `Zarr core development team -`_. - - -Document conventions -==================== - -Conformance requirements are expressed with a combination of -descriptive assertions and [RFC2119]_ terminology. The key words -"MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", -"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in the normative -parts of this document are to be interpreted as described in -[RFC2119]_. However, for readability, these words do not appear in all -uppercase letters in this specification. - -All of the text of this specification is normative except sections -explicitly marked as non-normative, examples, and notes. Examples in -this specification are introduced with the words "for example". - - -Object data types -================= -NumPy's object arrays are arrays where each element is an arbitrary Python -object. The array elements correspond to the -`numpy.object_ ` -type which has character code `'O'`. A common concrete use case for this type -is to have an array where each element is another array (and each array can -have a different length). Another use case is to store an array of variable -length strings. It is important to note that such an array actually just stores the references to the Python objects and not the objects themselves. Accessing -an element of the array returns the Python object it refers to. - -Data Types added by this extension -================================== - -.. list-table:: Data types - :header-rows: 1 - - * - Identifier - - Numerical type - - Size (no. bytes) - - Byte order - * - ``O`` (uppercase letter o) - - 8 (TODO: I assume this is actually a hardware-dependent memory address size?) - - address of a Python object - - None - - -References -========== - -.. [NumPy] NumPy Data type objects. NumPy version 1.22.0 - documentation. URL: - https://numpy.org/doc/1.22/reference/arrays.dtypes.html - -.. [H5Py variable length strings] Variable length strings - documentation. URL: - https://docs.h5py.org/en/stable/special.html#variable-length-strings - -Change log -========== - -@@TODO -