diff --git a/README.md b/README.md index 509e5b19b..1ad03e94e 100644 --- a/README.md +++ b/README.md @@ -93,11 +93,11 @@ as a [cluster library](https://docs.databricks.com/libraries/cluster-libraries.h %pip install databricks-mosaic ``` -Then enable it with +Then enable Mosaic (and namespace it in python) with ```python -from mosaic import enable_mosaic -enable_mosaic(spark, dbutils) +import mosaic as mos +mos.enable_mosaic(spark, dbutils) ``` ### Scala diff --git a/docs/source/api/spatial-functions.rst b/docs/source/api/spatial-functions.rst index b629dcee8..73be0ce27 100644 --- a/docs/source/api/spatial-functions.rst +++ b/docs/source/api/spatial-functions.rst @@ -238,58 +238,6 @@ st_bufferloop Fig 1. ST_BufferLoop(wkt, 0.02, 0.04) -st_centroid2D [Deprecated] -************************** - -.. function:: st_centroid2D(col) - - Returns the x and y coordinates representing the centroid of the input geometry. - - :param col: Geometry - :type col: Column - :rtype: Column: StructType[x: DoubleType, y: DoubleType] - - :example: - -.. tabs:: - .. code-tab:: py - - df = spark.createDataFrame([{'wkt': 'POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))'}]) - df.select(st_centroid2D('wkt')).show() - +---------------------------------------+ - |st_centroid(wkt) | - +---------------------------------------+ - |{25.454545454545453, 26.96969696969697}| - +---------------------------------------+ - - .. code-tab:: scala - - val df = List(("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))")).toDF("wkt") - df.select(st_centroid2D(col("wkt"))).show() - +---------------------------------------+ - |st_centroid(wkt) | - +---------------------------------------+ - |{25.454545454545453, 26.96969696969697}| - +---------------------------------------+ - - .. code-tab:: sql - - SELECT st_centroid2D("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))") - +---------------------------------------+ - |st_centroid(wkt) | - +---------------------------------------+ - |{25.454545454545453, 26.96969696969697}| - +---------------------------------------+ - - .. code-tab:: r R - - df <- createDataFrame(data.frame(wkt = "POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))")) - showDF(select(df, st_centroid2D(column("wkt"))), truncate=F) - +---------------------------------------+ - |st_centroid(wkt) | - +---------------------------------------+ - |{25.454545454545453, 26.96969696969697}| - +---------------------------------------+ st_centroid ************* @@ -533,8 +481,8 @@ st_dimension .. tabs:: .. code-tab:: py - >>> df = spark.createDataFrame([{'wkt': 'POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))'}]) - >>> df.select(st_dimension('wkt')).show() + df = spark.createDataFrame([{'wkt': 'POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))'}]) + df.select(st_dimension('wkt')).show() +-----------------+ |st_dimension(wkt)| +-----------------+ @@ -543,8 +491,8 @@ st_dimension .. code-tab:: scala - >>> val df = List("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))").toDF("wkt") - >>> df.select(st_dimension(col("wkt"))).show() + val df = List("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))").toDF("wkt") + df.select(st_dimension(col("wkt"))).show() +-----------------+ |st_dimension(wkt)| +-----------------+ @@ -553,7 +501,7 @@ st_dimension .. code-tab:: sql - >>> SELECT st_dimension("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))") + SELECT st_dimension("POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))") +-----------------+ |st_dimension(wkt)| +-----------------+ @@ -562,8 +510,8 @@ st_dimension .. code-tab:: r R - >>> df <- createDataFrame(data.frame(wkt = "POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))")) - >>> showDF(select(df, st_dimension(column("wkt")))) + df <- createDataFrame(data.frame(wkt = "POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))")) + showDF(select(df, st_dimension(column("wkt")))) +-----------------+ |st_dimension(wkt)| +-----------------+ diff --git a/docs/source/api/spatial-indexing.rst b/docs/source/api/spatial-indexing.rst index a35f1ae85..4521dd38c 100644 --- a/docs/source/api/spatial-indexing.rst +++ b/docs/source/api/spatial-indexing.rst @@ -1488,44 +1488,3 @@ grid_geometrykloopexplode .. raw:: html - - - -mosaic_explode [Deprecated] -*************************** - -.. function:: mosaic_explode(geometry, resolution, keep_core_geometries) - - This is an alias for :ref:`grid_tessellateexplode` - - -mosaicfill [Deprecated] -************************ - -.. function:: mosaicfill(geometry, resolution, keep_core_geometries) - - This is an alias for :ref:`grid_tessellate` - - -point_index_geom [Deprecated] -****************************** - -.. function:: point_index_geom(point, resolution) - - This is an alias for :ref:`grid_pointascellid` - - -point_index_lonlat [Deprecated] -******************************** - -.. function:: point_index_lonlat(point, resolution) - - This is an alias for :ref:`grid_longlatascellid` - - -polyfill [Deprecated] -********************** - -.. function:: polyfill(geom, resolution) - - This is an alias for :ref:`grid_polyfill` diff --git a/docs/source/images/function_describe.png b/docs/source/images/function_describe.png new file mode 100644 index 000000000..64d1ec122 Binary files /dev/null and b/docs/source/images/function_describe.png differ diff --git a/docs/source/images/functions_show.png b/docs/source/images/functions_show.png new file mode 100644 index 000000000..fda061673 Binary files /dev/null and b/docs/source/images/functions_show.png differ diff --git a/docs/source/usage/automatic-sql-registration.rst b/docs/source/usage/automatic-sql-registration.rst index c1403cec0..a653f75b9 100644 --- a/docs/source/usage/automatic-sql-registration.rst +++ b/docs/source/usage/automatic-sql-registration.rst @@ -34,7 +34,8 @@ To install Mosaic on your Databricks cluster, take the following steps: #. Upload Mosaic jar to a dedicated fuse mount location. E.g. "dbfs:/FileStore/mosaic/jars/". -#. Create an init script that fetches the mosaic jar and copies it to databricks/jars. +#. Create an init script that fetches the mosaic jar and copies it to "databricks/jars". + You can also use the output from (0.4 series) python function :code:`setup_fuse_install`, e.g. :code:`setup_fuse_intall(, jar_copy=True)` which can help to copy the JAR used in the init script below. @@ -73,17 +74,31 @@ To install Mosaic on your Databricks cluster, take the following steps: Testing ******* -To test the installation, create a new Python notebook and run the following command: +To test the installation, create a new Python notebook and run the following commands (similar for :code:`grid_` and :code:`rst_`, not shown): .. code-block:: python - spark.sql("""show functions""").where("startswith(function, 'st_')").display() + sql("""SHOW FUNCTIONS""").where("startswith(function, 'st_')").display() + +You should see all the supported :code:`ST_` functions registered by Mosaic appear in the output. -You should see all the supported functions registered by Mosaic appear in the output. +.. figure:: ../images/functions_show.png + :figclass: doc-figure + + Fig 1. Show Functions Example .. note:: You may see some :code:`ST_` functions from other libraries, so pay close attention to the provider. +.. code-block:: python + + sql("""DESCRIBE FUNCTION st_buffer""") + +.. figure:: ../images/function_describe.png + :figclass: doc-figure + + Fig 2. Describe Function Example + .. warning:: Mosaic 0.4.x SQL bindings for DBR 13 can register with Assigned clusters, but not Shared Access due to API changes, more `here `_. diff --git a/docs/source/usage/install-gdal.rst b/docs/source/usage/install-gdal.rst index d20cf7adc..97fba5d5e 100644 --- a/docs/source/usage/install-gdal.rst +++ b/docs/source/usage/install-gdal.rst @@ -107,5 +107,5 @@ code at the top of the notebook: GDAL 3.4.1, released 2021/12/27 .. note:: - You can configure init script from default ubuntu GDAL (3.4.1) to `ubuntugis ppa `_ (3.4.3) - with :code:`setup_gdal(with_ubuntugis=True)` \ No newline at end of file + You can configure init script from default ubuntu GDAL (3.4.1) to ubuntugis ppa @ https://launchpad.net/~ubuntugis/+archive/ubuntu/ppa (3.4.3) + with `setup_gdal(with_ubuntugis=True)` \ No newline at end of file diff --git a/docs/source/usage/installation.rst b/docs/source/usage/installation.rst index 7df5bc40c..94e1ca134 100644 --- a/docs/source/usage/installation.rst +++ b/docs/source/usage/installation.rst @@ -54,7 +54,7 @@ or from within a Databricks notebook using the :code:`%pip` magic command, e.g. If you need to install Mosaic 0.3 series for DBR 12.2 LTS, e.g. -.. code-block::bash +.. code-block:: bash %pip install "databricks-mosaic<0.4,>=0.3"