Skip to content

Commit

Permalink
Merge branch 'main' into update-main-interactive-starters
Browse files Browse the repository at this point in the history
  • Loading branch information
merelcht committed Aug 9, 2024
2 parents 2522d4f + 768b785 commit a29998c
Show file tree
Hide file tree
Showing 4 changed files with 26 additions and 41 deletions.
8 changes: 2 additions & 6 deletions databricks-iris/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,12 @@ kedro new --starter=databricks-iris
This starter has a base configuration that allows it to run natively on Databricks. Directories to store data and logs still need to be manually created in the user's Databricks DBFS instance:

```bash
/dbfs/FileStore/iris-databricks/data
/dbfs/FileStore/iris-databricks/logs
/dbfs/FileStore/iris_databricks/data
/dbfs/FileStore/iris_databricks/logs
```

See the documentation on deploying a packaged Kedro project to Databricks for more information.

### Databricks entry point

The starter contains a script and an entry point (`databricks_run.py`) that enables a packaged project created with this starter to run on Databricks. See the documentation on deploying a packaged Kedro project to Databricks for more information.

### Single configuration in `/conf/base/spark.yml`

While Spark allows you to specify many different [configuration options](https://spark.apache.org/docs/latest/configuration.html), this starter uses `/conf/base/spark.yml` as a single configuration location.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ readme = "README.md"
dynamic = ["dependencies", "version"]

[project.scripts]
databricks_run = "{{ cookiecutter.python_package }}.databricks_run:main"
{{ cookiecutter.repo_name }} = "{{ cookiecutter.python_package }}.__main__:main"

[project.entry-points."kedro.hooks"]

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""{{ cookiecutter.project_name }} file for ensuring the package is executable
as `{{ cookiecutter.repo_name }}` and `python -m {{ cookiecutter.python_package }}`
"""
import sys
from pathlib import Path

from kedro.framework.cli.utils import find_run_command
from kedro.framework.project import configure_project


def main(*args, **kwargs):
package_name = Path(__file__).parent.name
configure_project(package_name)

interactive = hasattr(sys, 'ps1')
kwargs["standalone_mode"] = not interactive

run = find_run_command(package_name)
run(*args, **kwargs)


if __name__ == "__main__":
main()

This file was deleted.

0 comments on commit a29998c

Please sign in to comment.