Skelebot Version 2.0 (#242)

* Remove deprecated Artifactory component.

* Remove support for R and deprecated python version in scaffolding.

* Clean up some imports and remove obsolete language key from configs.

* Small linting of skelebot systems sub-package.

* Update dockerfile dependencies generation.

* Update remaining tests.

* Remove deprecated base images.

* Update example project.

* Update docker image build scripts.

* Update build process and fix tests.

* Update build system

* Update docs

* Update circleci config.

* Update dev dependencies.

* Update changelog and bump version.

* Update pypi upload workflow.

.circleci/config.yml

@@ -2,21 +2,16 @@ version: 2.1
 jobs:
   build:
     docker:
-      - image: circleci/python:3.6.15
+      - image: cimg/python:3.9
     working_directory: ~/skelebot
     steps:
       - checkout
       - run:
-          name: pip env
-          command: |
-            sudo pip install pipenv==2022.4.8
-            pipenv install
-      - run:
-          name: install dependencies
-          command: pipenv run pip install -r ./requirements.txt
+          name: install package
+          command: pipenv run pip install .[dev]
       - run:
           name: run tests
-          command: pipenv run coverage run --source=skelebot setup.py test && pipenv run coverage xml -o codecov.xml
+          command: pipenv run coverage run --source=skelebot -m pytest && pipenv run coverage xml -o codecov.xml
       - run:
           name: upload coverage
           command: bash <(curl -s https://codecov.io/bash) -f codecov.xml

.dockerignore

@@ -3,7 +3,6 @@
 # Editing this file manually is not advised as all changes will be overwritten by Skelebot
 
 **/*.zip
-**/*.RData
 **/*.pkl
 **/*.csv
 **/*.model

.github/workflows/pypi-upload.yaml

@@ -9,19 +9,19 @@ jobs:
   deploy:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v4
     - name: Set up Python
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.8'
+        python-version: '3.9'
     - name: Install dependencies
       run: |
-        python -m pip install --upgrade pip
-        python -m pip install setuptools wheel
+        python3 -m pip install --upgrade pip
+        python3 -m pip install build
     - name: Build distribution
-      run: python setup.py sdist bdist_wheel
+      run: python3 -m build
     - name: Publish to PYPI
-      uses: pypa/gh-action-pypi-publish@master
+      uses: pypa/gh-action-pypi-publish@release/v1
       with:
         user: __token__
         password: ${{ secrets.PYPI_PASSWORD }}

CHANGELOG.md

@@ -3,11 +3,28 @@ Documenting All Changes to the Skelebot Project
 
 ---
 
-## v1.37.0
+## v2.0.0
 #### Changed
+- **Dependencies** | All project dependencies, regardless of source (individual packages, local or remote files, requirements files, pyproject scripts) are now installed together in a single `pip install ...` command.
+- **Build system** | Refactored build system to use a single `pyproject.toml` script. Switched build backend from `setuptools` to `hatchling`.
+
+#### Removed
+- **Config** | Removed now unnecessary `language` config variable. Existing projects may keep that variable and it will be ignored going forward.
+- **Artifactory** | Removed old deprecated Artifactory component.
+- **Python versions** | Support for python versions 3.6, 3.7, and 3.8, including base docker images, is **removed**.
+- **R support** | All support for `R` and `R+Python` projects, including base docker images, is **removed**.
+
+---
+
+## v1.37.0
+#### Merged: 2023-05-29
+#### Released: 2024-05-29
+#### Deprecated
 - **Python versions** | Support for python versions 3.6, 3.7, and 3.8, including base docker images, is **deprecated** and will be removed in skelebot 2.0
 - **R support** | All support for `R` and `R+Python` projects, including base docker images, is **deprecated** and will be removed in skelebot 2.0
-- **Python 3.11** | Adding support for Python 3.11
+
+#### Added
+- **Python 3.11** | Adding support for Python 3.11.
 
 ---
 

Dockerfile

@@ -2,12 +2,11 @@
 # This Dockerfile was generated by Skelebot
 # Editing this file manually is not advised as all changes will be overwritten by Skelebot
 
-FROM skelebot/python-base:3.6
+FROM skelebot/python-base:3.9
 MAINTAINER Sean Shookman <[email protected]>
 WORKDIR /app
 ENV TZ=America/Chicago
 RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
-COPY requirements.txt requirements.txt
-RUN ["pip", "install", "-r", "/app/requirements.txt"]
+RUN ["pip", "install", "PyYAML>=5.1.2", "dohq-artifactory>=0.1.17", "schema>=0.7.0", "colorama>=0.4.1", "boto3>=1.10", "tomli>=1.1.0 ; python_version < '3.11'", "pytest~=8.2", "coverage~=7.5"]
 COPY . /app
 ENTRYPOINT ["bash", "jobs/test.sh"]

README.md

@@ -62,38 +62,43 @@ Skelebot is a command-line tool for developing machine learning projects and exe
 
 ```
 [/code/my-iris-model] > skelebot -h
-usage: skelebot [-h] [-e ENV] [-s] [-n]
-                {loadData,train,score,push,pull,jupyter,plugin,bump,prime,exec}
+usage: skelebot [-h] [-v] [-e ENV] [-d HOST] [-s] [-n] [-c] [-V]
+                {loadData,train,score,push,pull,jupyter,plugin,bump,prime,exec,publish,envs}
                 ...
 
 Iris Example
 Example Skelebot Project
 -----------------------------------
 Version: 1.1.0
 Environment: None
-Skelebot Version: 1.8.5
+Skelebot Version: 2.0.0
 -----------------------------------
 
 positional arguments:
-  {loadData,train,score,push,pull,jupyter,plugin,bump,prime,exec}
+  {loadData,train,score,push,pull,jupyter,plugin,bump,prime,exec,publish,envs}
     loadData            Load the Iris Dataset and save it into the data folder for the train job to access (src/loadData.py)
     train               Use the data loaded in the loadData job to train the iris model (src/train.py)
     score               Use the model that was built in the train job to score new data against the iris model (src/score.py)
-    push                Push an artifact to artifactory
-    pull                Pull an artifact from artifactory
+    push                Push an artifact to Artifactory
+    pull                Pull an artifact from Artifactory
     jupyter             Spin up Jupyter in a Docker Container (port=8888, folder=.)
     plugin              Install a plugin for skelebot from a local zip file
     bump                Bump the skelebot.yaml project version
     prime               Generate Dockerfile and .dockerignore and build the docker image
     exec                Exec into the running Docker container
+    publish             Publish your versioned Docker Image to the registry
+    envs                Display the available environments for the project
 
 optional arguments:
   -h, --help            show this help message and exit
   -v, --version         Display the version number of Skelebot
   -e ENV, --env ENV     Specify the runtime environment configurations
+  -d HOST, --docker-host HOST
+                        Set the Docker Host on which the command will be executed
   -s, --skip-build      Skip the build process and attempt to use previous docker build
   -n, --native          Run natively instead of through Docker
   -c, --contact         Display the contact email of the Skelebot project
+  -V, --verbose         Print all job commands to the screen just before execution
 ```
 
 ## Install

VERSION

@@ -1 +1 @@
-1.37.0
+2.0.0

docs/api.md

@@ -3,7 +3,7 @@
 ---
 
 <h1 align='center'>Skelebot API</h1>
-<div align='center'>Version 1</div>
+<div align='center'>Version 2</div>
 
 ---
 
@@ -12,12 +12,12 @@
 Skelebot exposes a number of functions that can be called inside of a plugin in order to execute
 and leverage important parts of the Skelebot System.
 
-This document specifies the API contract for v1 of Skelebot.
+This document specifies the API contract for v2 of Skelebot.
 
 There are packages and classes outside of this API that are technically accessible via Skelebot,
-but since they are not included in this specification, they are not part of the v1 contract for
+but since they are not included in this specification, they are not part of the v2 contract for
 Skelebot. As such it is not advised to use any function or Class that is not specified in this
-document because it may be subject to change throughout iterations of v1.
+document because it may be subject to change throughout iterations of v2.
 
 ---
 
@@ -38,15 +38,4 @@ document because it may be subject to change throughout iterations of v1.
 
 ---
 
-### Updates
-
-_v1.1.0 Additions_
-
-- SkeleYaml
-  - schema (attribute)
-  - loadList (method)
-  - validate (method)
-
----
-
 <center><< <a href="timezone.html">Timezone</a>  |  <a href="index.html">Home</a> >></center>

docs/api/common.md

@@ -3,7 +3,7 @@
 ---
 
 <h1 align='center'>Skelebot API</h1>
-<div align='center'>Version 1</div>
+<div align='center'>Version 2</div>
 
 ---
 

docs/api/component.md

@@ -3,7 +3,7 @@
 ---
 
 <h1 align='center'>Skelebot API</h1>
-<div align='center'>Version 1</div>
+<div align='center'>Version 2</div>
 
 ---
 

docs/api/docker.md

@@ -3,7 +3,7 @@
 ---
 
 <h1 align='center'>Skelebot API</h1>
-<div align='center'>Version 1</div>
+<div align='center'>Version 2</div>
 
 ---
 

docs/api/dockerfile.md

@@ -3,7 +3,7 @@
 ---
 
 <h1 align='center'>Skelebot API</h1>
-<div align='center'>Version 1</div>
+<div align='center'>Version 2</div>
 
 ---