Move core to root directory and re-org for new repo.

.github/workflows/ci.yaml

@@ -0,0 +1,62 @@
+name: CI
+
+on:
+  workflow_dispatch:
+  pull_request:
+  push:
+    branches:
+      - main
+
+concurrency:
+  # A PR number if a pull request and otherwise the commit hash. This cancels
+  # queued and in-progress runs for the same PR (presubmit) or commit
+  # (postsubmit). The workflow name is prepended to avoid conflicts between
+  # different workflows.
+  group: ${{ github.workflow }}-${{ github.event.number || github.sha }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: "Unit Tests and Type Checking"
+    strategy:
+      matrix:
+        version: [3.11]
+        os: [ubuntu-latest]
+    runs-on: ${{matrix.os}}
+    env:
+      PIP_CACHE_DIR: "${{ github.workspace }}/.pip-cache"
+    steps:
+      - name: "Setting up Python"
+        id: setup_python
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{matrix.version}}
+
+      - name: "Checkout Code"
+        uses: actions/checkout@v3
+
+      - name: Cache Pip Packages
+        uses: actions/cache@v4
+        id: cache-pip
+        with:
+          path: ${{ env.PIP_CACHE_DIR }}
+          key: pip-${{ steps.setup_python.outputs.python-version }}-${{ hashFiles('*requirements.txt') }}
+
+      - name: Install pip deps
+        run: |
+          python -m pip install --no-compile --upgrade pip
+          # Note: We install in three steps in order to satisfy requirements
+          # from non default locations first. Installing the PyTorch CPU
+          # wheels saves multiple minutes and a lot of bandwidth on runner setup.
+          pip install --no-compile -r pytorch-cpu-requirements.txt
+          pip install -r requirements.txt -e .
+
+      - name: Run unit tests
+        if: ${{ !cancelled() }}
+        run: |
+          pytest -n 4 .
+
+      - name: MyPy Type Checking
+        if: ${{ !cancelled() }}
+        run: |
+          mypy

.github/workflows/pre-commit.yaml

@@ -0,0 +1,14 @@
+name: pre-commit
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: actions/setup-python@v3
+    - uses: pre-commit/[email protected]

.gitignore

@@ -0,0 +1,32 @@
+# Visual Studio files
+.env
+.vs/
+.vscode/
+*.sdf
+*.opensdf
+*.VC.opendb
+*.suo
+*.user
+
+# macOS files
+.DS_Store
+
+# CMake artifacts
+build/
+build-*/
+
+# Python
+__pycache__
+_python_build/
+deps/
+dist/
+wheelhouse
+*.egg-info
+*.whl
+*.venv
+
+#Model artifacts
+*.pt
+*.safetensors
+*.gguf
+*.vmfb

.pre-commit-config.yaml

@@ -0,0 +1,14 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: check-yaml
+    -   id: check-added-large-files
+-   repo: https://github.com/psf/black
+    rev: 22.10.0
+    hooks:
+    -   id: black

CONTRIBUTING.md

@@ -0,0 +1,20 @@
+# How to contribute
+
+We'd love to accept your patches and contributions to this project.
+
+To get started with contributing, please take a look at the
+[Contributing](https://iree.dev/developers/general/contributing/) guide.
+
+## Getting in touch
+
+*   [GitHub issues](https://github.com/iree-org/iree/issues): Feature requests,
+    bugs, and other work tracking
+*   [IREE Discord server](https://discord.gg/wEWh6Z9nMU): Daily development
+    discussions with the core team and collaborators
+*   [iree-discuss email list](https://groups.google.com/forum/#!forum/iree-discuss):
+    Announcements, general and low-priority discussion
+
+## Community guidelines
+
+This project follows the
+[OpenXLA Code of Conduct](https://github.com/openxla/community/blob/main/CODE-OF-CONDUCT.md).

README.md

@@ -2,11 +2,7 @@
 
 ![image](https://netl.doe.gov/sites/default/files/2020-11/Turbine-8412270026_83cfc8ee8f_c.jpg)
 
-Turbine is the set of development tools that the [SHARK Team](https://github.com/nod-ai/SHARK)
-is building for deploying all of our models for deployment to the cloud and devices. We
-are building it as we transition from our TorchScript-era 1-off export and compilation
-to a unified approach based on PyTorch 2 and Dynamo. While we use it heavily ourselves, it
-is intended to be a general purpose model compilation and execution tool.
+Turbine is IREE's frontend for PyTorch.
 
 Turbine provides a collection of tools:
 
@@ -21,35 +17,27 @@ Turbine provides a collection of tools:
   native PyTorch constructs and tracing. It is intended to complement for simple
   cases where direct emission to the underlying, cross platform, vector programming model
   is desirable.
-* *Turbine-LLM*: a repository of layers, model recipes, and conversion tools
-  from popular Large Language Model (LLM) quantization tooling.
-
-Under the covers, Turbine is based heavily on [IREE](https://github.com/openxla/iree) and
-[torch-mlir](https://github.com/llvm/torch-mlir) and we use it to drive evolution
-of both, upstreaming infrastructure as it becomes timely to do so.
-
-See [the roadmap](docs/roadmap.md) for upcoming work and places to contribute.
 
 ## Contact Us
 
-Turbine is under active development. If you would like to participate as it comes online,
-please reach out to us on the `#turbine` channel of the
-[nod-ai Discord server](https://discord.gg/QMmR6f8rGb).
+Turbine is under active development. Feel free to reach out on one of 
+[IREE's communication channels](https://github.com/iree-org/iree?tab=readme-ov-file#communication-channels) (specifically, we monitor the 
+#pytorch channel on the IREE Discord server).
 
 ## Quick Start for Users
 
 1. Install from source:
 
 ```
-pip install shark-turbine
+pip install iree-turbine
 # Or for editable: see instructions under developers
 ```
 
 The above does install some unecessary cuda/cudnn packages for cpu use. To avoid this you
 can specify pytorch-cpu and install via:
 ```
-pip install -r core/pytorch-cpu-requirements.txt
-pip install shark-turbine
+pip install -r pytorch-cpu-requirements.txt
+pip install iree-turbine
 ```
 
 (or follow the "Developers" instructions below for installing from head/nightly)
@@ -63,42 +51,65 @@ compiler, these should be compilable via IREE with `--iree-input-type=torch` for
 end to end execution. Dynamic shape support in torch-mlir is a work in progress,
 and not everything works at head with release binaries at present.
 
-  * [AOT MLP With Static Shapes](https://github.com/nod-ai/SHARK-Turbine/blob/main/core/examples/aot_mlp/mlp_export_simple.py)
-  * [AOT MLP with a dynamic batch size](https://github.com/nod-ai/SHARK-Turbine/blob/main/core/examples/aot_mlp/mlp_export_dynamic.py)
-  * [AOT llama2](https://github.com/nod-ai/SHARK-Turbine/blob/main/core/examples/llama2_inference/llama2.ipynb):
+  * [AOT MLP With Static Shapes](examples/aot_mlp/mlp_export_simple.py)
+  * [AOT MLP with a dynamic batch size](examples/aot_mlp/mlp_export_dynamic.py)
+  * [AOT llama2](examples/llama2_inference/llama2.ipynb):
     Dynamic sequence length custom compiled module with state management internal to the model.
-  * [Eager MNIST with `torch.compile`](https://github.com/nod-ai/SHARK-Turbine/blob/main/core/examples/eager_mlp/mlp_eager_simple.py)
+  * [Eager MNIST with `torch.compile`](examples/eager_mlp/mlp_eager_simple.py)
 
 ## Developers
 
-### Getting Up and Running
+Use this as a guide to get started developing the project using pinned,
+pre-release dependencies. You are welcome to deviate as you see fit, but
+these canonical directions mirror what the CI does.
+
+### Setup a venv
+
+We recommend setting up a virtual environment (venv). The project is configured
+to ignore `.venv` directories, and editors like VSCode pick them up by default.
+
+```
+python -m venv --prompt iree-turbine .venv
+source .venv/bin/activate
+```
+
+### Install PyTorch for Your System
 
-If only looking to develop against this project, then you need to install Python
-deps for the following:
+If no explicit action is taken, the default PyTorch version will be installed.
+This will give you a current CUDA-based version. Install a different variant
+by doing so explicitly first:
 
-* PyTorch
-* iree-compiler (with Torch input support)
-* iree-runtime
+*CPU:*
 
-The pinned deps at HEAD require pre-release versions of all of the above, and
-therefore require additional pip flags to install. Therefore, to satisfy
-development, we provide a `requirements.txt` file which installs precise
-versions and has all flags. This can be installed prior to the package:
+```
+pip install -r pytorch-cpu-requirements.txt
+```
+
+*ROCM:*
+
+```
+pip install -r pytorch-rocm-requirements.txt
+```
 
-Installing into a venv is highly recommended.
+### Install Development Packages
 
 ```
-pip install -r core/pytorch-cpu-requirements.txt
-pip install --upgrade -r core/requirements.txt
-pip install --upgrade -e "core[torch-cpu-nightly,testing]"
+# Install editable local projects.
+pip install -r requirements.txt -e .
 ```
 
-Run tests:
+### Running Tests
 
 ```
-pytest core/
+pytest .
 ```
 
+### Optional: Pre-commits and developer settings
+
+This project is set up to use the `pre-commit` tooling. To install it in
+your local repo, run: `pre-commit install`. After this point, when making
+commits locally, hooks will run. See https://pre-commit.com/
+
 ### Using a development compiler
 
 If doing native development of the compiler, it can be useful to switch to

core/pytorch-cpu-requirements.txt → pytorch-cpu-requirements.txt

@@ -1,3 +1,5 @@
 --pre
 --index-url https://download.pytorch.org/whl/test/cpu
--r pytorch-requirements.txt
+torch==2.3.0
+torchaudio
+torchvision

pytorch-rocm-requirements.txt

@@ -0,0 +1,7 @@
+--pre
+--index-url https://download.pytorch.org/whl/nightly/rocm6.0
+# TODO: PyTorch ROCM doesn't seem to have a 2.3 RC published, so we just
+# get a nightly.
+torch>=2.3.0.dev1,<2.4
+torchaudio
+torchvision

requirements.txt

@@ -0,0 +1,14 @@
+numpy==1.26.3
+pytest==8.0.0
+pytest-xdist==3.5.0
+mypy==1.8.0
+
+# It is expected that you have installed a PyTorch version/variant specific
+# to your needs, so we only include a minimum version spec.
+# TODO: Use a versioned release once 2.3.0 drops.
+torch>=2.3.0.dev1
+torchaudio
+torchvision
+
+# Used for managing pre-commit flows.
+pre-commit

core/setup.py → setup.py

@@ -12,7 +12,7 @@
 from setuptools import find_namespace_packages, setup
 
 THIS_DIR = os.path.realpath(os.path.dirname(__file__))
-REPO_DIR = os.path.dirname(THIS_DIR)
+REPO_DIR = THIS_DIR
 VERSION_INFO_FILE = os.path.join(REPO_DIR, "version_info.json")
 
 # Transitional as we migrate from shark-turbine -> iree-turbine.
@@ -58,9 +58,7 @@ def load_requirement_pins(requirements_file: str):
     requirement_pins.update(dict(pin_pairs))
 
 
-load_requirement_pins("iree-requirements.txt")
-load_requirement_pins("misc-requirements.txt")
-load_requirement_pins("pytorch-cpu-requirements.txt")
+load_requirement_pins("requirements.txt")
 
 
 def get_version_spec(dep: str):
@@ -105,19 +103,10 @@ def initialize_options(self):
         f"numpy{get_version_spec('numpy')}",
         f"iree-compiler{get_version_spec('iree-compiler')}",
         f"iree-runtime{get_version_spec('iree-runtime')}",
-        # Use the [torch-cpu-nightly] spec to get a more recent/specific version.
-        # Note that during the transition to torch 2.3.0 we technically support
-        # back to torch 2.1, which is why we pin here in this way. However,
-        # the CI tests on 2.3.
-        "torch>=2.1.0",
+        "torch>=2.3.0",
     ],
     extras_require={
-        "torch-cpu-nightly": [f"torch{get_version_spec('torch')}"],
-        "onnx": [
-            f"onnx{get_version_spec('onnx')}",
-        ],
         "testing": [
-            f"onnx{get_version_spec('onnx')}",
             f"pytest{get_version_spec('pytest')}",
             f"pytest-xdist{get_version_spec('pytest-xdist')}",
         ],