Skip to content

Commit

Permalink
Merge pull request #673 from oddgrd/production
Browse files Browse the repository at this point in the history 
feat: merge main into prod
  • Loading branch information
@oddgrd
oddgrd authored Mar 2, 2023
2 parents cfe678d + d0f73e5 commit b1c8aba
Show file tree
Hide file tree
Showing 48 changed files with 826 additions and 410 deletions.
166 changes: 94 additions & 72 deletions .circleci/config.yml
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
version: 2.1

orbs:
orbs:
docker-buildx: sensu/[email protected]
aws-ecr: circleci/[email protected]
win: circleci/[email protected]
Expand Down Expand Up @@ -154,7 +154,7 @@ jobs:
- install-protoc
- run: cargo fmt --all --check
- run: cargo install cargo-sort
# TODO: this is incompatible with workspace inheritance, uncomment when
# TODO: this is incompatible with workspace inheritance, uncomment when
# https://github.com/DevinR528/cargo-sort/pull/29 is merged
# - run: cargo sort --check --workspace
- run: cargo check --workspace --all-targets
Expand Down Expand Up @@ -191,7 +191,7 @@ jobs:
- apply-patches
- run: cargo fmt --all --check --manifest-path << parameters.path >>/Cargo.toml
- run: cargo install cargo-sort
# TODO: this is incompatible with workspace inheritance, uncomment when
# TODO: this is incompatible with workspace inheritance, uncomment when
# https://github.com/DevinR528/cargo-sort/pull/29 is merged
# - run: cargo sort --check << parameters.path >>
- run: |
Expand Down Expand Up @@ -388,79 +388,101 @@ jobs:
workflows:
ci:
jobs:
- workspace-fmt
- workspace-clippy:
name: workspace-clippy-<< matrix.framework >>
requires:
- workspace-fmt
matrix:
parameters:
framework: ["web-actix-web", "web-axum", "web-rocket", "web-poem", "web-thruster", "web-tide", "web-tower","web-warp", "web-salvo", "bot-serenity", "bot-poise"]
- check-standalone:
matrix:
parameters:
path:
- resources/aws-rds
- resources/persist
- resources/secrets
- resources/shared-db
- resources/static-folder
- service-test:
requires:
- workspace-clippy
- platform-test:
requires:
- workspace-clippy
matrix:
parameters:
crate: ["shuttle-auth", "shuttle-deployer", "cargo-shuttle", "shuttle-codegen", "shuttle-common", "shuttle-proto", "shuttle-provisioner"]
- e2e-test:
requires:
- service-test
- platform-test
- check-standalone
filters:
branches:
only: production
- build-and-push:
requires:
- e2e-test
filters:
branches:
only: production
- build-binaries-linux:
name: build-binaries-x86_64-gnu
image: ubuntu-2204:2022.04.1
target: x86_64-unknown-linux-gnu
resource_class: medium
filters:
branches:
only: production
- build-binaries-linux:
name: build-binaries-x86_64-musl
image: ubuntu-2204:2022.04.1
target: x86_64-unknown-linux-musl
resource_class: medium
filters:
branches:
only: production
- build-binaries-linux:
name: build-binaries-aarch64
image: ubuntu-2004:202101-01
target: aarch64-unknown-linux-musl
resource_class: arm.medium
filters:
branches:
only: production
- build-binaries-windows:
- workspace-fmt
- workspace-clippy:
name: workspace-clippy-<< matrix.framework >>
requires:
- workspace-fmt
matrix:
parameters:
framework:
[
"web-actix-web",
"web-axum",
"web-rocket",
"web-poem",
"web-thruster",
"web-tide",
"web-tower",
"web-warp",
"web-salvo",
"bot-serenity",
"bot-poise",
]
- check-standalone:
matrix:
parameters:
path:
- resources/aws-rds
- resources/persist
- resources/secrets
- resources/shared-db
- resources/static-folder
- service-test:
requires:
- workspace-clippy
- platform-test:
requires:
- workspace-clippy
matrix:
parameters:
crate:
[
"shuttle-auth",
"shuttle-deployer",
"cargo-shuttle",
"shuttle-codegen",
"shuttle-common",
"shuttle-proto",
"shuttle-provisioner",
]
- e2e-test:
requires:
- service-test
- platform-test
- check-standalone
filters:
branches:
only: production
- build-and-push:
requires:
- e2e-test
filters:
branches:
only: production
- build-binaries-linux:
name: build-binaries-x86_64-gnu
image: ubuntu-2204:2022.04.1
target: x86_64-unknown-linux-gnu
resource_class: medium
filters:
branches:
only: production
- build-binaries-mac:
- build-binaries-linux:
name: build-binaries-x86_64-musl
image: ubuntu-2204:2022.04.1
target: x86_64-unknown-linux-musl
resource_class: medium
filters:
branches:
only: production
- publish-github-release:
only: production
- build-binaries-linux:
name: build-binaries-aarch64
image: ubuntu-2004:202101-01
target: aarch64-unknown-linux-musl
resource_class: arm.medium
filters:
branches:
only: production
- build-binaries-windows:
filters:
branches:
only: production
- build-binaries-mac:
filters:
branches:
only: production
- publish-github-release:
requires:
- build-binaries-x86_64-gnu
- build-binaries-x86_64-musl
Expand All @@ -469,4 +491,4 @@ workflows:
- build-binaries-mac
filters:
branches:
only: production
only: production
61 changes: 61 additions & 0 deletions .github/ISSUE_TEMPLATE/BUG-REPORT.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Bug Report
description: File a bug report
title: "[Bug]: "
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
value: "A bug happened!"
validations:
required: true
- type: input
id: version
attributes:
label: Version
description: What version of `cargo-shuttle` are you running (`cargo shuttle --version`)?
placeholder: "v0.11.0"
validations:
required: true
- type: dropdown
id: os
attributes:
label: Which operating systems are you seeing the problem on?
multiple: true
options:
- macOS
- Windows
- Linux
validations:
required: true
- type: dropdown
id: architecture
attributes:
label: Which CPU architectures are you seeing the problem on?
multiple: true
options:
- x86_64
- ARM64
- Other
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: sh
- type: checkboxes
id: duplicate
attributes:
label: Duplicate declaration
description: Please confirm that you are not creating a duplicate issue.
options:
- label: I have searched the issues and there are none like this.
required: true

5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Discord
url: https://discord.gg/shuttle
about: Feel free to reach out on our Discord should you have any questions!
9 changes: 9 additions & 0 deletions .github/pull_request_template.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
## Description of change

Please write a summary of your changes and why you made them.

Be sure to reference any related issues by adding `closes issue #`.

## How Has This Been Tested (if applicable)?

Please describe the tests that you ran to verify your changes.
3 changes: 2 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
# Generated by Cargo
# will have compiled files and executables
**/target/
Cargo.lock
# Ignore the cargo lockfiles not in the workspace root
*/**/Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk
Expand Down
25 changes: 17 additions & 8 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,23 @@
## Raise an Issue

Raising [issues](https://github.com/shuttle-hq/shuttle/issues) is encouraged.

## Docs

If you found an error in our docs, or you simply want to make them better, contributions to our [docs](https://github.com/shuttle-hq/shuttle-docs)
are always appreciated!

## Running Locally

You can use Docker and docker-compose to test shuttle locally during development. See the [Docker install](https://docs.docker.com/get-docker/)
and [docker-compose install](https://docs.docker.com/compose/install/) instructions if you do not have them installed already.

> Note for Windows: The current [Makefile](https://github.com/shuttle-hq/shuttle/blob/main/Makefile) does not work on Windows systems by itself - if you want to build the local environment on Windows you could use [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/install). Additional Windows considerations are listed at the bottom of this page.
> Note for Linux: When building on Linux systems, if the error unknown flag: --build-arg is received, install the docker-buildx package using the package management tool for your particular system.
Clone the shuttle repository (or your fork):

```
```bash
git clone [email protected]:shuttle-hq/shuttle.git
cd shuttle
```
Expand All @@ -28,9 +29,11 @@ You should now be ready to setup a local environment to test code changes to cor
From the root of the shuttle repo, build the required images with:

```bash
make images
USE_PANAMAX=disable make images
```

> Note: The stack uses [panamax](https://github.com/panamax-rs/panamax) by default to mirror crates.io content. We do this in order to avoid overloading upstream mirrors and hitting rate limits. After syncing the cache, expect to see the panamax volume take about 100GiB of space. This may not be desirable for local testing. To avoid using panamax, run `USE_PANAMAX=disable make images` instead.
The images get built with [cargo-chef](https://github.com/LukeMathWalker/cargo-chef) and therefore support incremental builds (most of the time). So they will be much faster to re-build after an incremental change in your code - should you wish to deploy it locally straight away.

You can now start a local deployment of shuttle and the required containers with:
Expand All @@ -39,13 +42,15 @@ You can now start a local deployment of shuttle and the required containers with
make up
```

> Note: `make up` does not start [panamax](https://github.com/panamax-rs/panamax) by default, if you do need to start panamax for local development, run this command with `make COMPOSE_PROFILES=panamax up`.
> Note: Other useful commands can be found within the [Makefile](https://github.com/shuttle-hq/shuttle/blob/main/Makefile).
The API is now accessible on `localhost:8000` (for app proxies) and `localhost:8001` (for the control plane). When running `cargo run --bin cargo-shuttle` (in a debug build), the CLI will point itself to `localhost` for its API calls.

In order to test local changes to the library crates, you may want to add the below to a `.cargo/config.toml` file. (See [Overriding Dependencies](https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html) for more)

``` toml
```toml
[patch.crates-io]
shuttle-service = { path = "[base]/shuttle/service" }
shuttle-common = { path = "[base]/shuttle/common" }
Expand All @@ -58,7 +63,7 @@ shuttle-static-folder = { path = "[base]/shuttle/resources/static-folder" }
```

Before we can login to our local instance of shuttle, we need to create a user.
The following command inserts a user into the gateway state with admin privileges:
The following command inserts a user into the `auth` state with admin privileges:

```bash
docker compose --file docker-compose.rendered.yml --project-name shuttle-dev exec auth /usr/local/bin/service --state=/var/lib/shuttle-auth init --name admin --key test-key
Expand Down Expand Up @@ -182,6 +187,7 @@ We will squash commits before merging to main. If you do want to squash commits,
after the review process has started, the commit history can be useful for reviewers.

Before committing:

- Make sure your commits don't trigger any warnings from Clippy by running: `cargo clippy --tests --all-targets`. If you have a good reason to contradict Clippy, insert an `#[allow(clippy::<lint>)]` macro, so that it won't complain.
- Make sure your code is correctly formatted: `cargo fmt --all --check`.
- Make sure your `Cargo.toml`'s are sorted: `cargo +nightly sort --workspace`. This command uses the [cargo-sort crate](https://crates.io/crates/cargo-sort) to sort the `Cargo.toml` dependencies alphabetically.
Expand Down Expand Up @@ -227,6 +233,7 @@ graph BT
First, `provisioner`, `gateway`, `deployer`, and `cargo-shuttle` are binary crates with `provisioner`, `gateway` and `deployer` being backend services. The `cargo-shuttle` binary is the `cargo shuttle` command used by users.

The rest are the following libraries:

- `common` contains shared models and functions used by the other libraries and binaries.
- `codegen` contains our proc-macro code which gets exposed to user services from `service` by the `codegen` feature flag. The redirect through `service` is to make it available under the prettier name of `shuttle_service::main`.
- `service` is where our special `Service` trait is defined. Anything implementing this `Service` can be loaded by the `deployer` and the local runner in `cargo-shuttle`.
Expand All @@ -238,20 +245,22 @@ Lastly, the `user service` is not a folder in this repository, but is the user s

## Windows Considerations

Currently, if you try to use 'make images' on Windows, you may find that the shell files cannot be read by Bash/WSL. This is due to the fact that Windows may have pulled the files in CRLF format rather than LF[^1], which causes problems with Bash as to run the commands, Linux needs the file in LF format.
Currently, if you try to use 'make images' on Windows, you may find that the shell files cannot be read by Bash/WSL. This is due to the fact that Windows may have pulled the files in CRLF format rather than LF[^1], which causes problems with Bash as to run the commands, Linux needs the file in LF format.

Thankfully, we can fix this problem by simply using the `git config core.autocrlf` command to change how Git handles line endings. It takes a single argument:

```
```bash
git config --global core.autocrlf input
```

This should allow you to run `make images` and other Make commands with no issues.

If you need to change it back for whatever reason, you can just change the last argument from 'input' to 'true' like so:
```

```bash
git config --global core.autocrlf true
```

After you run this command, you should be able to checkout projects that are maintained using CRLF (Windows) again.

[^1]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf
Loading

0 comments on commit b1c8aba

Please sign in to comment.