chore(CI): improve workflow merging steps and parallelizing image build #37352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

sampaiodiego merged 6 commits into develop from improve/ci

Nov 5, 2025

.github/actions/build-docker/action.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -13,18 +13,10 @@ inputs: @@
         required: true
         description: 'Deno version'
         type: string
-      platform:
+      service:
         required: false
-        description: 'Platform'
-        default: 'alpine'
-      build-containers:
-        required: false
-        description: 'Containers to build along with Rocket.Chat'
+        description: 'Container to build'
         type: string
-      turbo-cache:
-        required: false
-        description: 'Enable turbo cache'
-        default: 'true'
       publish-image:
         required: false
         description: 'Publish image'
@@ Expand All / @@ -36,6 +28,10 @@ inputs: @@
       NPM_TOKEN:
         required: false
         description: 'NPM token'
+      type:
+        required: false
+        description: 'production or coverage'
+        default: 'production'
     runs:
       using: composite
@@ Expand All / @@ -49,61 +45,55 @@ runs: @@
             username: ${{ inputs.CR_USER }}
             password: ${{ inputs.CR_PAT }}
-        - name: Restore build
+        - name: Restore packages build
           uses: actions/download-artifact@v4
           with:
-            name: build
-            path: /tmp/build
+            name: packages-build
+            path: /tmp
-        - name: Unpack build
+        - name: Unpack packages build
           shell: bash
           run: |
-            cd /tmp/build
-            tar xzf Rocket.Chat.tar.gz
-            rm Rocket.Chat.tar.gz
+            tar -xzf /tmp/RocketChat-packages-build.tar.gz -C .
-        - uses: rharkor/[email protected]
-          # if we are testing a PR from a fork, we already called the turbo cache at this point, so it should be false
-          if: inputs.turbo-cache == 'true'
-        - name: Setup NodeJS
-          uses: ./.github/actions/setup-node
-          if: inputs.setup == 'true'
-          with:
-            node-version: ${{ inputs.node-version }}
-            deno-version: ${{ inputs.deno-version }}
-            cache-modules: true
-            install: true
-            NPM_TOKEN: ${{ inputs.NPM_TOKEN }}
-        - name: Restore turbo build
+        - name: Restore meteor build
+          if: inputs.service == 'rocketchat'
           uses: actions/download-artifact@v4
           with:
-            name: turbo-build
-            path: .turbo/cache
+            name: build-${{ inputs.type }}
+            path: /tmp/build
-        - run: yarn build
-          if: inputs.setup == 'true'
+        - name: Unpack meteor build
+          if: inputs.service == 'rocketchat'
           shell: bash
+          run: |
+            cd /tmp/build
+            tar xzf Rocket.Chat.tar.gz
+            rm Rocket.Chat.tar.gz
         - name: Build Docker images
           shell: bash
           run: |
-            args=(rocketchat ${{ inputs.build-containers }})
             export DENO_VERSION="${{ inputs.deno-version }}"
-            docker compose -f docker-compose-ci.yml build "${args[@]}"
+            docker compose -f docker-compose-ci.yml build ${{ inputs.service }}
         - name: Publish Docker images to GitHub Container Registry
           if: inputs.publish-image == 'true' && github.actor != 'dependabot[bot]' && (github.event.pull_request.head.repo.full_name == github.repository || github.event_name == 'release' || github.ref == 'refs/heads/develop')
           shell: bash
           run: |
-            args=(rocketchat ${{ inputs.build-containers }})
+            set -o xtrace
+            # Get image name from docker-compose-ci.yml since rocketchat image is different from service name (rocket.chat)
+            IMAGE=$(docker compose -f docker-compose-ci.yml config --format json 2>/dev/null | jq -r --arg s "${{ inputs.service }}" '.services[$s].image')
+            IMAGE_NO_TAG=$(echo "$IMAGE"  | sed 's/:.*$//')
+            docker tag ${IMAGE} ${IMAGE}-gha-run-${{ github.run_id }}
-            docker compose -f docker-compose-ci.yml push "${args[@]}"
+            docker push --all-tags ${IMAGE_NO_TAG}
         - name: Clean up temporary files
+          if: inputs.service == 'rocketchat'
           shell: bash
           run: |
-            sudo rm -rf /tmp/bundle
+            sudo rm -rf /tmp/build

.github/actions/meteor-build/action.yml

-Original file line number
+Diff line change
@@ -1,10 +1,10 @@
     name: 'Meteor Build'
     inputs:
-      coverage:
+      type:
         required: false
-        description: 'Enable coverage'
-        type: boolean
+        description: 'production or coverage'
+        default: 'production'
       reset-meteor:
         required: false
         description: 'Reset Meteor'
@@ Expand All / @@ -20,18 +20,33 @@ inputs: @@
         required: true
         description: 'Deno version'
         type: string
+      source-hash:
+        required: true
+        description: 'Source code hash'
+        type: string
     runs:
       using: composite
       steps:
+        - name: Cache build
+          uses: actions/cache@v4
+          id: cache-build
+          with:
+            path: /tmp/Rocket.Chat.tar.gz
+            key: ${{ runner.OS }}-rc-build-${{ inputs.source-hash }}
+            restore-keys: |
+              ${{ runner.os }}-rc-build-
         - name: Set Swap Space
           uses: pierotofy/set-swap-space@master
+          if: steps.cache-build.outputs.cache-hit != 'true'
           with:
             swap-size-gb: 4
         - name: Setup NodeJS
           uses: ./.github/actions/setup-node
+          if: steps.cache-build.outputs.cache-hit != 'true'
           with:
             node-version: ${{ inputs.node-version }}
             deno-version: ${{ inputs.deno-version }}
@@ Expand All / @@ -47,6 +62,7 @@ runs: @@
         - name: Cache vite
           uses: actions/cache@v3
+          if: steps.cache-build.outputs.cache-hit != 'true'
           with:
             path: ./node_modules/.vite
             key: vite-local-cache-${{ runner.OS }}-${{ hashFiles('package.json') }}
@@ Expand All / @@ -55,6 +71,7 @@ runs: @@
         - name: Cache meteor local
           uses: actions/cache@v3
+          if: steps.cache-build.outputs.cache-hit != 'true'
           with:
             path: ./apps/meteor/.meteor/local
             key: meteor-local-cache-${{ runner.OS }}-${{ hashFiles('apps/meteor/.meteor/versions') }}
@@ Expand All / @@ -63,6 +80,7 @@ runs: @@
         - name: Cache meteor
           uses: actions/cache@v3
+          if: steps.cache-build.outputs.cache-hit != 'true'
           with:
             path: ~/.meteor
             key: meteor-cache-${{ runner.OS }}-${{ hashFiles('apps/meteor/.meteor/release') }}
@@ Expand All / @@ -71,6 +89,7 @@ runs: @@
         - name: Install Meteor
           shell: bash
+          if: steps.cache-build.outputs.cache-hit != 'true'
           run: |
             # Restore bin from cache
             set +e
@@ Expand All / @@ -91,6 +110,7 @@ runs: @@
         - name: Versions
           shell: bash
+          if: steps.cache-build.outputs.cache-hit != 'true'
           run: |
             npm --versions
             yarn -v
@@ Expand All / @@ -100,19 +120,31 @@ runs: @@
             meteor node -v
             git version
-        - uses: rharkor/caching-for-turbo@v1.5
+        - uses: rharkor/caching-for-turbo@v1.8
         - name: Reset Meteor
           shell: bash
-          if: ${{ inputs.reset-meteor == 'true' }}
+          if: ${{ steps.cache-build.outputs.cache-hit != 'true' && inputs.reset-meteor == 'true' }}
           working-directory: ./apps/meteor
           run: meteor reset
+        - name: Restore packages build
+          uses: actions/download-artifact@v4
+          with:
+            name: packages-build
+            path: /tmp
+        - name: Unpack packages build
+          shell: bash
+          run: |
+            tar -xzf /tmp/RocketChat-packages-build.tar.gz -C .
         - name: Build Rocket.Chat
           shell: bash
+          if: steps.cache-build.outputs.cache-hit != 'true'
           env:
             METEOR_PROFILE: 1000
-            BABEL_ENV: ${{ inputs.coverage == 'true' && 'coverage' || '' }}
+            BABEL_ENV: ${{ inputs.type }}
           run: |
             # check if BABEL_ENV is set to coverage
             if [[ $BABEL_ENV == "coverage" ]]; then
@@ Expand All / @@ -124,18 +156,20 @@ runs: @@
         - name: Translation check
           shell: bash
+          if: steps.cache-build.outputs.cache-hit != 'true'
           run: yarn turbo run translation-check
         - name: Prepare build
           shell: bash
+          if: steps.cache-build.outputs.cache-hit != 'true'
           run: |
             cd /tmp/dist
             tar czf /tmp/Rocket.Chat.tar.gz bundle
         - name: Store build
           uses: actions/upload-artifact@v4
           with:
-            name: build
+            name: build-${{ inputs.type }}
             path: /tmp/Rocket.Chat.tar.gz
             overwrite: true
             include-hidden-files: true

.github/workflows/ci-code-check.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -41,19 +41,19 @@ jobs: @@
               install: true
               NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
-          # - name: Free disk space
-          #   run: |
-          #     sudo apt clean
-          #     docker rmi $(docker image ls -aq)
-          #     df -h
           - uses: rharkor/[email protected]
-          - name: Restore turbo build
+          - name: Restore packages build
             uses: actions/download-artifact@v4
             with:
-              name: turbo-build
-              path: .turbo/cache
+              name: packages-build
+              path: /tmp
+          - name: Unpack packages build
+            shell: bash
+            run: |
+              tar -xzf /tmp/RocketChat-packages-build.tar.gz -C .
           - name: Cache TypeCheck
             uses: actions/cache@v4
             if: matrix.check == 'ts'
@@ Expand Down @@

.github/workflows/ci-test-e2e.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -12,12 +12,6 @@ on:
  
          lowercase-repo:

            required: true

            type: string

          rc-dockerfile:

            required: true

            type: string

          rc-docker-tag:

            required: true

            type: string

          gh-docker-tag:

            required: true

            type: string

    @@ -72,21 +66,19 @@ env:
  
      TOOL_NODE_FLAGS: ${{ vars.TOOL_NODE_FLAGS }}

      LOWERCASE_REPOSITORY: ${{ inputs.lowercase-repo }}

      DOCKER_TAG: ${{ inputs.gh-docker-tag }}

      DOCKER_TAG_SUFFIX_ROCKETCHAT: '-cov'

    jobs:

      test:

        runs-on: ubuntu-24.04

        env:

          RC_DOCKERFILE: ${{ inputs.rc-dockerfile }}.${{ (matrix.mongodb-version == '8.2' && 'debian' && false) || 'alpine' }}

          RC_DOCKER_TAG: ${{ inputs.rc-docker-tag }}.${{ (matrix.mongodb-version == '8.2' && 'debian' && false) || 'alpine' }}

        strategy:

          fail-fast: false

          matrix:

            mongodb-version: ${{ fromJSON(inputs.mongodb-version) }}

            shard: ${{ fromJSON(inputs.shard) }}

        name: MongoDB ${{ matrix.mongodb-version }}${{ inputs.db-watcher-disabled == 'false' && ' [legacy watchers]' || '' }} (${{ matrix.shard }}/${{ inputs.total-shard }}) - ${{ (matrix.mongodb-version == '8.2' && 'Debian' && false) || 'Alpine (Official)' }}

        name: MongoDB ${{ matrix.mongodb-version }}${{ inputs.db-watcher-disabled == 'false' && ' [legacy watchers]' || '' }} (${{ matrix.shard }}/${{ inputs.total-shard }})

        steps:

          - name: Collect Workflow Telemetry

    @@ -133,13 +125,16 @@ jobs:
  
          - uses: rharkor/[email protected]

          - name: Restore turbo build

          - name: Restore packages build

            uses: actions/download-artifact@v4

            with:

              name: turbo-build

              path: .turbo/cache

              name: packages-build

              path: /tmp

          - run: yarn build

          - name: Unpack packages build

            shell: bash

            run: |

              tar -xzf /tmp/RocketChat-packages-build.tar.gz -C .

          # if we are testing a PR from a fork, we need to build the docker image at this point

          - uses: ./.github/actions/build-docker

    @@ -148,11 +143,10 @@ jobs:
  
              CR_USER: ${{ secrets.CR_USER }}

              CR_PAT: ${{ secrets.CR_PAT }}

              node-version: ${{ inputs.node-version }}

              # we already called the turbo cache at this point, so it should be false

              turbo-cache: false

              # the same reason we need to rebuild the docker image at this point is the reason we dont want to publish it

              publish-image: false

              setup: false

              service: 'rocketchat'

              NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

          - name: Set DEBUG_LOG_LEVEL (debug enabled)

.github/workflows/ci-test-storybook.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -37,11 +37,16 @@ jobs: @@
           - uses: rharkor/[email protected]
-          - name: Restore turbo build
+          - name: Restore packages build
             uses: actions/download-artifact@v4
             with:
-              name: turbo-build
-              path: .turbo/cache
+              name: packages-build
+              path: /tmp
+          - name: Unpack packages build
+            shell: bash
+            run: |
+              tar -xzf /tmp/RocketChat-packages-build.tar.gz -C .
           - uses: ./.github/actions/setup-playwright
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(CI): improve workflow merging steps and parallelizing image build #37352

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!