Skip to content

Commit

Permalink
Migrate to ES Module structure/runtime - Node 20, Vite, Vitest, Story…
Browse files Browse the repository at this point in the history
…book 8 (#272)

* - update lerna and node, add eslint esm preset
- update typescript and ts-node config
- file path and extension changes
- migrate away from styled-icons/boxicons due to typing issue with esm
- update flatgeobuf, remove old workarounds
- add skipLibCheck to resolve flatgeobuf internal import error
- update turf to v7 prerelease, migrate to exporting latest geojson types, tighten up geometry generics
- ignore rbush type import errors
- switch from ts-node to tsx for prepare script
- migrating use of __dirname to new method

* switch CI to use node 20

* Migrate base-project and template packages to esm

* disable run of test suite on commit until resolved

* missed test

* fix fetchPolyfill import

* switch to nodenext module and moduleResolution.  Add assert to json imports

* clean up import of CJS dependencies that don't have a true default import. Migrate use of require in some places, enough to get init working.

* switch to tsx for translation extract and migrate babel config to cjs extension

* fix report name for template-blank-project

* Switch from Jest to Vitest (#273)

* start switch to vitest

* drop jest stuff, other dirname and import dep fixes, reset lerna.json

* use default-import to use and extend rbush

* add vitest dep formally, add vitest-setup loading jest-dom

* Migrate more tests, make them runnable standalone in vscode debug with direct import

* update geoblaze, update overlap output changes related to turf area. skip many remaining tests

* fix prepare

* comment out and skip tests with jest mocks

* initial migrate of scripts to vitest, skipping failing precalc datasource tests

* stub vitest project runner

* clean up vitest config type error, and punt on browser test mock error and bump vi* versions

* skip remaining failing tests, add back test commit gate

* try making vitest regular dependency

* update runner to config project vitest

* merge cli options

* fix vitest cli command

* switch to TEST_ROOT env var

* shift jsdom dep for project use

* fix TEST_ROOT

* simplify test commands, tune test config

* update vitest comments

* add data server back to test command

* vitest comments

* drop ts-node for tsx

* add testing of base and template packages, skipping most.  Migrate remaining commands to vitest. Migrate to turf 7 in projects

* fix type error, upgrade node-fetch, fix launcher paths, VectorDatasource not fetching properly?

* add new world fgb and confirm test passing with latest unstable gp docker workspace

* downgrade flatgeobuf to resolve timeout error, add back skipped tests

* migrate to new import.meta.dirname available with node 20.12

* update fetch polyfill and upgrade again to latest flatgeobuf.  subdivided datasource fetch also resolved.

* unskip remaining tests utilizing flatgeobuf, update sideEffects for fetch polyfill

* Switch to esbuild (#278)

* convert function builder to esbuild

* fix other function output dir, cleanup negative zero in cleanCoords

* add buildClient using esbuild with html plugin, refactor App to receive lazy loaded report clients, fix loose client-ui exports

* add esbuild polyfill, address esbuild strict module import - pare down projectClient import paths, switch client away from using top-level gp lib import, shift top-level indexes to prevent pollution of node code in ui code, guard use of process env var

* Switch to generating ReportApp.tsx in .build-web with lazy load of report clients

* on build copy language assets to .build-web

* bring back template smoke tests

* AWS - deploy pure ESM code for Lambda and client, on node 20 (#279)

* upgrade aws-cdk-lib and aws-sdk, add package.json to build to get esm enabled in lambda, move esbuild dep to top of monorepo, add node 20 runtime

* update data scripts to work in monorepo

* fix precalc script env

* build - create isolated mini package per function with own entry point. Add type assertion to language json import

* upgrade zx, refactor translation install and extract to use scripts clarify build script

* add missing esbuild plugins add missing babel plugin for translation extract, add client polyfill globals process true

* cleanup isHostedOnLambda

* process empty

* bump

* directly set client lambda env vars

* add esbuild metafile output

* add ANALYZE  env vars

* fix metafile output path

* Switch to gp core imports in client to not pull backend modules

* Allow minify to be overridden, defaults to true

* fix minify arg passing

* add cloufront access to client bucket to be able to access client files in directories.  Fix dynamic import of translations in browser client

* update translationInstall to be more comprehensive

* update dirname in importTerms and publishTerms, loosen gitignore on i18n dir

* Make sure .build dir created, remove old createManifest

* bump

* simplify generateHandler, add guard to .build dir copy

* Cut out intermediate build in gp .build dir, go straight to project path .build

* simplify build client gp path discovery

* make build client use gp dependencies. removes need for esbuild deps in project

* replace use of esbuild-plugin-html which doesn't support entry point outside of current directory

* upgrade to typescript 5.4.5

* swtich from assert json to with json

* fully switch from assert to with import syntax, esbuild still converts to older assert syntax but should change at some point

* rebuild package lockfile, major non-breaking npm upgrades

* additional upgrades:
- polygon-clipping upgrade resulted in additional vertice in some clip results slightly changing value (<.04 sq meters).
- Update base translations. Fix translations extract with babel import assertion plugin.

* upgrade babel proposal plugins to official

* update husky prepare command, husky install is deprecated

* migrate to new styled-components .attrs() to define caller-defined attributes to pass to components

* switch to transient prop for duration ProgressBar

* Vite for client dev, build, storybook (#282)

* switch to transient prop for duration ProgressBar

* start conversion to Vite for client build, still need to fix import attributes and add dev server

* Migrate start:client to Vite, language dynamic import not working or keeping with:json import assertion

* Migrate gp lib to storybook 8 using Vite, missing project storybook still

* drop babel from gp vite config, causing require error

* add back storybook to build docs

* migrate from start-storybook to just storybook

* refactor report stories to use code generator with story-cache

* add story-cache to base project gitignore

* change report-stories to example-stories. Migrate templates to use them.  reports required to use named export

* migrate create:report to node 20 patterns.  Fix classColumn type error

* glob delete story caches on startup, add vite types using d.ts file for asset import

* switch to Vite glob loader for language modules

* no need to copy language modules into build-web anymore

* add vite to base-project

* export pathUtils, simplify tsconfig

* Drop unused TranslatorSync component

* shift storybook to dependencies for project user

* Change story folder structure

* Add turf/helpers to base-project since common project use

* add all to dependencies to try and solve storybook start error

* export datasource helpers, remove unused vitest workspace, update min node and npm in package.json

* Make project storybook+vite independent of gp version

* make styled-components props transient, drop unused main from base-project package.json

* vite configure web worker

* remove vite worker config since web worker not being used

* fix story sort order

* fix blank-project functions

* attempt fix to favicon install command

* add storybook install script for migraters.  Add initial Migrating docs

* add back default export to report clients require for React.lazy load

* nit Migrating
  • Loading branch information
twelch committed Jun 25, 2024
1 parent 1cd2aea commit 4a78caf
Show file tree
Hide file tree
Showing 511 changed files with 37,309 additions and 240,442 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
node-version: [20.x]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-test-api-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
node-version: [20.x]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-api-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
node-version: [20.x]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,6 @@ packages/example-*/
report.*
test-projects
**/.env
packages/*/src/i18n
packages/base-project/src/i18n
packages/template-*/src/i18n
.story-cache
3 changes: 2 additions & 1 deletion .husky/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
. "$(dirname -- "$0")/_/husky.sh"

npx lerna run prepare # transpiles core geoprocessing code for templates
npx lerna run test:local # tests all packages including e2e tests not run by Github CI (GDAL, etc.)
# run tests including e2e tests not run by Github CI (GDAL, etc.)
npx lerna run test:local
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
16
20
83 changes: 41 additions & 42 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,39 @@
"configurations": [

{
"name": "geoprocessing - debug tests",
"name": "geoprocessing - debug src tests",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--group=-e2e"
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.node.js"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "geoprocessing - debug tests with pattern",
"type": "node",
"request": "launch",
"name": "geoprocessing - debug current test file",
"autoAttachChildProcesses": true,
"skipFiles": ["<node_internals>/**", "**/node_modules/**"],
"program": "${workspaceRoot}/node_modules/vitest/vitest.mjs",
"args": ["run", "${relativeFile}"],
"smartStep": true,
"console": "integratedTerminal"
}, {
"name": "geoprocessing - debug src tests with pattern",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.node.js",
"-t",
"${input:testNamePattern}"
],
Expand All @@ -34,80 +47,66 @@
"port": 9229
},
{
"name": "geoprocessing - debug tests scripts",
"name": "geoprocessing - debug scripts tests",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--projects",
"scripts"
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.scripts.node.js"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "geoprocessing - debug tests scripts with pattern",
"name": "geoprocessing - debug scripts tests with pattern",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--projects",
"scripts",
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.scripts.node.js",
"-t",
"${input:testNamePattern}"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "geoprocessing - debug e2e tests",
"name": "geoprocessing - debug src e2e tests",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--runInBand",
"--group=e2e"
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.e2e.js"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "geoprocessing - debug e2e tests with pattern",
"name": "geoprocessing - debug src e2e tests with pattern",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--runInBand",
"--group=e2e",
"${workspaceRoot}/node_modules/.bin/vitest",
"-c",
"vitest.config.e2e.js",
"-t",
"${input:testNamePattern}"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "geoprocessing scripts- debug tests",
"type": "node",
"request": "launch",
"cwd": "${workspaceFolder}/packages/geoprocessing",
"runtimeArgs": [
"--inspect-brk",
"${workspaceRoot}/packages/geoprocessing/node_modules/.bin/jest",
"--projects",
"scripts"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"port": 9229
}, {
"name": "template-ocean-eez - debug tests",
"type": "node",
"request": "launch",
Expand All @@ -125,8 +124,8 @@
"request": "launch",
"name": "translation - debug importTerms",
"runtimeArgs": [
"-r",
"ts-node/register",
"--import",
"tsx",
],
"args": [
"${workspaceRoot}/packages/geoprocessing/src/i18n/bin/importTerms.ts"
Expand All @@ -136,8 +135,8 @@
"request": "launch",
"name": "translation - debug publishTerms",
"runtimeArgs": [
"-r",
"ts-node/register",
"--import",
"tsx",
],
"args": [
"${workspaceRoot}/packages/geoprocessing/src/i18n/bin/publishTerms.ts"
Expand All @@ -162,8 +161,8 @@
"request": "launch",
"name": "run any ts-node script",
"runtimeArgs": [
"-r",
"ts-node/register/transpile-only",
"--import",
"tsx",
],
"args": [
"${workspaceRoot}/packages/geoprocessing/${input:scriptPath}"
Expand All @@ -180,7 +179,7 @@
{
"type": "promptString",
"id": "testNamePattern",
"description": "jest -t 'regex pattern to select tests'",
"description": "vitest -t 'regex pattern to select tests'",
"default": "name of my test"
},
{
Expand Down
4 changes: 2 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
},
"files.exclude": {},
"git.ignoreLimitWarning": true,
"jest.rootPath": "packages/geoprocessing",
"workbench.colorCustomizations": {
"activityBar.activeBackground": "#004b99",
"activityBar.activeBorder": "#ff007d",
Expand Down Expand Up @@ -61,5 +60,6 @@
"i18n-ally.keysInUse": [
"description.part2_whatever"
],
"i18n-ally.defaultNamespace": "translation"
"i18n-ally.defaultNamespace": "translation",
"vitest.disableWorkspaceWarning": true
}
1 change: 1 addition & 0 deletions babel.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ module.exports = {
keyAsDefaultValueForDerivedKeys: true,
},
],
["@babel/plugin-syntax-import-attributes"],
],
overrides: [
{ test: /\.js$/, presets: common },
Expand Down
4 changes: 1 addition & 3 deletions lerna.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"packages": [
"packages/*"
],
"$schema": "node_modules/lerna/schemas/lerna-schema.json",
"version": "6.1.2"
}
Loading

0 comments on commit 4a78caf

Please sign in to comment.