build(package): refactoring build scripts with project ref / incremental #5032

kwonoj · 2019-09-23T05:44:49Z

Description:
This PR aims to resolve to improve our build scripts, for perf & simplicity both. It have number of changes and aim to satisfy compat as much (non-major-breaking). Still I may able to miss some, requires eye to review changes.

Changes are below:

Use project references / incremental
npm run compile is now default command for daily dev workflow. It uses tsc's incremental and project references, will try to compile minimal changes as much as it can detect.
Simplifying npm scripts for build / package
npm run compile : just run compiler to check build passes
npm run build:package: run postprocess (rollup minify, subpath aliases..) to prepare actual package publish. Don't need to care about complex build steps anymore.
Using top-level package publishing instead of manual build steps
all build outputs are placed under dist/* and now it is expected to publish right away from top-level root of repo.
Subpath alias now generated by scripts (bulid:package)
Previously we articulated structures to lay out top-level of package to be cjs output, and then copy whole src, then manually modify sourcemap url to conform those structures - and cjs output included subpath alias package.json. Now we publish top-level right away, simply generate those alias on-the-fly when package script runs.
dist/esm2015 -> dist/esm
It's preparation of real-esm-module publishing, as it is no longer 2015. it should affect any runtime as we have module / esm2015 points correct entrypoint per each unless someone explicitly do subpath import which is unsupported sceanario.
use files in package.json
It's hard to block lot of files in .npmignore, instead we use package.json's files property to allow pick up what we want to include. This allows we can use dist to emit whatever output we want without clearing before publish as it won't be picked up ever.
emits declarationMap
now editors can navigate to source of types instead of type definition when try to locate definitions.

Since we're doing alpha for 7.0, I think it's good chance we can try these changes to collect feedback if we see any breaking changes need to be fixed before beta / public release.

kwonoj · 2019-09-23T06:53:59Z

package.json

@@ -1,8 +1,11 @@
 {
-  "name": "@reactivex/rxjs",
+  "name": "rxjs",


since this PR expects publish from top level root, pkg name property changed.

kwonoj · 2019-09-23T06:55:33Z

package.json

    "test:browser": "echo \"Browser test is not working currently\" && exit -1 && npm-run-all build:spec:browser && opn spec/support/mocha-browser-runner.html",
    "test:circular": "dependency-cruise --validate .dependency-cruiser.json -x \"^node_modules\" dist/esm5",
    "test:systemjs": "node integration/systemjs/systemjs-compatibility-spec.js",
    "test:side-effects": "check-side-effects --test integration/side-effects/side-effects.json",
    "test:side-effects:update": "npm run test:side-effects -- --update",
-    "tests2png": "mkdirp docs_app/content/img && mocha --opts spec/support/tests2png.opts \"spec/**/*-spec.ts\""
+    "tests2png": "mkdirp docs_app/content/img && mocha --opts spec/support/tests2png.opts \"spec/**/*-spec.ts\"",
+    "compile": "tsc -b --verbose ./src/tsconfig.cjs.json ./src/tsconfig.esm.json ./src/tsconfig.esm5.json ./src/tsconfig.esm5.rollup.json ./src/tsconfig.types.json ./spec/tsconfig.json",


single compile runs over all config with incremental build.

kwonoj · 2019-09-23T06:55:52Z

package.json

-    "npm": ">=2.0.0"
-  },
-  "typings": "./dist/package/Rx.d.ts"
+  "files": [


this property will allow specific files to be included in published package.

kwonoj · 2019-09-23T06:56:15Z

spec/support/default.opts

@@ -1,8 +1,8 @@
 --require ts-node/register


we still need ts-node for path mapping resolve with tsconfig-paths.

kwonoj · 2019-09-23T06:57:14Z

src/tsconfig.cjs.json

-  }
+  },
+  "exclude": [
+    "./internal/umd.ts"


cjs doesn't emit umd entrypoint anyway, but this makes compiler confuses to retry incremental build always (cause incremental build checks build output exists). simply exclude it so incremental build works properly.

kwonoj · 2019-09-23T07:19:31Z

package.json

-    "test": "cross-env TS_NODE_PROJECT=spec/tsconfig.json mocha --opts spec/support/default.opts \"spec/**/*-spec.ts\"",
-    "test_no_cache": "cross-env TS_NODE_PROJECT=spec/tsconfig.json TS_NODE_CACHE=false mocha --opts spec/support/default.opts \"spec/**/*-spec.ts\"",
-    "test_transpile_only": "cross-env TS_NODE_PROJECT=spec/tsconfig.json TS_NODE_TRANSPILE_ONLY=true mocha --opts spec/support/default.opts \"spec/**/*-spec.ts\"",
+    "test": "npm run compile && shx cp -Rf ./dist/cjs ./dist/src && mocha --opts spec/support/default.opts \"dist/spec/**/*-spec.js\"",


spec have imports to src (../src/**), instead of trying to resolve it in complex way simply just copy cjs output to src allow spec resolves it.

most notably, now mocha runs via transpiled results so no more cache related problem should occur + runs faster as compilation only runs against changed files instead of running all files using ts-node.

benlesh · 2019-09-25T20:45:12Z

cc @kolodny

- BREAKING CHANGES: esm2015 renamed to esm. (though this is undocumented) scoped package @reactivex/rxjs is being deprecated

kwonoj · 2019-09-25T21:09:26Z

What's being changed (potential breaking changes)

if you're explicitly traversing subpath esm2015, it is changed to esm: it is not supported to traverse subpath directly anyway
scoped package @reactivex/rxjs supposed to be depreacted, as this PR chanegs pkg manifest to use rxjs package name.

Package structure change

This is inner change, shouldn't create user-facing issue, but layout of structure has changed:

*Previous:

cjs was placed as top level, as well as we placed src by copying it. other build outputs are placed next to it as well.

*Changed:

Now dist contains all build outputs in its subpath, src stays on top as we package on repo root. subpath aliasing (ajax, operators...) are placed on top level as well to preserve current subpath import behavior.

cartant

LGTM and it's so much faster 🏎

The dtslint tests fail for me, but I can look at that after this is merged.

kwonoj requested review from benlesh and cartant September 23, 2019 05:44

build(tsconfig): enable incremental, project references

d5ce5ad

kwonoj force-pushed the build-dist branch from d0f19ba to 5e34b7d Compare September 23, 2019 05:52

kwonoj added 5 commits September 22, 2019 23:00

build(package): generate package using top level root as-is

040f418

test(sideeffect): update import site snapshot

dbb931b

build(package): support subpath alias with top-level root

1e95a43

ci(circle): update ci scripts

d6032b2

build(package): remove unused dependencies

c128157

kwonoj force-pushed the build-dist branch from 5e34b7d to c128157 Compare September 23, 2019 06:09

kwonoj added the target: master only label Sep 23, 2019

build(package): ensure alias script not throw

d9b08cb

kwonoj commented Sep 23, 2019

View reviewed changes

kwonoj force-pushed the build-dist branch from 41e6b0d to 0f3b9e5 Compare September 23, 2019 07:19

kwonoj commented Sep 23, 2019

View reviewed changes

kwonoj force-pushed the build-dist branch from 0f3b9e5 to 9250429 Compare September 23, 2019 07:22

test(mocha): run test directly from compiled results

32f8500

kwonoj force-pushed the build-dist branch from 9250429 to 44cc3e0 Compare September 23, 2019 07:26

build(tsconfig): fix sourceroot for source, typemaps

444e4ff

- BREAKING CHANGES: esm2015 renamed to esm. (though this is undocumented) scoped package @reactivex/rxjs is being deprecated

kwonoj force-pushed the build-dist branch from 44cc3e0 to 444e4ff Compare September 25, 2019 20:56

kwonoj mentioned this pull request Oct 2, 2019

Eliminate @reactivex/rxjs? #2916

Closed

cartant approved these changes Oct 6, 2019

View reviewed changes

benlesh approved these changes Oct 15, 2019

View reviewed changes

benlesh merged commit de6f1ef into ReactiveX:master Oct 15, 2019

lock bot locked as resolved and limited conversation to collaborators Nov 14, 2019

kwonoj deleted the build-dist branch November 19, 2019 02:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build(package): refactoring build scripts with project ref / incremental #5032

build(package): refactoring build scripts with project ref / incremental #5032

kwonoj commented Sep 23, 2019 •

edited

Loading

kwonoj Sep 23, 2019

kwonoj Sep 23, 2019

kwonoj Sep 23, 2019

kwonoj Sep 23, 2019

kwonoj Sep 23, 2019

kwonoj Sep 23, 2019

benlesh commented Sep 25, 2019

kwonoj commented Sep 25, 2019 •

edited

Loading

cartant left a comment

build(package): refactoring build scripts with project ref / incremental #5032

build(package): refactoring build scripts with project ref / incremental #5032

Conversation

kwonoj commented Sep 23, 2019 • edited Loading

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

kwonoj Sep 23, 2019

Choose a reason for hiding this comment

benlesh commented Sep 25, 2019

kwonoj commented Sep 25, 2019 • edited Loading

cartant left a comment

Choose a reason for hiding this comment

kwonoj commented Sep 23, 2019 •

edited

Loading

kwonoj commented Sep 25, 2019 •

edited

Loading