Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: hook up ci to publish to npm #226

Merged
merged 40 commits into from
Jan 31, 2019
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
c92f420
chore: add missing fields to package.json
Jan 28, 2019
a80b87a
feat: add postlight org scope to package name
Jan 28, 2019
1366d82
chore: yarn lock file
Jan 28, 2019
b067b56
chore: bump version
Jan 28, 2019
d34aebe
v1.0.15
Jan 28, 2019
7acef8c
feat: automate npm publish
JadTermsani Jan 29, 2019
0d455ac
test: npm publish without filters
JadTermsani Jan 29, 2019
1fb7503
fix: add docker image
JadTermsani Jan 29, 2019
e36c705
test: change directory
JadTermsani Jan 29, 2019
7b6d7d7
test: add working directory
JadTermsani Jan 29, 2019
7ad046b
fix: defaults sytax
JadTermsani Jan 29, 2019
ea7f431
test: add workspace
JadTermsani Jan 29, 2019
438c41a
fix: attach workspace
JadTermsani Jan 29, 2019
8677812
Merge branch 'feat-deploy-to-npm' of https://github.com/postlight/mer…
Jan 29, 2019
3e1448c
chore: bump version
JadTermsani Jan 29, 2019
be6bfda
Merge branch 'feat-deploy-to-npm' of https://github.com/postlight/mer…
Jan 29, 2019
e13b985
fix: use standard mercury email
Jan 30, 2019
fa08b8c
Merge branch 'master' into feat-deploy-to-npm
george-haddad Jan 30, 2019
c6c7082
fix: use ISO time format and preserve original timezone offset
Jan 30, 2019
ae81557
fix: do not match time zone offset
Jan 30, 2019
cdc2c86
chore: move babel runtime-corejs2 to prod deps
Jan 30, 2019
fa32561
chore: version bump
Jan 30, 2019
6a14d38
chore: version bump and yarn.lock file
Jan 30, 2019
2000987
chore: build:web first before test:web
Jan 30, 2019
250f157
Revert "chore: build:web first before test:web"
Jan 30, 2019
1d08261
Revert "chore: version bump and yarn.lock file"
Jan 30, 2019
43eef3f
chore: version bump
Jan 30, 2019
2c044f0
chore: test package without tree-shaking
Jan 30, 2019
c0a0a32
chore: version bump
Jan 30, 2019
0570530
Revert "chore: test package without tree-shaking"
Jan 30, 2019
e4bbf8d
chore: browser test
Jan 30, 2019
342dff1
Revert "chore: browser test"
Jan 30, 2019
c5aa595
chore: version bump and bundledDependencies
Jan 30, 2019
311dc33
chore: uncomment config to deploy on git tag
Jan 30, 2019
0e2e78a
feat: publish to npm public
Jan 30, 2019
6cc53c9
Merge branch 'master' into feat-deploy-to-npm
adampash Jan 30, 2019
ae8fa2e
adding browser-request
adampash Jan 30, 2019
f552d86
Merge branch 'master' into feat-deploy-to-npm
adampash Jan 31, 2019
4735218
chore: roll version back to original state
Jan 31, 2019
d4bbd9b
Merge branch 'feat-deploy-to-npm' of https://github.com/postlight/mer…
Jan 31, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 46 additions & 18 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,46 +1,74 @@
version: 2

defaults: &defaults
working_directory: ~/repo
docker:
- image: circleci/node:8.10-browsers

jobs:
test-node:
docker:
- image: circleci/node:8.10-browsers
<<: *defaults
steps:
- checkout
# For some reason phantomjs-prebuild is failing w/yarn, but npm installing works
- run: "npm install phantomjs-prebuilt"
- run: "yarn install"
- run: "yarn lint:ci"
- run: "yarn build:ci"
- run: "yarn test:node --maxWorkers=4"
- run: "./scripts/pr-parser-preview.sh"

- run: 'npm install phantomjs-prebuilt'
- run: 'yarn install'
- run: 'yarn lint:ci'
- run: 'yarn build:ci'
- run: 'yarn test:node --maxWorkers=4'
- run: './scripts/pr-parser-preview.sh'
- store_artifacts:
path: tmp/artifacts
- persist_to_workspace:
root: ~/repo
paths: .

test-web:
docker:
- image: circleci/node:8.10-browsers
<<: *defaults
steps:
- attach_workspace:
at: ~/repo
- checkout
- run: "yarn install"
- run: "yarn add karma-cli --dev"
- run: "yarn test:web --maxWorkers=4"
- run: "yarn build:web:ci --maxWorkers=4"
- run: 'yarn install'
- run: 'yarn add karma-cli --dev'
- run: 'yarn test:web --maxWorkers=4'
- run: 'yarn build:web:ci --maxWorkers=4'

deploy:
<<: *defaults
steps:
- attach_workspace:
at: ~/repo
- run:
name: Authenticate with registry
command: echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/repo/.npmrc
- run:
name: Publish package
command: npm publish --access public

update-fixtures:
docker:
- image: circleci/node:8.10
steps:
- checkout
- run: "yarn install"
- run: "node ./scripts/update-fixtures.js"

- run: 'yarn install'
- run: 'node ./scripts/update-fixtures.js'

workflows:
version: 2
build_and_test:
jobs:
- test-node
- test-web
- deploy:
requires:
- test-node
- test-web
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
# UNCOMMENT WHEN READY TO AUTOMATE THIS TASK
# scheduled_tasks:
# triggers:
Expand Down
43 changes: 32 additions & 11 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,24 @@
{
"name": "mercury-parser",
"version": "1.0.13",
"description": "",
"repository": "github:postlight/mercury-parser",
"name": "@postlight/mercury-parser",
"version": "1.0.20",
"description": "Mercury transforms web pages into clean text. Publishers and programmers use it to make the web make sense, and readers use it to read any web article comfortably.",
"author": "Postlight <[email protected]>",
"homepage": "https://mercury.postlight.com",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/postlight/mercury-parser.git"
},
"bugs": {
"url": "https://github.com/postlight/mercury-parser/issues"
},
"keywords": [
"mercury",
"parser",
"reader",
"web",
"content"
],
"main": "./dist/mercury.js",
"scripts": {
"lint": "eslint . --fix",
Expand All @@ -25,8 +41,9 @@
"add-contributor": "all-contributors add",
"generate-contributors": "all-contributors generate"
},
"author": "Postlight",
"license": "ISC",
"engines": {
"node": ">=8.10"
},
"devDependencies": {
"@babel/core": "^7.0.0",
"@babel/plugin-external-helpers": "^7.2.0",
Expand All @@ -39,7 +56,6 @@
"@babel/polyfill": "^7.0.0",
"@babel/preset-env": "^7.0.0",
"@babel/runtime": "^7.0.0",
"@babel/runtime-corejs2": "^7.2.0",
"@jesses/circle-github-bot": "^2.1.0",
"@octokit/rest": "^16.9.0",
"all-contributors-cli": "^5.10.0",
Expand Down Expand Up @@ -94,22 +110,27 @@
"watchify": "^3.11.0"
},
"dependencies": {
"browser-request": "postlight/browser-request#feat-add-headers-to-response",
"browser-request": "github:postlight/browser-request#feat-add-headers-to-response",
"cheerio": "^0.22.0",
"difflib": "postlight/difflib.js",
"difflib": "github:postlight/difflib.js",
"ellipsize": "0.1.0",
"iconv-lite": "0.4.24",
"jquery": "^3.3.1",
"moment": "^2.23.0",
"moment-parseformat": "3.0.0",
"moment-timezone": "0.5.23",
"request": "czardoz/request",
"request": "github:czardoz/request",
"request-promise": "^4.2.2",
"string-direction": "^0.1.2",
"url": "^0.11.0",
"valid-url": "^1.0.9",
"wuzzy": "^0.1.3"
"wuzzy": "^0.1.3",
"@babel/runtime-corejs2": "^7.2.0"
},
"bundleDependencies": [
"jquery",
"moment-timezone"
],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've not done this before, but wouldn't bundleDependencies include all of the browser dependencies below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the node_modules directory of the package and it only included those 2 single packages. The test create-react-app won't start without them and this was the only way to get them bundled in the npm package.

"browser": {
"main": "./dist/mercury.web.js",
"cheerio": "./src/shims/cheerio-query",
Expand Down
77 changes: 47 additions & 30 deletions src/extractors/custom/www.fastcompany.com/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ describe('WwwFastcompanyComExtractor', () => {
beforeAll(() => {
url =
'https://www.fastcompany.com/3067012/the-only-five-email-folders-your-inbox-will-ever-need';
const html =
fs.readFileSync('./fixtures/www.fastcompany.com/1547124373499.html');
result =
Mercury.parse(url, html, { fallback: false });
const html = fs.readFileSync(
'./fixtures/www.fastcompany.com/1547124373499.html'
);
result = Mercury.parse(url, html, { fallback: false });
});

it('is selected properly', () => {
Expand All @@ -30,53 +30,62 @@ describe('WwwFastcompanyComExtractor', () => {
});

it('returns the title', async () => {
// To pass this test, fill out the title selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
// To pass this test, fill out the title selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
const { title } = await result;

// Update these values with the expected values from
// the article.
assert.equal(title, 'The Only Five Email Folders Your Inbox Will Ever Need');
// Update these values with the expected values from
// the article.
assert.equal(
title,
'The Only Five Email Folders Your Inbox Will Ever Need'
);
});

it('returns the author', async () => {
// To pass this test, fill out the author selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
// To pass this test, fill out the author selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
const { author } = await result;

// Update these values with the expected values from
// the article.
// Update these values with the expected values from
// the article.
assert.equal(author, 'Zach Hanlon');
});

it('returns the date_published', async () => {
// To pass this test, fill out the date_published selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
// To pass this test, fill out the date_published selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
const { date_published } = await result;

// Update these values with the expected values from
// the article.
assert.equal(date_published, '2017-01-09T05:00:00.000Z');
// Update these values with the expected values from
// the article.
assert.equal(date_published.split('T')[0], '2017-01-09');
});

it('returns the dek', async () => {
// To pass this test, fill out the dek selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
// To pass this test, fill out the dek selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
const { dek } = await result;

// Update these values with the expected values from
// the article.
assert.equal(dek, 'Stop “organizing” your emails by subject and start thinking of them in terms of deadlines.');
// Update these values with the expected values from
// the article.
assert.equal(
dek,
'Stop “organizing” your emails by subject and start thinking of them in terms of deadlines.'
);
});

it('returns the lead_image_url', async () => {
// To pass this test, fill out the lead_image_url selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
// To pass this test, fill out the lead_image_url selector
// in ./src/extractors/custom/www.fastcompany.com/index.js.
const { lead_image_url } = await result;

// Update these values with the expected values from
// the article.
assert.equal(lead_image_url, 'https://images.fastcompany.net/image/upload/w_1280,f_auto,q_auto,fl_lossy/fc/3067012-poster-p-1-the-only-five-email-folders-your-inbox-will-ever-need.jpg');
// Update these values with the expected values from
// the article.
assert.equal(
lead_image_url,
'https://images.fastcompany.net/image/upload/w_1280,f_auto,q_auto,fl_lossy/fc/3067012-poster-p-1-the-only-five-email-folders-your-inbox-will-ever-need.jpg'
);
});

it('returns the content', async () => {
Expand All @@ -88,11 +97,19 @@ describe('WwwFastcompanyComExtractor', () => {

const $ = cheerio.load(content || '');

const first13 = excerptContent($('*').first().text(), 13);
const first13 = excerptContent(
$('*')
.first()
.text(),
13
);

// Update these values with the expected values from
// the article.
assert.equal(first13, 'For years, my approach to email was like slaying a hydra. For every');
assert.equal(
first13,
'For years, my approach to email was like slaying a hydra. For every'
);
});
});
});
6 changes: 3 additions & 3 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1511,7 +1511,7 @@ browser-process-hrtime@^0.1.2:
version "0.1.3"
resolved "https://registry.yarnpkg.com/browser-process-hrtime/-/browser-process-hrtime-0.1.3.tgz#616f00faef1df7ec1b5bf9cfe2bdc3170f26c7b4"

browser-request@postlight/browser-request#feat-add-headers-to-response:
"browser-request@github:postlight/browser-request#feat-add-headers-to-response":
version "0.3.2"
resolved "https://codeload.github.com/postlight/browser-request/tar.gz/38faa5b85741aabfca61aa37d1ef044d68969ddf"
dependencies:
Expand Down Expand Up @@ -2549,7 +2549,7 @@ diffie-hellman@^5.0.0:
miller-rabin "^4.0.0"
randombytes "^2.0.0"

difflib@postlight/difflib.js:
"difflib@github:postlight/difflib.js":
version "0.2.6"
resolved "https://codeload.github.com/postlight/difflib.js/tar.gz/32e8e38c7fcd935241b9baab71bb432fd9b166ed"
dependencies:
Expand Down Expand Up @@ -7115,7 +7115,7 @@ request@^2.87.0:
tunnel-agent "^0.6.0"
uuid "^3.3.2"

request@czardoz/request:
"request@github:czardoz/request":
version "2.74.1"
resolved "https://codeload.github.com/czardoz/request/tar.gz/6ebb4651b72bee7292e54943adfa6b503465adfb"
dependencies:
Expand Down