Skip to content

Commit

Permalink
fix: nytimes custom parser title selector (#181)
Browse files Browse the repository at this point in the history
* fix: nytimes custom parser title selector

* upgrade node version

* circle ci tweak
  • Loading branch information
adampash authored Oct 12, 2018
1 parent 7fcd9b6 commit 5663660
Show file tree
Hide file tree
Showing 4 changed files with 83 additions and 3 deletions.
5 changes: 3 additions & 2 deletions circle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ machine:

node:
version:
4.3.2
6.10

## Customize dependencies
dependencies:
pre:
- curl -o- -L https://yarnpkg.com/install.sh | bash
- nvm install 7.0.0
- nvm install 6.10.0
# For some reason phantomjs-prebuild is failing w/yarn, but npm installing works
- npm install phantomjs-prebuilt

Expand All @@ -29,5 +30,5 @@ test:
- nvm use 7.0 && yarn test:web -- --maxWorkers=4 && yarn build:web -- --maxWorkers=4:
parallel: true
# Switch to check on 4.3.2
- nvm use 4.3.2 && yarn build && yarn test:node -- --maxWorkers=4:
- nvm use 6.10 && yarn build && yarn test:node -- --maxWorkers=4:
parallel: true
69 changes: 69 additions & 0 deletions fixtures/www.nytimes.com/1539194812689.html

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion src/extractors/custom/www.nytimes.com/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ export const NYTimesExtractor = {

title: {
selectors: [
'.g-headline',
'h1.g-headline',
'h1[itemprop="headline"]',
'h1.headline',
],
},
Expand Down
9 changes: 9 additions & 0 deletions src/extractors/custom/www.nytimes.com/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -102,4 +102,13 @@ describe('NYTimesExtractor', () => {
assert.equal(author, 'The New York Times');
assert.equal(text, 'T he Smithsonian’s N');
});

it('returns the title on most recent articles', async () => {
const html = fs.readFileSync('./fixtures/www.nytimes.com/1539194812689.html');
const uri = 'https://www.nytimes.com/2018/10/09/us/politics/nikki-haley-united-nations.html';

const { title } = await Mercury.parse(uri, html);

assert.equal(title, 'Nikki Haley to Resign as Trump’s Ambassador to the U.N.');
});
});

0 comments on commit 5663660

Please sign in to comment.