Skip to content
This repository has been archived by the owner on Apr 5, 2018. It is now read-only.

Commit

Permalink
Merge pull request #2 from rvagg/update-to-latest-core
Browse files Browse the repository at this point in the history 
Updated to node v7.8.0
  • Loading branch information
@mcollina
mcollina authored Apr 6, 2017
2 parents 0cd36da + 53f692c commit 5dd33eb
Show file tree
Hide file tree
Showing 14 changed files with 1,333 additions and 659 deletions.
68 changes: 48 additions & 20 deletions LICENSE
View file
Original file line number Diff line number Diff line change
@@ -1,20 +1,48 @@
Copyright Joyent, Inc. and other Node contributors.

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the
following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
Node.js is licensed for use as follows:

"""
Copyright Node.js contributors. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
"""

This license applies to parts of Node.js originating from the
https://github.com/joyent/node repository:

"""
Copyright Joyent, Inc. and other Node contributors. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
"""

31 changes: 27 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,30 @@
**string_decoder.js** (`require('string_decoder')`) from Node.js core
# string_decoder

Copyright Joyent, Inc. and other Node contributors. See LICENCE file for details.
***Node-core v7.0.0 string_decoder for userland***

Version numbers match the versions found in Node core, e.g. 0.10.24 matches Node 0.10.24, likewise 0.11.10 matches Node 0.11.10. **Prefer the stable version over the unstable.**

The *build/* directory contains a build script that will scrape the source from the [joyent/node](https://github.com/joyent/node) repo given a specific Node version.
[![NPM](https://nodei.co/npm/string_decoder.png?downloads=true&downloadRank=true)](https://nodei.co/npm/string_decoder/)
[![NPM](https://nodei.co/npm-dl/string_decoder.png?&months=6&height=3)](https://nodei.co/npm/string_decoder/)


[![Sauce Test Status](https://saucelabs.com/browser-matrix/string_decoder.svg)](https://saucelabs.com/u/string_decoder)

```bash
npm install --save string_decoder
```

***Node-core string_decoderstring_decoder for userland***

This package is a mirror of the string_decoder implementation in Node-core.

Full documentation may be found on the [Node.js website](https://nodejs.org/dist/v7.8.0/docs/api/).

As of version 1.0.0 **string_decoder** uses semantic versioning.

## Previous versions

Previous version numbers match the versions found in Node core, e.g. 0.10.24 matches Node 0.10.24, likewise 0.11.10 matches Node 0.11.10.

## Update

The *build/* directory contains a build script that will scrape the source from the [nodejs/node](https://github.com/nodejs/node) repo given a specific Node version.
165 changes: 119 additions & 46 deletions build/build.js
View file
Original file line number Diff line number Diff line change
@@ -1,45 +1,89 @@
#!/usr/bin/env node

const hyperquest = require('hyperzip')(require('hyperdirect'))
const hyperquest = require('hyperquest')
, bl = require('bl')
, fs = require('fs')
, path = require('path')
, cheerio = require('cheerio')

, tar = require('tar-fs')
, gunzip = require('gunzip-maybe')
, babel = require('babel-core')
, glob = require('glob')
, pump = require('pump')
, rimraf = require('rimraf')
, encoding = 'utf8'
, urlRegex = /^https?:\/\//
, nodeVersion = process.argv[2]
, nodeVersionRegexString = '\\d+\\.\\d+\\.\\d+'
, usageVersionRegex = RegExp('^' + nodeVersionRegexString + '$')
, readmeVersionRegex =
RegExp('((?:Node-core )|(?:https\:\/\/nodejs\.org\/dist\/)v)' + nodeVersionRegexString, 'g')

, readmePath = path.join(__dirname, '..', 'README.md')
, files = require('./files')
, testReplace = require('./test-replacements')

, srcurlpfx = 'https://raw.github.com/joyent/node/v' + process.argv[2] + '-release/'
, libsrcurl = srcurlpfx + 'lib/'
, testsrcurl = srcurlpfx + 'test/simple/'
, testlisturl = 'https://github.com/joyent/node/tree/v' + process.argv[2] + '-release/test/simple'
, libourroot = path.join(__dirname, '../')
, testourroot = path.join(__dirname, '../test/simple/')
, downloadurl = `https://nodejs.org/dist/v${nodeVersion}/node-v${nodeVersion}.tar.gz`
, src = path.join(__dirname, `node-v${nodeVersion}`)
, libsrcurl = path.join(src, 'lib/')
, testsrcurl = path.join(src, 'test/parallel/')
, libourroot = path.join(__dirname, '../lib/')
, testourroot = path.join(__dirname, '../test/parallel/')


function processFile (url, out, replacements) {
hyperquest(url).pipe(bl(function (err, data) {
if (err)
throw err
if (!usageVersionRegex.test(nodeVersion)) {
console.error('Usage: build.js xx.yy.zz')
return process.exit(1);
}

// `inputLoc`: URL or local path.
function processFile (inputLoc, out, replacements) {
var file = fs.createReadStream(inputLoc, encoding)

file.pipe(bl(function (err, data) {
if (err) throw err

console.log('Processing', inputLoc)
data = data.toString()
replacements.forEach(function (replacement) {
data = data.replace.apply(data, replacement)
const regexp = replacement[0]
var arg2 = replacement[1]
if (typeof arg2 === 'function')
arg2 = arg2.bind(data)
data = data.replace(regexp, arg2)
})

fs.writeFile(out, data, 'utf8', function (err) {
if (err)
throw err
if (inputLoc.slice(-3) === '.js') {
const transformed = babel.transform(data, {
plugins: [
'transform-es2015-parameters',
'transform-es2015-arrow-functions',
'transform-es2015-block-scoping',
'transform-es2015-template-literals',
'transform-es2015-shorthand-properties',
'transform-es2015-for-of',
'transform-es2015-destructuring'
]
})
data = transformed.code
}
fs.writeFile(out, data, encoding, function (err) {
if (err) throw err

console.log('Wrote', out)
})
}))
}

function deleteOldTests(){
const files = fs.readdirSync(path.join(__dirname, '..', 'test', 'parallel'));
for (let file of files) {
let name = path.join(__dirname, '..', 'test', 'parallel', file);
console.log('Removing', name);
fs.unlinkSync(name);
}
}
function processLibFile (file) {
var replacements = files[file]
, url = libsrcurl + file
, out = path.join(libourroot, replacements.out || file)
, out = path.join(libourroot, file)

processFile(url, out, replacements)
}
Expand All @@ -56,39 +100,68 @@ function processTestFile (file) {
processFile(url, out, replacements)
}

//--------------------------------------------------------------------
// Download the release from nodejs.org
console.log(`Downloading ${downloadurl}`)
pump(
hyperquest(downloadurl),
gunzip(),
tar.extract(__dirname),
function (err) {
if (err) {
throw err
}

if (!/0\.1\d\.\d+/.test(process.argv[2])) {
console.log('Usage: build.js <node version>')
return process.exit(-1)
}

//--------------------------------------------------------------------
// Grab & process files in ../lib/

//--------------------------------------------------------------------
// Grab & process files in ../lib/
Object.keys(files).forEach(processLibFile)

Object.keys(files).forEach(processLibFile)

//--------------------------------------------------------------------
// Discover, grab and process all test-string-decoder* files on joyent/node
//--------------------------------------------------------------------
// Discover, grab and process all test-stream* files on the given release

glob(path.join(testsrcurl, 'test-string-decoder*.js'), function (err, list) {
if (err) {
throw err
}

hyperquest(testlisturl).pipe(bl(function (err, data) {
if (err)
throw err
list.forEach(function (file) {
file = path.basename(file)
processTestFile(file)
})
})

var $ = cheerio.load(data.toString())

$('table.files .js-directory-link').each(function () {
var file = $(this).text()
if (/^test-string-decoder/.test(file) || file == 'common.js')
processTestFile(file)
})
}))
//--------------------------------------------------------------------
// Grab the nodejs/node test/common.js

//--------------------------------------------------------------------
// Grab the joyent/node test/common.js
processFile(
testsrcurl.replace(/parallel\/$/, 'common.js')
, path.join(testourroot, '../common.js')
, testReplace['common.js']
)

processFile(
testsrcurl + '../common.js'
, path.join(testourroot, '../common.js')
, testReplace['common.js']
)
//--------------------------------------------------------------------
// Update Node version in README

processFile(readmePath, readmePath, [
[readmeVersionRegex, "$1" + nodeVersion]
])
}
)

// delete the current contents of test/parallel so if node removes any tests
// they are removed here
deleteOldTests();

process.once('beforeExit', function () {
rimraf(src, function (err) {
if (err) {
throw err
}

console.log('Removed', src)
})
})
67 changes: 58 additions & 9 deletions build/files.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,29 +8,78 @@

module.exports['string_decoder.js'] = [

// pull in Bufer as a require
// add Buffer.isEncoding where missing
// we do not need internal/util anymore
[
/^(\/\/ USE OR OTHER DEALINGS IN THE SOFTWARE\.)/m
, '$1\n\nvar Buffer = require(\'buffer\').Buffer;'
+ '\n'
+ '\nvar isBufferEncoding = Buffer.isEncoding'
/const internalUtil = require\('internal\/util'\);/
, ''
]

, [
/const Buffer = require\('buffer'\).Buffer;/
, 'var Buffer = require(\'buffer\').Buffer;\n' +
'var bufferShim = require(\'buffer-shims\');'
]

// allocUnsafe

, [
/Buffer\.((?:alloc)|(?:allocUnsafe)|(?:from))/g,
'bufferShim.$1'
]

// add Buffer.isEncoding where missing
, [
/const isEncoding = Buffer\[internalUtil.kIsEncodingSymbol\];/
, '\nvar isEncoding = Buffer.isEncoding'
+ '\n || function(encoding) {'
+ '\n encoding = \'\' + encoding'
+ '\n switch (encoding && encoding.toLowerCase()) {'
+ '\n case \'hex\': case \'utf8\': case \'utf-8\': case \'ascii\': case \'binary\': case \'base64\': case \'ucs2\': case \'ucs-2\': case \'utf16le\': case \'utf-16le\': case \'raw\': return true;'
+ '\n default: return false;'
+ '\n }'
+ '\n }'
+ '\n'

+ '\nfunction _normalizeEncoding(enc) {'
+ '\n if (!enc) return \'utf8\';'
+ '\n var retried;'
+ '\n while (true) {'
+ '\n switch (enc) {'
+ '\n case \'utf8\':'
+ '\n case \'utf-8\':'
+ '\n return \'utf8\';'
+ '\n case \'ucs2\':'
+ '\n case \'ucs-2\':'
+ '\n case \'utf16le\':'
+ '\n case \'utf-16le\':'
+ '\n return \'utf16le\';'
+ '\n case \'latin1\':'
+ '\n case \'binary\':'
+ '\n return \'latin1\';'
+ '\n case \'base64\':'
+ '\n case \'ascii\':'
+ '\n case \'hex\':'
+ '\n return enc;'
+ '\n default:'
+ '\n if (retried) return; // undefined'
+ '\n enc = (\'\' + enc).toLowerCase();'
+ '\n retried = true;'
+ '\n }'
+ '\n }'
+ '\n };'
]


// use custom Buffer.isEncoding reference
, [
/Buffer\.isEncoding\(/g
, 'isBufferEncoding\('
, 'isEncoding\('
]

]
// use _normalizeEncoding everywhere
, [
/internalUtil\.normalizeEncoding/g
, '_normalizeEncoding'
]

module.exports['string_decoder.js'].out = 'index.js'
]
Loading

0 comments on commit 5dd33eb

Please sign in to comment.