retext is an extensible natural language processor with support for multiple languages. Retext provides a pluggable system for analysing and manipulating natural language in JavaScript. Node and the browser. 100% coverage.

Rather than being a do-all library for Natural Language Processing (such as NLTK or OpenNLP), retext aims to be useful for more practical use cases (such as checking for insensitive words or decoding emoticons) instead of more academic goals (research purposes). retext is inherently modular—it uses plugins (similar to mdast for markdown) instead of providing everything out of the box (such as Natural). This makes retext a viable tool for use on the web.

Installation

npm:

npm install retext

retext is also available for bower, and duo, and as an AMD, CommonJS, and globals module, uncompressed and compressed.

Usage

The following example uses retext-emoji to show emoji and retext-smartypants for smart punctuation.

Require dependencies:

var retext = require('retext');
var emoji = require('retext-emoji');
var smartypants = require('retext-smartypants');

Create an instance using retext-emoji and -smartypants:

var processor = retext().use(smartypants).use(emoji, {
    'convert' : 'encode'
});

Process a document:

var doc = processor.process([
    'The three wise monkeys [. . .] sometimes called the three mystic',
    'apes--are a pictorial maxim. Together they embody the proverbial',
    'principle to ("see no evil, hear no evil, speak no evil"). The',
    'three monkeys are Mizaru (:see_no_evil:), covering his eyes, who',
    'sees no evil; Kikazaru (:hear_no_evil:), covering his ears, who',
    'hears no evil; and Iwazaru (:speak_no_evil:), covering his mouth,',
    'who speaks no evil.'
].join('\n'));

Yields (you need a browser which supports emoji to see them):

The three wise monkeys […] sometimes called the three mystic
apes—are a pictorial maxim. Together they embody the proverbial
principle to (“see no evil, hear no evil, speak no evil”). The
three monkeys are Mizaru (🙈), covering his eyes, who
sees no evil; Kikazaru (🙉), covering his ears, who
hears no evil; and Iwazaru (🙊), covering his mouth,
who speaks no evil.

API

retext.use(plugin[, options])

Change the way retext works by using a plugin.

Signatures

• processor = retext.use(plugin, options?);
• processor = retext.use(plugins).

Parameters

• plugin (Function) — A Plugin;

• plugins (Array.<Function>) — A list of Plugins;

• options (Object?) — Passed to the plugin. Specified by its documentation.

Returns

Object — an instance of Retext: The returned object functions just like retext (it has the same methods), but caches the used plugins. This provides the ability to chain use calls to use multiple plugins, but ensures the functioning of the retext module does not change for other dependents.

retext.process(value[, done])

Parse a text document, apply plugins to it, and compile it into something else.

Signatures

• doc = mdast.process(value[, done]).

Parameters

• value (VFile or string) — Text document;

• done (Function, optional).

Returns

string?: A document. Formatted in whatever plugins generate. The result is null if a plugin is asynchronous, in which case the callback done should’ve been passed (don’t worry: plugin creators make sure you know its async).

function done(err, file, doc)

Callback invoked when the output is generated with either an error, or the processed document (represented as a virtual file and a string).

Parameters

• err (Error?) — Reason of failure;
• file (VFile?) — Virtual file;
• doc (string?) — Generated document.

Plugin

function attacher(retext[, options])

A plugin is a function, which takes the Retext instance a user attached the plugin on as a first parameter and optional configuration as a second parameter.

A plugin can return a transformer.

function transformer(node, file[, next])

A transformer changes the provided document (represented as a node and a virtual file).

Transformers can be asynchronous, in which case next must be invoked (optionally with an error) when done.

List of Plugins

• retext-directionality — (demo) — Detect the direction text is written in;

• retext-dom — (demo) — Create a (living) DOM tree from a TextOM tree;

• retext-double-metaphone — (demo) — Implementation of the Double Metaphone algorithm;

• retext-dutch — Dutch language support;

• retext-english — English language support;

• retext-emoji — (demo) — Encode or decode Gemojis;

• retext-equality — Warn about possible insensitive, inconsiderate language;

• retext-keywords — (demo) — Extract keywords and keyphrases;

• retext-lancaster-stemmer — (demo) — Implementation of the Lancaster (Paice/Husk) algorithm;

• retext-language — (demo) — Detect the language of text;

• retext-metaphone — (demo) — Implementation of the Metaphone algorithm;

• retext-porter-stemmer — (demo) — Implementation of the Porter stemming algorithm;

• retext-pos — (demo) — Part-of-speech tagger;

• retext-sentiment — (demo) — Detect sentiment in text;

• retext-smartypants — (demo) — Implementation of SmartyPants;

• retext-soundex — (demo) — Implementation of the Soundex algorithm;

• retext-syllable — (demo) — Syllable count;

List of Utilities

The following projects are useful when working with the syntax tree, NLCST:

• wooorm/nlcst-to-string — Stringify a node;

• wooorm/nlcst-is-literal — Check whether a node is meant literally;

• wooorm/nlcst-test — Validate a NLCST node;

In addition, see wooorm/unist for other utilities which work with retext nodes, but also with mdast nodes.

And finally, see wooorm/vfile for a list of utilities for working with virtual files.

Related

• nlcst
• unist
• unified

License

MIT © Titus Wormer