gstore-node

Entities modeling for Google's Datastore

🎉 NEWS: new maintainer!

A few weeks ago I announced that gstore-node was deprecated as I currently don't have bandwidth to work on it. A few days later, Hendrik Schalekamp stepped in and offered to be a maintainer of the project. I was thrilled! 😊 Hendrik will be the lead of the project and I will be around to provide any necessary guidance. Thanks Hendrik!

Documentation | Example | Demo Application | Support | Changelog

gstore-node is a Google Datastore entities modeling library for Node.js inspired by Mongoose and built on top of the @google-cloud/datastore client.
It is not a replacement of @google-cloud/datastore but a layer on top of it to help modeling your entities through Schemas and to help validating the data saved in the Datastore.

Highlight

• explicit Schema declaration for entities
• properties type validation
• properties value validation
• shortcuts queries
• pre & post middleware (hooks)
• custom methods on entity instances
• Joi schema definition and validation
• Advanced cache layer
• Typescript support
• populate() support to fetch reference entities and do cross Entity Type "joins" when querying one or multiple entities (since v5.0.0)

Please don’t forget to star this repo if you find it useful :)

Installation

npm install gstore-node --save
# or
yarn add gstore-node

Important: gstore-node requires Node version 8+

Getting started

Import gstore-node and @google-cloud/datastore and configure your project.
For the information on how to configure @google-cloud/datastore read the docs here.

const { Gstore } = require('gstore-node');
const { Datastore } = require('@google-cloud/datastore');

const gstore = new Gstore();
const datastore = new Datastore({
    projectId: 'my-google-project-id',
});

// Then connect gstore to the datastore instance
gstore.connect(datastore);

After connecting gstore to the datastore, gstore has 2 aliases set up

• gstore.ds
  The @google/datastore instance. This means that you can access all the API of the Google library when needed.

• gstore.transaction. Alias of the same google-cloud/datastore method

Documentation

The complete documentation of gstore-node is in gitbook.
If you find any mistake in the docs or would like to improve it, feel free to open a PR.

Example

Initialize gstore-node in your server file

// server.js

const { Gstore, instances } = require('gstore-node');
const { Datastore } = require('@google-cloud/datastore');

const gstore = new Gstore();
const datastore = new Datastore({
    projectId: 'my-google-project-id',
});

gstore.connect(datastore);

// Save the gstore instance
instances.set('unique-id', gstore);

Create your Model

// user.model.js

const { instances } = require('gstore-node');
const bscrypt = require('bcrypt-nodejs');

// Retrieve the gstore instance
const gstore = instances.get('unique-id');
const { Schema } = gstore;

/**
 * A custom validation function for an embedded entity
 */
const validateAccessList = (value, validator) => {
    if (!Array.isArray(value)) {
        return false;
    }

    return value.some((item) => {
        const isValidIp = !validator.isEmpty(item.ip) && validator.isIP(item.ip, 4);
        const isValidHostname = !validator.isEmpty(item.hostname);

        return isValidHostname && isValidIp;
    });
}

/**
 * Create the schema for the User Model
*/
const userSchema = new Schema({
    firstname: { type: String, required: true },
    lastname: { type: String, optional: true  },
    email: { type: String, validate: 'isEmail', required: true },
    password: { type: String, read: false, required: true },
    createdOn: { type: String, default: gstore.defaultValues.NOW, write: false, read: false },
    address: { type: Schema.Types.Key, ref: 'Address' }, // Entity reference
    dateOfBirth: { type: Date },
    bio: { type: String, excludeFromIndexes: true },
    website: { validate: 'isURL', optional: true },
    ip: {
        validate: {
            rule: 'isIP',
            args: [4],
        }
    },
    accessList: {
        validate: {
            rule: validateAccessList,
        }
    },
});

// Or with **Joi** schema definition
// You need to have joi as a dependency of your project ("npm install joi --save")
const userSchema = new Schema({
    firstname: { joi: Joi.string().required() },
    email: { joi: Joi.string().email() },
    password: { joi: Joi.string() },
    ...
}, {
    joi: {
        extra: {
            // validates that when "email" is present, "password" must be too
            when: ['email', 'password'],
        },
    }
});

/**
 * List entities query shortcut
 */
const listSettings = {
    limit: 15,
    order: { property: 'lastname' }
};
userSchema.queries('list', listSettings);

/**
 * Pre "save" middleware
 * Each time the entity is saved or updated, if there is a password passed, it will be hashed
*/
function hashPassword() {
    // scope *this* is the entity instance
    const _this = this;
    const password = this.password;

    if (!password) {
        return Promise.resolve();
    }

    return new Promise((resolve, reject) => {
        bcrypt.genSalt(5, function onSalt(err, salt) {
            if (err) {
                return reject(err);
            };

            bcrypt.hash(password, salt, null, function onHash(err, hash) {
                if (err) {
                    // reject will *not* save the entity
                    return reject(err);
                };

                _this.password = hash;

                // resolve to go to next middleware or save method
                return resolve();
            });
        });
    });
}

// add the "pre" middleware to the save method
userSchema.pre('save', hashPassword);

/**
 * Export the User Model
 * It will generate "User" entity kind in the Datastore
*/
module.exports = gstore.model('User', userSchema);

Use it in your Controller

// user.constroller.js

const User = require('./user.model');

const getUsers = (req ,res) => {
    const pageCursor = req.query.cursor;

    // List users with the Query settings defined on Schema
    User.list({ start: pageCursor })
        .then((entities) => {
            res.json(entities);
        })
        .catch(err => res.status(400).json(err));
};

const getUser = (req, res) => {
    const userId = +req.params.id;
    User.get(userId)
        .populate('address') // Retrieve the reference entity
        .then((entity) => {
            res.json(entity.plain());
        })
        .catch(err => res.status(400).json(err));
};

const createUser = (req, res) => {
    const entityData = User.sanitize(req.body);
    const user = new User(entityData);

    user.save()
        .then((entity) => {
            res.json(entity.plain());
        })
        .catch((err) => {
            // If there are any validation error on the schema
            // they will be in this error object
            res.status(400).json(err);
        })
};

const updateUser = (req, res) => {
    const userId = +req.params.id;
    const entityData = User.sanitize(req.body); // { email: '[email protected]' }

    /**
     * This will fetch the entity, merge the data and save it back to the Datastore
    */
    User.update(userId, entityData)
        .then((entity) => {
            res.json(entity.plain());
        })
        .catch((err) => {
            // If there are any validation error on the schema
            // they will be in this error object
            res.status(400).json(err);
        });
};

const deleteUser = (req, res) => {
    const userId = +req.params.id;
    User.delete(userId)
        .then((response) => {
            res.json(response);
        })
        .catch(err => res.status(400).json(err));
};

module.exports = {
    getUsers,
    getUser,
    createUser,
    updateUser,
    deleteUser
};

Demo application

If you want to see an example on how to use gstore-node in your Node.js app, check the demo blog application repository.

Development

1. Install npm dependencies

yarn install

2. Run the unit tests

yarn test:unit

3. Run the integration tests

Prerequise:
In order to run the integration tests you need to have the Google Datastore Emulator installed as well as Redis.

• Launch the Redis server

# From the folder where you've installed the Redis SDK run: 
cd src && ./redis-server

• Launch the Datastore Emulator (separate terminal window)

# From the root of the project
yarn local-datastore

• Execute the integration tests (separate terminal window)

# From the root of the project
yarn test:integration

Meta

Sébastien Loix – @sebloix
Hendrik Schalekamp - @carnun

Distributed under the MIT license. See LICENSE for more information.

Contributing

1. Fork it (https://github.com/sebelga/gstore-node/fork)
2. Create your feature branch (git checkout -b feature/fooBar)
3. Generate a conventional commit message (npm run commit)
4. Push to the branch (git push origin feature/fooBar)
5. Rebase your feature branch and squash (git rebase -i master)
6. Create a new Pull Request

Credits

I have been heavily inspired by Mongoose to write gstore. Credits to them for the Schema, Model and Entity definitions, as well as 'hooks', custom methods and other similarities found here. Not much could neither have been done without the great work of the guys at googleapis.