Skip to content

Tensorflow + Keras machine learning inside a PostgreSQL database using PL/Python

Notifications You must be signed in to change notification settings

TheTedLab/PostgresML

Repository files navigation

postgresml_logo

Table of contents

Introduction

This project uses Tensorflow + Keras machine learning inside a PostgreSQL database using PL/Python functions and procedures, specifically the plpython3u extension.

The key feature of the project is to run the neural network training, while the entire dataset for training and testing is located inside the PostgreSQL database tables. All the resulting models are also saved in a separate database table as a json file.

The project presents three different uses for neural networks: a MNIST dataset of handwritten digits, a Cats Vs Dogs dataset for classifying cats and dogs, and a wheat disease recognition neural network from digital images with noise based on Haralick texture features.

Datasets

Haralick features

To create digital images, the original image was divided into 6 color components: R, G, B, RB, RG, GB and 4 Haralic parameters were counted for each. Obtained matrices of 6x4 dimension act as digital images of original images.

digital-image-calc

Gray Level Co-occurence Matrix (GLCM)

Haralick's features reflect the texture of the image by counting the Gray Level Co-occurence Matrix (GLCM). Digital images based on such parameters help to extend the dataset, have compact data, and use simpler neural network configurations.

glcm_matrix_creation

Haralick's features formulas

haralick_params

View of the digital image

sample-test

Network Config

The neural network is created, trained, and saved to the database using the functions from mldb_api.sql. The neural network consists of several convolutional and max-pooling layers and is completed with a flatten sweep and several dense layers with dropout, at the end 8 wheat disease classes are allocated.

neural_network_config_FFNN

MNIST dataset

MNIST dataset of handwritten images of digits was taken as an object of neural network training and tested on the created samples.

Cats VS Dogs

The Cats-vs-Dogs dataset was taken as the training object of the neural network and tested on the created samples.

Results

Haralick features

results confusion_matrixes results-tests

MNIST

mnist-results

Cats VS Dogs

final-train-and-val-acc 9

About

Tensorflow + Keras machine learning inside a PostgreSQL database using PL/Python

Topics

Resources

Stars

Watchers

Forks