Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neataptic LSTM bool sequence tunning #147

Open
kotopuz opened this issue Dec 21, 2018 · 0 comments
Open

neataptic LSTM bool sequence tunning #147

kotopuz opened this issue Dec 21, 2018 · 0 comments

Comments

@kotopuz
Copy link

kotopuz commented Dec 21, 2018

Dear all,
Looked at examples of this interesting library, and it looks great with simple examples provided. After I've added a little complexity somehow it fails to work a expected.
Info used
https://wagenaartje.github.io/neataptic/docs/builtins/lstm/
https://stackoverflow.com/questions/43574799/dead-simple-example-of-synaptic-js-lstm-rnn-algorithm/43734612

Live working example
https://jsfiddle.net/9t2787k5/

In live example I did not get why this for is needed
for(var i in trainingData){
var input = trainingData[i].input;
var output = Math.round(network.activate([input]));
$('html').append('

Input: ' + input[0] + ', output: ' + output + '

');
}

Why not just to take latest object in training.input ?
like:
input = trainingData[trainingData.length - 1].output

I've deleted all HTML stuff to have pure javascript to run it in any browser console. I use use input to enter any sequence I need.
Then after every new entry I have prediction of next value. I've tried to console log some stuff to see which one is correct. As I do not understand why
var input = output; is present in code I've made console log before and after.
Tried to tune LSTM neurons number LSTM(1,8,1); then LSTM(1,50,1);
Still LSTM not working good at all.
I wanted to have few patterns in sequence like 0001 and then 1110 and 010101
Somehow LSTM works really bad even with hyge learning time and amount of neurons(over 40 mins traning).
Data sample 0001 0001 0001 (prediction works fine) 1110 1110 1110 (sometime it's Ok, not stable) then 01010101 11 (should predict 1 here, but it's always different) then 01010(should predict 1 but it's not working)

Full sample (I separate by 4 digits to easier reading)
0001 0001 0001 1110 1110 1110 01010 1110 0001 010101

var network = new neataptic.Architect.LSTM(1,8,1);
	var trainingData = [];
	var arr_input = [];  // sequnec is x,X+1 should be. x> input x and X+1 to output x+1 and x+1 to input x+1 then predict output x+1; it means output y = input y+1
    var arr_output = [];
	arr_input.push(prompt("Enter a input number" + " " + i));
    for(var i = 0; i < 5; i++){
     arr_output.push(prompt("Enter an output number" + " " + i));
     arr_input.push(arr_output[arr_output.length - 1]); // next input is last output value in array
	}
    arr_output.push(prompt("Enter last output number"));
	console.log("input array: " + arr_input.join(', '));// alert the results
	console.log("output array: " + arr_output.join(', '));// alert the results

    // create initial training data
    for(var iii in arr_input){
	 trainingData.push({
	 input: [arr_input[iii]],
	 output: [arr_output[iii]]
	 });
	 }
// JAVASCRIPT loop
MYLOOP: do {

    arr_input.push(arr_output[arr_output.length - 1]); // this ouput (last array element) should be equal to next input
    arr_output.push(prompt("Emulate next time series value new value 1 or 0")); // next time series value  emulation from user input
    console.log("new time series:   " + arr_output[arr_output.length - 1]);

    trainingData.push({
	 input: [arr_input[arr_input.length - 1]],
	 output: [arr_output[arr_output.length - 1]]
	 });
	//console.table(trainingData);

	network.train(trainingData, {
	  log: 1000,
	  iterations: 6000,
	  error: 0.03,
	  clear: true,
	  rate: 0.05,
	});

	for(var iiii in trainingData){
	  var input2 = trainingData[iiii].input;
      var output2 = trainingData[iiii].output;
	  //var output = Math.round(network.activate([input]));
	  console.log("Num: " + iiii + "  Input: " + input2 + ", output: " + output2 + "!!!");
	}
    ///// neznau zachem utut loop
    for(iiii in trainingData){
	  var input = trainingData[iiii].input;
      var output = Math.round(network.activate([input]));
	  //console.log("Num: " + iiii + "  Input: " + input + ", output: " + output + "!!!");
	}
	console.log("Next prediction input before: " + Math.round(network.activate([input])) + "!!!");
    alert("Next prediction input before: " + Math.round(network.activate([input])) + "  !!!");
    console.log("Next prediction output before: " + Math.round(network.activate([output])) + "!!!");
    input = output;
    //input = trainingData[trainingData.length - 1].output
    //alert(input);
    console.log("Next prediction input after: " + Math.round(network.activate([input])) + "!!!");
    console.log("Next prediction output after: " + Math.round(network.activate([output])) + "!!!");

  //if (prompt("1 - exit, 0 -continue")==1) break MYLOOP;
  continue MYLOOP;// Not necessary since you can just put do {} while (1) but it     illustrates
} while (1)

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant