Question: trainer network error

Author: alexge233Created Mar 2, 2016Updated Dec 1, 2020

Hi,

Thanks a ton for the fantastic library! I'm using a deep network for NLP, with a varying input size of 12000 down to 4000 nodes. I've made sure to enable 8GB for RAM for node:

node --max-old-space-size=8192

My network looks like this (although I'd like to try different types and architectures):

var layers = [];
layers.push({type: 'input', out_sx: 1, out_sy: 1, out_depth: input_size});
layers.push({type: 'fc', num_neurons: 200, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 100, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 50, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 25, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 10, activation: 'relu'});
layers.push({type: 'softmax', num_classes: 2});

var net = new convet.Net();
net.makeLayers(layers);

My data has been parsed in a json array of objects, which I then randomly shuffle and partition into a training set and testing set.

Each json object has a vector (which is simply an array of floats), and a score which is a single value.

My training loop is basically the following:

var trainer = new convnet.Trainer(network, {learning_rate: 0.1, l2_decay: 001});
var epochs = 1000;
for (var i = 0; i < epochs; i++)
{
    for (var index in dataset.training())
    {
        var input = new convnet.Vol(json[index].vector);
        var output = new convnert.Vol(json[index].score);
        trainer.train(input, output);
    }
}

It runs, but I have no way of validating it. Is there a Mean Square Error or Average Cross Entropy or any other network-error measurement? AFAIK, the only way to test the network's accuracy is to cross-validate using my testing samples, and see (a) if they are classified correctly, or (b) how far the actual output is from my target/ideal output.

I took a peek into the convent.js source file but I don't see Trainer.train to be returning any type of network error (unless I missed something - very possible!).

Last but not least, referencing your library, do you have a citation you'd like me to use?

PS: is there a way to save a trained network?

Best regards, Alex