Question: trainer network error
Hi,
Thanks a ton for the fantastic library! I'm using a deep network for NLP, with a varying input size of 12000 down to 4000 nodes. I've made sure to enable 8GB for RAM for node:
node --max-old-space-size=8192
My network looks like this (although I'd like to try different types and architectures):
var layers = [];
layers.push({type: 'input', out_sx: 1, out_sy: 1, out_depth: input_size});
layers.push({type: 'fc', num_neurons: 200, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 100, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 50, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 25, activation: 'relu'});
layers.push({type: 'fc', num_neurons: 10, activation: 'relu'});
layers.push({type: 'softmax', num_classes: 2});
var net = new convet.Net();
net.makeLayers(layers);My data has been parsed in a json array of objects, which I then randomly shuffle and partition into a training set and testing set.
Each json object has a vector (which is simply an array of floats), and a score which is a single value.
My training loop is basically the following:
var trainer = new convnet.Trainer(network, {learning_rate: 0.1, l2_decay: 001});
var epochs = 1000;
for (var i = 0; i < epochs; i++)
{
for (var index in dataset.training())
{
var input = new convnet.Vol(json[index].vector);
var output = new convnert.Vol(json[index].score);
trainer.train(input, output);
}
}It runs, but I have no way of validating it.
Is there a Mean Square Error or Average Cross Entropy or any other network-error measurement? AFAIK, the only way to test the network's accuracy is to cross-validate using my testing samples, and see (a) if they are classified correctly, or (b) how far the actual output is from my target/ideal output.
I took a peek into the convent.js source file but I don't see Trainer.train to be returning any type of network error (unless I missed something - very possible!).
Last but not least, referencing your library, do you have a citation you'd like me to use?
PS: is there a way to save a trained network?
Best regards, Alex
Source: karpathy/convnetjs