-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi Hidden Layer Support for nn.js #61
base: master
Are you sure you want to change the base?
Conversation
This is really impressive, thank you! I think I'd like to implement this as a tutorial. . . it's similar to how I was thinking about encapsulating the "Layer" object. I'm trying to decide if I should merge now or wait until I cover it in a video? |
Thanks, it's my first GitHub journey, normally i don't release anything because i end up writing hacky command line applications for myself. Updated it to the current branch, well you still got the 10.18 version somewhere here, i really would like to see your approach on that. |
This is similar to how I have done it in the past. I think each layer should keep its work values both before and after activation. This would allow two things: a wider variety of activation functions per #70, and code reuse from the forward pass when training. Duplicating the same code in both the Also, if you calculate all of the gradients first the weights can potentially be updated in parallel. I think the advantages outweigh the increased memory use. |
@Versatilus yeah i know but because these changes are not implemented yet i'll wait for the master branch to update before i push any changes of that kind.. but i like it, also an other problem is that the neuralnetwork is to tied to the supervised learning thing, i would like to refactor the training and predict things aswell |
This is really nice, I've been playing a bit and have to say works good. Now that we have multi hidden layer, any advice on how to implement dropout and LSTM ? |
Tested on mnist and xor with default settings and multi hidden layers.
Examples doesn't need to be rewriten.
Does the same:
NeuralNetwork(2, 4, 1)
NeuralNetwork(2, [4], 1)
For more layers adding nodes to the hidden_nodes array
thx @xxMrPHDxx for helping to fix the serialize <3