from Hacker News

Tinker with a Neural Network in Your Browser

by shancarter on 4/12/16, 9:57 PM with 116 comments

• by erostrate on 4/13/16, 9:21 AM

  The swiss roll problem also illustrates nicely the idea behind deep learning.

  Before deep learning people would manually design all these extra features sin(x_1), x_1^2, etc. because they thought it was necessary to fit this swiss roll dataset. So they would use a shallow network with all these features like this: http://imgur.com/H1cvt8d

  Then the deep learning guys realized that you don't have to engineer all these extra features, you can just use basic features x_1, x_2 and let the network learn more complicated transformations in subsequent layers. So they would use a deep network with only x_1, x_2 as inputs: http://imgur.com/XBRjROP

  Both these approaches work here (loss < 0.01). The difference is that for the first one you have to manually choose the extra features sin(x_1), x_1^2, ... for each problem. And the more complicated the problem the harder it is to design good features. People in the computer vision community spent years and years trying to design good features for e.g. object recognition. But finally some people realized that deep networks could learn these features themselves. And that's the main idea in deep learning.

• by eggy on 4/13/16, 1:30 AM

  I started reading about ANNs in the 1980s, and had similar confusion to those here, since it was just for fun. I suggest reading a basic book or online information that goes over the basics [1]. I struggled through $200 text books, and jumped from one to the other as an autodidact. I am now studying TWEANNs (Topology and Weight Evolving Artificial Neural Networks), which basically are what you see here with the exception that they are able to not only change their weights, but also their topology, that is how many and where the neurons and layers are. ANNs (Artificial Neural Networks - as opposed to biological ones) can be a lot of fun, and are very relevant to machine learning and big data nowadays. It was exploratory for me. I used them for generative art and music programs. Be careful: soon you'll be reading about genetic algorithms, genetic programming [2], and artificial life ;) Genetic Programming can be used to evolve neural networks as well as generate computer programs to solve a problem in a specified domain. Hint: You'll probably want to use Lisp/Scheme for genetic programming!

    [1] http://natureofcode.com/book/chapter-10-neural-networks/
    [2] http://www.genetic-programming.com

• by nl on 4/13/16, 3:57 AM

  This is great, but I think they should make it clear that this isn't using TensorFlow.

  From the title and domain I though they either had ported TF to Javascript(!) or we connecting to a server.

• by minimaxir on 4/12/16, 10:13 PM

  When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome.

  On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png

• by danielvf on 4/12/16, 10:17 PM

  In case you are an idiot like me, you have to train your neural network by pressing "play".

• by gojomo on 4/12/16, 11:58 PM

  While it doesn't involve training, these 'confusion matrix' animations of NNs classifying images or digits are fun, too:

  http://ml4a.github.io/dev/demos/cifar_confusion.html http://ml4a.github.io/dev/demos/mnist_confusion.html

  Something about the high-speed updating makes me think of WOPR, in 'War Games', scoring nuclear-war scenarios.

• by timroy on 4/13/16, 2:02 AM

  This demonstration goes really well with Michael Nielsen's http://neuralnetworksanddeeplearning.com/. At the bottom of the page the author gives a shout out to Nielsen, Bengio, and others.

  For someone (like me) who's done a bit of reading but not much implementation, this playground is fantastic!

• by CGamesPlay on 4/12/16, 10:52 PM

  Neat stuff, fun to play with. I wasn't able to get a net to classify the swiss roll. Last time I was playing around with this stuff I found the single biggest factor in the success was the optimizer used. Is this just using a simple gradient descent? I would like to see a drop down for different optimizers.

• by _AllisonMobley on 4/12/16, 11:54 PM

  Can somebody explain what I'm watching when I press play?

• by karpathy on 4/13/16, 12:00 AM

  this is very nice! I think that the reason swiss roll doesn't work as easily might be because of initialization. In 2 dimensions you have to be very careful with initializing the weights or biases because small networks get more easily stuck in bad local minima.

• by danblick on 4/13/16, 5:59 AM

  Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?

• by halotrope on 4/13/16, 9:25 AM

  You could totally optimise network architecture by crowdsourcing topology discovery for different problems into a multiplayer game with loss as a score.

• by Your_Creator on 4/13/16, 5:17 PM

  So glad anns are becoming mainstream

  Eventually it will have to be recognized as a new species of life, so I hope programmers, tinkerers and everyone else keeps that in mind because all life must be respected

  And this particular form will be our responsibility, we can either embrace it as we continue to merge with our technology, or we can allow ourselves to go extinct like so many other species already have

  For the naysayers - ever notice how attached we are to our phones? Many behave as if they are missing a limb without it - it's because they are, the brain adapts rapidly and for many, the brain has adapted to outsourcing our cognition. It used to be books, day runners, journals, diaries - now we have devices and soon they'll be implants or prosthetics

  The writers at marvel who came up with the idea of calling iron man's suit a prosthetic were definately onto something and suits like that are probably our best chance of successful colonization of other planets. We'll need ai to be our friend out there, working with us

• by aab0 on 4/12/16, 10:41 PM

  This is a lot of fun. The default dataset is too easy, though, try out the Swiss Roll one!

• by sparky_ on 4/13/16, 4:48 AM

  This is a very cool toy. As someone with no experience in ML, this is an interesting visual approach to the absolute basics.

  And great for challenging your friends in an epic battle of convergence!

• by pkaye on 4/13/16, 2:40 AM

  I'm not well versed in neural networks but a lot of the new neural network software stacks coming out seem to be quite plug and plug. What kind of expertise would engineers need to have a few years from now when the technology is well developed and it doesn't need to be rewritten from scratch every time?

• by plafl on 4/13/16, 9:20 AM

  Beautiful. The next time someone asks what is machine learning about I'm going to send a link to this page.

• by nxzero on 4/13/16, 1:03 AM

  "Don’t Worry, You Can’t Break It. We Promise."

  (Nice, but it's completely unclear what's going on.)

• by nkozyra on 4/13/16, 3:45 AM

  Is a 50/50 training:test a normal default ratio for an ANN? I expected to see a higher amount of training data represented as the initial setting.

• by hyh1048576 on 4/13/16, 4:19 AM

  One of the finest data visualization I've seen.

• by icelancer on 4/13/16, 5:01 AM

  This is so great. An easy way to show my friends WTF I do sometimes for math/CS work. Thank you so much.

• by imaginenore on 4/13/16, 1:20 AM

  I wish they had more interesting data sets.