Peeking into the Neural Network Black Box

If you have ever come in touch with neural networks, you are probably familiar with the black box problem [1, 2]. Compared to many other algorithms from the glass box category [2], neural networks are inherently difficult to dissect. This should come as no surprise. We use neural networks to find solutions to problems which are difficult for humans to put into the language of algorithms. Whenever it is difficult for an expert to find features which would help any other Machine Learning algorithm, that’s typically where neural networks come into the picture and blow the competition out of the water.

So what can be done? As it turns out, the inherent difficulty of explaining how neural networks work does not deter everyone and there are actually people who, I would say, are even drawn to the difficulty [1]. I still think the proposed methods are far from explaining how a neural network works on the entire dataset, on all classes etc. Right now, in my opinion, the explanation methods are most powerful in a sample-by-sample examination. They are, for example, able to tell you which parts of the image play the most important role in an image classification task. I would still argue that there are important conclusions to be drawn about the task as a whole.

Even though the methods are very general, in the end, I will of course focus on models we use at Rossum for invoices. If you last until the end of this blog post, I promise you will see some nice results regarding invoices and tips on how to use those in your own projects. Continue reading


Does Adding One Neuron Help Real World Networks?

Our researcher, Tonda Hoskovec, has long been thinking about the behavior of neural network training in the case of non-convex tasks with many local minima. This makes training difficult or inconsistent for many machine learning problems. Also the theory is lagging behind in practice and not much is guaranteed. A recent theoretical paper aims at solving this problem in a new ingenious way and caught Tonda’s attention. He decided to do the first experimental test of this theory. Is it practical, and does it work? Read on to learn the outcome!

Continue reading