4

The article here: http://novanoid.github.io/2014/09/26/training-a-neural-network-to-recognize-handwritten-digits/ discusses and implements a way to recognize handwritten digits. For images with a quality of 256 square pixels and an output vector of magnitude 10, the network is constructed quickly and efficiently on a modern machine.

However, for Chinese characters, the output vector has a size over 50,000, although we typically only need to be able to write between 3000 and 8000 characters.

Training a single neural network using the approach from the article seems difficult on machines today. I attempted a neural network that would recognize only 3000 characters and quickly ran into memory limitations.

Is it possible to create multiple smaller networks to recognize sets of characters and then have a mechanism for running all networks simultaneously to parse handwritten characters in real time? What would this approach look like? Or is there a better/different approach to recognize handwritten Chinese characters?

Franck Dernoncourt
  • 42,093
  • 30
  • 155
  • 271
waylonion
  • 141
  • 2
  • If you are running all networks simultaneously, wouldn't you have the same memory issue? – Franck Dernoncourt Jan 10 '17 at 16:56
  • What programming language have you used to develop the network? Once, I had to translate my algorithm from Python to C for speed. – Milos Jan 10 '17 at 16:57
  • 6
    Possible duplicate of [Deep Learning with many categories](http://stats.stackexchange.com/questions/253863/deep-learning-with-many-categories) – Franck Dernoncourt Jan 10 '17 at 17:19
  • I am currently working with Java as I intended to develop a mobile interface for testing and ease of use. – waylonion Jan 10 '17 at 17:27
  • @FranckDernoncourt, Perhaps my intuition was wrong, but I was under the impression that a group of smaller networks mapping to non overlapping subsets of a given set of characters would be smaller than a single network mapping to the entire given set. – waylonion Jan 10 '17 at 17:29
  • 1
    I do not speak or read Chinese but I am under the impression that Chinese characters share many common subelements. Perhaps you could start with a more limited goal? – jwimberley Jan 10 '17 at 18:22
  • 1
    @jwimberley, Right, Chinese characters do seem to share common subelements. However, how would a neural network recognize this and then act upon it? How do we overcome the memory limitations in regards to a single neural network? – waylonion Jan 11 '17 at 00:17
  • 1
    @WayWay Well, a neural network can have many layers that respond to features on different scales, so even within a single neural network common subelements could be recognized at one of the top layers. However, I mean something simpler: start with a simpler goal. First get a neural net just identifying one of these subelements working and try to make it as concise as possible. Do the same for other subelements (with as few modifications as possible). Later you can combine these into a bigger network -- or use some other technique for synthesizing their outputs. Don't try to do it all at once. – jwimberley Jan 11 '17 at 00:26
  • 1
    @WayWay And by "some other technique" don't restrict yourself to machine techniques. Maybe investigate how uniquely the 50,000 Chinese characters can be identified by their subelements, just by having a team of people go through them (this has likely been categorized somewhere if its possible). If separate neural networks can identity the subelements and the subelements can identify the character through hard-coded, human created rules, this solves your problem equally well. – jwimberley Jan 11 '17 at 02:23
  • 1
    I recognize that the OP shares the same problem as the proposed duplicate, but his question seems to me much more narrow. Whether and how to train many small networks to replace a larger strikes me as distinct from the broad question of how to handle learning many categories. Am I missing something? – Sean Easter Jan 11 '17 at 15:29

0 Answers0