Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [theano]

Python library for parallel GPU-based computations with multidimensional arrays. Often used to implement deep neural networks.

44 questions
26
votes
6 answers

Deep learning : How do I know which variables are important?

In terms of neural network lingo (y = Weight * x + bias) how would I know which variables are more important than others? I have a neural network with 10 inputs, 1 hidden layer with 20 nodes, and 1 output layer which has 1 node. I'm not sure how to…
user1367204
  • 831
  • 3
  • 10
  • 18
19
votes
2 answers

Keras: why does loss decrease while val_loss increase?

I setup a grid search for a bunch of params. I am trying to find the best parameters for a Keras neural net that does binary classification. The output is either a 1 or a 0. There are about 200 features. When I did a grid search, I got a bunch of…
user1367204
  • 831
  • 3
  • 10
  • 18
13
votes
2 answers

Neural Nets: One-hot variable overwhelming continuous?

I have raw data that has about 20 columns (20 features). Ten of them are continuous data and 10 of them are categorical. Some of the categorical data can have like 50 different values (U.S. States). After I pre-process the data the 10 continuous…
user1367204
  • 831
  • 3
  • 10
  • 18
8
votes
1 answer

How do I implement a deep autoencoder?

I'm trying to replicate results of this paper using Theano. The problem at the moment is, all Theano-related tutorials are only for MNIST classifiers, which isn't much use in unsupervised image retrieval. I have following idea how to approach the…
jnovacho
  • 181
  • 1
  • 3
5
votes
2 answers

Deep Learning: Wild differences after model is retrained on the same data, what to do?

I am using keras to train a 5 layer regression model to predict 1000 different thermometers. I train a model and then ask it to predict what the reading will be based on 20 other instruments. I wanted to see if I am doing things correctly, so I…
user1367204
  • 831
  • 3
  • 10
  • 18
5
votes
3 answers

Why do Deep Learning libraries force the cost function to output a scalar?

Let's say we have a neural net with: 5 input neurons some arbitrary amount of hidden layers 3 output neurons Let's say we're using minibatches of size 32. So, if we input a 5x32 matrix into the neural net, we will then get out a 3x32 matrix of…
Bill
  • 313
  • 2
  • 10
5
votes
1 answer

If we have auto differentiate tool, do we also need EM algorithm?

I my opinion, EM algorithm is used to estimate the parameters of some complex log likelihood function. Because sometimes, it's hard to get the derivative, we can use EM algorithm. But if we have some auto differentiate tools such as theano, we can…
maple
  • 279
  • 3
  • 8
5
votes
1 answer

What is the purpose of the scaling factor used in dropout?

I have a question related to the dropout function in the LSTM tutorial: http://deeplearning.net/tutorial/code/lstm.py def dropout_layer(state_before, use_noise, trng): proj = tensor.switch(use_noise, (state_before * …
user5016984
  • 121
  • 5
4
votes
1 answer

Keras Theano conv/deconv autoencoder dimension mis-match

I'm trying to build an autoencoder in keras based on examples in this blog post but with different layers. The model compiles but throws an error (Theano) ValueError: Input dimension mis-match. (input[0].shape[2] = 1, input[6].shape[2] = 28) ...…
user390458
  • 43
  • 3
4
votes
2 answers

How are convolutional layers connected in Theano?

How are feature maps connected between two layers in Theano/Caffe/TensorFlow? For instance, if we have 32 feature maps in Conv Layer 1, and 64 feature maps in Conv Layer 2, with 64 kernels, how does the implementation connect the two layers? Is it…
user135237
  • 271
  • 1
  • 3
4
votes
0 answers

Nesterov vs. momentum gradient descent

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that nesterov is an improvement over momentum. My question…
thc
  • 388
  • 2
  • 16
4
votes
2 answers

Robust softmax solutions for Theano?

I am implementing multilayer perceptrons with the softmax activation function over Theano. In some extreme cases I am running into problems with too high/low values in the softmax function that originate some distributions that are in some places…
gsmafra
  • 257
  • 2
  • 10
3
votes
0 answers

Recurrent Neural Network on Panel Data

There are 2 parts to this question. Suppose we are looking at sales $S$ of a product across $> 1000$ stores where a it sells. For each of these $1000$ stores we have 24 months recorded data. We want to be able to predict $S_t \leftarrow…
Sandipan Bhattacharyya
  • 71
  • 1
  • 8
3
votes
1 answer

How can I re-code this hierarchical model in PyMC 3?

I wish to model data from an experiment using a hierarchical Bayesian logistic regression. The experiment involved many subjects, and many trials collected from each subject. The DV is the outcome of each trial, coded as 0s and 1s. I have numerous…
sammosummo
  • 743
  • 3
  • 13
3
votes
3 answers

Recursive neural network implementation in Theano

Is there any available recursive neural network implementation in Theano? Theano's deeplearning.net tutorial does not present any recursive neural networks. Most Theano code I've found is CNN, LSTM, GRU, vanilla recurrent neural networks or MLP. I…
Franck Dernoncourt
  • 42,093
  • 30
  • 155
  • 271
1
2 3