Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [gru]

A Gated Recurrent Unit (GRU) is a type of unit in a recurrent neural network.

33 questions
37
votes
5 answers

Difference between feedback RNN and LSTM/GRU

I am trying to understand different Recurrent Neural Network (RNN) architectures to be applied to time series data and I am getting a bit confused with the different names that are frequently used when describing RNNs. Is the structure of Long…
Josie
  • 473
  • 1
  • 4
  • 5
12
votes
2 answers

How many parameters are in a gated recurrent unit (GRU) recurrent neural network (RNN) layer?

The title says it all -- how many trainable parameters are there in a GRU layer? This kind of question comes up a lot when attempting to compare models of different RNN layer types, such as long short-term memory (LSTM) units vs GRU, in terms of the…
Sycorax
  • 76,417
  • 20
  • 189
  • 313
9
votes
1 answer

Sudden accuracy drop when training LSTM or GRU in Keras

My recurrent neural network (LSTM, resp. GRU) behaves in a way I cannot explain. The training starts and it trains well (the results look quite good) when suddenly accuracy drops (and loss rapidly increases) - both training and testing metrics.…
Marek
  • 395
  • 5
  • 11
8
votes
2 answers

What is the output of a tf.nn.dynamic_rnn()?

I am not sure about what I understand from the official documentation, which says: Returns: A pair (outputs, state) where: outputs: The RNN output Tensor. If time_major == False (default), this will be a Tensor shaped: [batch_size, max_time,…
MiloMinderbinder
  • 1,622
  • 2
  • 15
  • 31
5
votes
1 answer

State-of-the-art algorithms for the training of neural networks with GRU or LSTM units

I recently read a lot about neural networks using GRU or LSTM units. There are many easy to use frameworks like tensorflow that do not even require high knowledge about programming. Unfortunately, I never really found good information on how the…
Sandreal
  • 53
  • 6
5
votes
1 answer

Is anyone stacking LSTM and GRU cells together and why?

TensorFlow allows you to create MultiRNNCell composed sequentially of multiple simple cells (LSTM and GRU). I usually use same type of cell when creating MultiRNNCell but I was wondering if there could be some benefits in using both LSTM and…
Drag0
  • 650
  • 6
  • 12
4
votes
0 answers

RNN vs ResNet for multivariate time series prediction

All others being equal, would a ResNet-based or RNN-based neural network (with/without an attention mechanism) perform better for forecasting a multivariate time series? Related: Deep learning for time series classification: a review explains that…
LMB
  • 257
  • 2
  • 10
4
votes
1 answer

Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?

I would like to compute the similarity between audio signals of different length. One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature vectors (of same dimension) of each audio. The…
Christopher Chong
  • 41
  • 1
4
votes
1 answer

What Possible Models For Time-Series when Data is Scarce

Each financial quarter I collect data on the number of potential clients, contacted potential clients, and potential clients that become actual clients. I have this quarterly data going back only 6 years (6*4=24 total time steps). If I want to…
Adrix
  • 143
  • 4
4
votes
1 answer

How does Keras generate an LSTM layer. What's the dimensionality?

In Keras I can define the input shape of an LSTM (and GRU) layers by defining the number of training data sets inside my batch (batch_size), the number of time steps and the number of features. So I could configure an LSTM or a GRU like…
Laokoon
  • 161
  • 1
  • 5
2
votes
1 answer

How GRU solves vanishing gradient

I am learning the GRU model in deep learning and reading this article where details of BPTT are explained. Towards the end the author explained the values of the partial derivative $\frac{\partial h_i}{\partial h_{i-1}}$ in 3 cases: (1). $z= 1$.…
siegfried
  • 318
  • 1
  • 7
2
votes
1 answer

Number of cells in an RNN

I have been reading about RNNs and I have some confusion between the number of timesteps and number of units in an RNN layer (which after searching for an answer) seems to be a common thing. I understand that the idea behind using a RNN is to…
Mohammad Nur
  • 31
  • 5
2
votes
1 answer

GRU Hidden State Output Formula Difference

Looking into GRU equation I see 2 type for final output. One is from, https://d2l.ai/chapter_recurrent-modern/gru.html#hidden-state, that is, $ \mathbf{R}_t = \sigma(\mathbf{X}_t \mathbf{W}_{xr} + \mathbf{H}_{t-1} \mathbf{W}_{hr} + \mathbf{b}_r)…
B200011011
  • 121
  • 5
2
votes
1 answer

Parameters count in GRU layer keras

I have this model structure and want to know the formula for calculating the parameters count in the GRU layer. I did not find that in the docs. model = Sequential() model.add(layers.GRU(32, input_shape=(None, float_data_1.shape[-1])))…
Hesham
  • 21
  • 2
2
votes
2 answers

Is there any difference between forget gates and remember gates?

In this youtube tutorial by nervada the following diagrams can be seen: The remember gate in Gated Recurrent Units (GRUs) in the diagram to the right seems to have an analogous function to the forget gate in LSTM (left diagram): filter out how much…
Antoni Parellada
  • 23,430
  • 15
  • 100
  • 197
1
2 3