During forward propagation in an RNN, the output at time step t is heavily influenced by
earlier inputs. Which mathematical property of RNN weight matrices is primarily responsible
for this temporal influence and also the root cause of gradient issues?

Question

During forward propagation in an RNN, the output at time step t is heavily influenced by 
 earlier inputs. Which mathematical property of RNN weight matrices is primarily responsible 
 for this temporal influence and also the root cause of gradient issues?

Answer

a.  Repeated multiplication of the hidden state weight matrix Wh

Answer

b.  Sigmoid activation causing zeroing

Answer

c.  Orthogonality of matrices

Answer

d.  Exploding values due to high norms

Crowdly

During forward propagation in an RNN, the output at time step t is heavily influ...

Want instant access to all verified answers on moodle.iitdh.ac.in?