Steerable representations with layer-wise surrogate objectives for deep neural networks

 Steerable representations with layer-wise surrogate objectives for deep neural networks

Luke Darlow

 

Neural networks are typically trained end-to-end via gradient back propagation using stochastic gradient descent and a global optimisation function. This induces internal representations of the data that learn progressive linear separability. Mimicking this behaviour, and thus the benefits thereof, using layer-wise training has yet to be achieved. This research studies surrogate objectives to steer internal representations toward advantageous characteristics better suited to match end-to-end training paradigms. Layer-wise learning enables progressive neural network growth and efficient optimisation.


Supervisors: Amos Storkey & Chris Williams