Steerable representations with layer-wise surrogate objectives for deep neural networks
Neural networks are typically trained end-to-end via gradient back propagation using stochastic gradient descent and a global optimisation function. This induces internal representations of the data that learn progressive linear separability. this behaviour, and thus the benefits thereof, using layer-wise training has yet to be achieved. This research studies surrogate objectives to steer internal toward advantageous better suited to match end-to-end training paradigms. Layer-wise learning enables progressive neural network growth and efficient optimisation.