Publication Date

Spring 4-4-2023

Abstract

Large neural networks have become ubiquitous in machine learning. Despite their widespread use, the optimization process for training a neural network remains com-putationally expensive and does not necessarily create networks that generalize well to unseen data. In addition, the difficulty of training increases as the size of the neural network grows. In this thesis, we introduce the novel MGDrop and SMGDrop algorithms which use a multigrid optimization scheme with a dropout coarsening operator to train neural networks. In contrast to other standard neural network training schemes, MGDrop explicitly utilizes information from smaller sub-networks which act as approximations of the full network. We empirically show that both MGDrop and SMGDrop perform comparably to existing training algorithms and in some cases are able to beat current algorithms in terms of accuracy and training time. In addition, we derive theoretical descriptions of the underlying update rules and their effects on network gradients.

Degree Name

Mathematics

Level of Degree

Masters

Department Name

Mathematics & Statistics

First Committee Member (Chair)

Jacob Schroder

Second Committee Member

Mohammed Motamed

Third Committee Member

Eric Cyr

Language

English

Keywords

machine learning, multigrid, optimization, dropout

Document Type

Thesis

Download

Share

COinS