Advanced search options

Advanced Search Options 🞨

Browse by author name (“Author name starts with…”).

Find ETDs with:

in
/  
in
/  
in
/  
in

Written in Published in Earliest date Latest date

Sorted by

Results per page:

You searched for subject:(mini batch). One record found.

Search Limiters

Last 2 Years | English Only

No search limiters apply to these results.

▼ Search Limiters


Vanderbilt University

1. Wang, Xiaotian. Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition.

Degree: MS, Computer Science, 2018, Vanderbilt University

The stochastic gradient descent algorithm for training neural networks is widely used in many machine learning, especially deep learning tasks. The stochastic gradient descent algorithm operates by choosing a small fraction of the training data, called a mini-batch, at each iteration to compute an approximation of the gradient of the objective function to be optimized. In practice, researchers tend to use small batch sizes, and the training data fed into the neural network is usually of various categories and is in random order. Researchers have shown the advantages of smaller sizes of mini-batches quantitatively, yet in the past, there were very few formal investigations into the question of how the order of training data would affect the training efficiency and generalizability of the neural network. To gain more insight into this problem, we have investigated effects of training order and the composition of a mini-batch by conducting a series of controlled experiments. In our experiments, we retrained an existing neural network model for object recognition with images from the ImageNet dataset and from a newly-collected dataset called the Toy-Box dataset. We investigated using optimization techniques like genetic algorithms and simulated annealing to optimize the order of training data. Also, we compared training efficiency for different compositions of mini-batches. Advisors/Committee Members: Richard Alan Peters (committee member), Maithilee Kunda (chair).

Subjects/Keywords: training efficiency; mini-batch; stochastic gradient descent; training order

Record DetailsSimilar RecordsGoogle PlusoneFacebookTwitterCiteULikeMendeleyreddit

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6th Edition):

Wang, X. (2018). Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition. (Masters Thesis). Vanderbilt University. Retrieved from http://etd.library.vanderbilt.edu/available/etd-03232018-130812/ ;

Chicago Manual of Style (16th Edition):

Wang, Xiaotian. “Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition.” 2018. Masters Thesis, Vanderbilt University. Accessed December 13, 2019. http://etd.library.vanderbilt.edu/available/etd-03232018-130812/ ;.

MLA Handbook (7th Edition):

Wang, Xiaotian. “Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition.” 2018. Web. 13 Dec 2019.

Vancouver:

Wang X. Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition. [Internet] [Masters thesis]. Vanderbilt University; 2018. [cited 2019 Dec 13]. Available from: http://etd.library.vanderbilt.edu/available/etd-03232018-130812/ ;.

Council of Science Editors:

Wang X. Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition. [Masters Thesis]. Vanderbilt University; 2018. Available from: http://etd.library.vanderbilt.edu/available/etd-03232018-130812/ ;

.