Advanced search options

Advanced Search Options 🞨

Browse by author name (“Author name starts with…”).

Find ETDs with:

in
/  
in
/  
in
/  
in

Written in Published in Earliest date Latest date

Sorted by

Results per page:

You searched for id:"oai:yorkspace.library.yorku.ca:10315/37485". One record found.

Search Limiters

Last 2 Years | English Only

No search limiters apply to these results.

▼ Search Limiters

1. Rashidi, Zana. Adaptive Momentum for Neural Network Optimization.

Degree: MSc -MS, Computer Science, 2020, York University

In this thesis, we develop a novel and efficient algorithm for optimizing neural networks inspired by a recently proposed geodesic optimization algorithm. Our algorithm, which we call Stochastic Geodesic Optimization (SGeO), utilizes an adaptive coefficient on top of Polyaks Heavy Ball method effectively controlling the amount of weight put on the previous update to the parameters based on the change of direction in the optimization path. Experimental results on strongly convex functions with Lipschitz gradients and deep Autoencoder benchmarks show that SGeO reaches lower errors than established first-order methods and competes well with lower or similar errors to a recent second-order method called K-FAC (Kronecker-Factored Approximate Curvature). We also incorporate Nesterov style lookahead gradient into our algorithm (SGeO-N) and observe notable improvements. We believe that our research will open up new directions for high-dimensional neural network optimization where combining the efficiency of first-order methods and the effectiveness of second-order methods proves a promising avenue to explore. Advisors/Committee Members: An, Aijun (advisor).

Subjects/Keywords: Computer science; Machine learning; Optimization; Momentum; Neural networks; Geodesics; Artificial intelligence

Record DetailsSimilar RecordsGoogle PlusoneFacebookTwitterCiteULikeMendeleyreddit

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6th Edition):

Rashidi, Z. (2020). Adaptive Momentum for Neural Network Optimization. (Masters Thesis). York University. Retrieved from https://yorkspace.library.yorku.ca/xmlui/handle/10315/37485

Chicago Manual of Style (16th Edition):

Rashidi, Zana. “Adaptive Momentum for Neural Network Optimization.” 2020. Masters Thesis, York University. Accessed July 08, 2020. https://yorkspace.library.yorku.ca/xmlui/handle/10315/37485.

MLA Handbook (7th Edition):

Rashidi, Zana. “Adaptive Momentum for Neural Network Optimization.” 2020. Web. 08 Jul 2020.

Vancouver:

Rashidi Z. Adaptive Momentum for Neural Network Optimization. [Internet] [Masters thesis]. York University; 2020. [cited 2020 Jul 08]. Available from: https://yorkspace.library.yorku.ca/xmlui/handle/10315/37485.

Council of Science Editors:

Rashidi Z. Adaptive Momentum for Neural Network Optimization. [Masters Thesis]. York University; 2020. Available from: https://yorkspace.library.yorku.ca/xmlui/handle/10315/37485

.