Advanced search options

Advanced Search Options 🞨

Browse by author name (“Author name starts with…”).

Find ETDs with:

in
/  
in
/  
in
/  
in

Written in Published in Earliest date Latest date

Sorted by

Results per page:

You searched for id:"handle:10754/667327". One record found.

Search Limiters

Last 2 Years | English Only

No search limiters apply to these results.

▼ Search Limiters


King Abdullah University of Science and Technology

1. Alnasser, Ali. An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training.

Degree: Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, 2021, King Abdullah University of Science and Technology

Neural networks optimizers are dominated by rst-order methods, due to their inexpensive computational cost per iteration. However, it has been shown that rstorder optimization is prone to reaching sharp minima when trained with large batch sizes. As the batch size increases, the statistical stability of the problem increases, a regime that is well suited for second-order optimization methods. In this thesis, we study a distributed ellipsoidal trust region model for neural networks. We use a block diagonal approximation of the Hessian, assigning consecutive layers of the network to each process. We solve in parallel for the update direction of each subset of the parameters. We show that our optimizer is t for large batch training as well as increasing number of processes. Advisors/Committee Members: Keyes, David E. (advisor), Wonka, Peter (committee member), Zhang, Xiangliang (committee member).

Subjects/Keywords: optimization; trust region; distributed computing; deep learning; machine learning

Record DetailsSimilar RecordsGoogle PlusoneFacebookTwitterCiteULikeMendeleyreddit

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6th Edition):

Alnasser, A. (2021). An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training. (Thesis). King Abdullah University of Science and Technology. Retrieved from http://hdl.handle.net/10754/667327

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16th Edition):

Alnasser, Ali. “An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training.” 2021. Thesis, King Abdullah University of Science and Technology. Accessed March 01, 2021. http://hdl.handle.net/10754/667327.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7th Edition):

Alnasser, Ali. “An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training.” 2021. Web. 01 Mar 2021.

Vancouver:

Alnasser A. An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training. [Internet] [Thesis]. King Abdullah University of Science and Technology; 2021. [cited 2021 Mar 01]. Available from: http://hdl.handle.net/10754/667327.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Alnasser A. An Empirical Study of the Distributed Ellipsoidal Trust Region Method for Large Batch Training. [Thesis]. King Abdullah University of Science and Technology; 2021. Available from: http://hdl.handle.net/10754/667327

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

.