Advanced search options

Advanced Search Options 🞨

Browse by author name (“Author name starts with…”).

Find ETDs with:

in
/  
in
/  
in
/  
in

Written in Published in Earliest date Latest date

Sorted by

Results per page:

You searched for id:"handle:1773/46825". One record found.

Search Limiters

Last 2 Years | English Only

No search limiters apply to these results.

▼ Search Limiters


University of Washington

1. Campos, Daniel. Explorations In Curriculum Learning Methods For Training Language Models.

Degree: 2021, University of Washington

Understanding language depending on the context of its usage has always been one of thecore goals of natural language processing. Recently, contextual word representations created by language models like ELMo, BERT, ELECTRA, and RoBERTA have provided robust representations of natural language which serve as the language understanding component for a diverse range of downstream tasks like information retrieval, question answering, and information extraction. Curriculum learning is a method that employs a structured training regime instead of the traditional random sampling. Research areas like computer vision and machine translation have used curriculum learning methods in model training to improve model training speed and model performance. While language models have proven transformational for the natural language processing community, these models have proven expensive, energy-intensive, and challenging to train, which has inspired researchers to explore new training methods. In this thesis, we explore the effect of curriculum learning in the training of language models. Using wikitext-2 and wikitext-103 textual datasets and evaluating word representation transfer learning on the GLUE Benchmark, we find that curriculum learning methods produce models that outperform their traditionally trained counterparts when the training corpus is small, but as the training corpora scale, curriculum methods become less effective than traditional stochastic sampling. Advisors/Committee Members: Steinert-Threlkel, Shane (advisor).

Subjects/Keywords: Curriculum Learning; Deep Learning; Language Model; Computer science; Language; Linguistics

Record DetailsSimilar RecordsGoogle PlusoneFacebookTwitterCiteULikeMendeleyreddit

APA · Chicago · MLA · Vancouver · CSE | Export to Zotero / EndNote / Reference Manager

APA (6th Edition):

Campos, D. (2021). Explorations In Curriculum Learning Methods For Training Language Models. (Thesis). University of Washington. Retrieved from http://hdl.handle.net/1773/46825

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Chicago Manual of Style (16th Edition):

Campos, Daniel. “Explorations In Curriculum Learning Methods For Training Language Models.” 2021. Thesis, University of Washington. Accessed April 22, 2021. http://hdl.handle.net/1773/46825.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

MLA Handbook (7th Edition):

Campos, Daniel. “Explorations In Curriculum Learning Methods For Training Language Models.” 2021. Web. 22 Apr 2021.

Vancouver:

Campos D. Explorations In Curriculum Learning Methods For Training Language Models. [Internet] [Thesis]. University of Washington; 2021. [cited 2021 Apr 22]. Available from: http://hdl.handle.net/1773/46825.

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Campos D. Explorations In Curriculum Learning Methods For Training Language Models. [Thesis]. University of Washington; 2021. Available from: http://hdl.handle.net/1773/46825

Note: this citation may be lacking information needed for this citation format:
Not specified: Masters Thesis or Doctoral Dissertation

.