ML Wiki
Search
Search
Explorer
Tag: pre-training
9 items with this tag.
Apr 20, 2026
Denoising
concept
pre-training
self-supervised-learning
Apr 20, 2026
BART: Denoising Sequence-to-Sequence Pre-training
source
encoder-decoder
pre-training
denoising
fine-tuning
masked-language-model
Apr 18, 2026
Evaluating Large Language Models Trained on Code (Codex)
source
code-generation
pre-training
fine-tuning
sampling
scaling-laws
Apr 17, 2026
Language Models are Unsupervised Multitask Learners
source
pre-training
language-model
zero-shot
gpt-2
scaling
multitask
Apr 17, 2026
Llama 2: Open Foundation and Fine-Tuned Chat Models
source
pre-training
rlhf
sft
gqa
alignment
llm
meta-ai
Apr 17, 2026
Masked Autoencoders Are Scalable Vision Learners
source
self-supervised-learning
vision-transformer
pre-training
masked-language-model
patch-embeddings
Apr 17, 2026
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)
source
transfer-learning
pre-training
encoder-decoder
fine-tuning
scaling-laws
nlp
Apr 13, 2026
Masked Language Model
concept
transformers
pre-training
nlp
Apr 13, 2026
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
source
transformers
attention
pre-training
nlp