ML Wiki

Tag: pre-training

9 items with this tag.

  • Apr 20, 2026

    Denoising

    • concept
    • pre-training
    • self-supervised-learning
  • Apr 20, 2026

    BART: Denoising Sequence-to-Sequence Pre-training

    • source
    • encoder-decoder
    • pre-training
    • denoising
    • fine-tuning
    • masked-language-model
  • Apr 18, 2026

    Evaluating Large Language Models Trained on Code (Codex)

    • source
    • code-generation
    • pre-training
    • fine-tuning
    • sampling
    • scaling-laws
  • Apr 17, 2026

    Language Models are Unsupervised Multitask Learners

    • source
    • pre-training
    • language-model
    • zero-shot
    • gpt-2
    • scaling
    • multitask
  • Apr 17, 2026

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    • source
    • pre-training
    • rlhf
    • sft
    • gqa
    • alignment
    • llm
    • meta-ai
  • Apr 17, 2026

    Masked Autoencoders Are Scalable Vision Learners

    • source
    • self-supervised-learning
    • vision-transformer
    • pre-training
    • masked-language-model
    • patch-embeddings
  • Apr 17, 2026

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)

    • source
    • transfer-learning
    • pre-training
    • encoder-decoder
    • fine-tuning
    • scaling-laws
    • nlp
  • Apr 13, 2026

    Masked Language Model

    • concept
    • transformers
    • pre-training
    • nlp
  • Apr 13, 2026

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    • source
    • transformers
    • attention
    • pre-training
    • nlp