ML Wiki

Tag: mixture-of-experts

4 items with this tag.

  • Apr 27, 2026

    Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context

    • source
    • long-context
    • mixture-of-experts
    • multimodal
    • architecture
    • scaling
    • in-context-learning
  • Apr 24, 2026

    Mixture of Depths: Dynamically Allocating Compute in Transformer LLMs

    • source
    • inference-efficiency
    • mixture-of-experts
    • dynamic-computation
    • training
  • Apr 16, 2026

    Switch Transformers: Scaling to Trillion Parameter Models with Sparse MoE

    • source
    • mixture-of-experts
    • moe
    • scaling
    • efficiency
    • sparse
  • Apr 10, 2026

    Mixtral of Experts

    • source
    • mixtral
    • mixture-of-experts
    • moe
    • sparse-moe
    • inference-efficiency
    • open-weights