Technology term
Sparse Mixture of Experts (Sparse MoE)
A neural-network architecture containing many specialized expert modules but activating only a small subset for each input token, increasing total capacity without using every parameter for every computation.
English
Sparse Mixture of Experts
Arabic
مزيج الخبراء المتناثر
First appeared in
Mistral Released Large 4’s Weights With Cybersecurity as the Main Test
