Home Glossary Mixture of Experts (MoE)

Mixture of Experts (MoE) - Page 12

A mixture of experts, or MoE, is a model architecture that contains multiple specialized subnetworks and a routing system that selects which experts handle each input. Only a subset may be activated for a given token or example, allowing the model to contain many parameters without using all of them for every calculation. This can expand capacity while controlling inference cost. Training remains complex because routing must distribute work effectively, avoid overloading a few experts, and maintain communication across hardware. An MoE model is not automatically interpretable simply because its components are called experts; their learned specializations may be unstable or difficult to describe and must be evaluated empirically.

Andrew Tulloch Leaves $12B AI Startup to Join Meta After Turning Down $1.5B Offer
By • 3 mins read
AI & Machine Learning, Immersive Reality (AR, VR, MR, and XR), News, Startups & Investment

Andrew Tulloch Leaves $12B AI Startup to Join Meta After Turning Down $1.5B Offer

By • 3 mins read

Andrew Tulloch, co-founder of the $12 billion AI startup Thinking Machines Lab, has joined Meta after previously rejecting what reports described as a $1.5 billion offer — a figure Meta has since called ‘inaccurate and ridiculous.’