[其他] ICML-BASE Layers Simplifying Training of Large, Sparse Models [推广有奖]

1关注
167
粉丝

已卖：5684份资源

泰斗

还不是VIP/贵宾

威望: 0 级
论坛币: 13312 个
通用积分: 746.7483
学术水平: 459 点
热心指数: 484 点
信用等级: 414 点
经验: 116309 点
帖子: 4687
精华: 0
在线时间: 16731 小时
注册时间: 2013-1-2
最后登录: 2026-1-30

楼主

Mujahida

发表于 2025-8-10 20:37:44 |AI写论文

是否 +2 论坛币

k人参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

BASE Layers: Simplifying Training of Large, Sparse Models

      Mike Lewis 1 Shruti Bhosale 1 Tim Dettmers 1 2 Naman Goyal 1 Luke Zettlemoyer 1 2

         Abstract                               Worker 1    Worker 2

We introduce a new balanced assignment of ex-    Re-route to original worker  Dogs  bark Cats  purr
perts (BASE) layer for large language models that
greatly simplifies existing high capacity sparse    Mix in expert output:
                           ...