If you wanted to finetune a Mixtral 8x7B, what would you use? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		robrenaud on April 4, 2024 \| parent \| context \| favorite \| on: JetMoE: Reaching LLaMA2 performance with 0.1M doll... If you wanted to finetune a Mixtral 8x7B, what would you use?

_ea1k on April 5, 2024 [–]

Given the relative availability, I'd probably try to do it with a couple of rtx4090s on tensordock.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact