Deep Noodle
/
Cerebras runs Llama 3.1 405B at 969 tokens per second, a new inference record
Cerebras reported running Meta’s Llama 3.1 405B at 969 tokens per second, roughly twelve times the fastest GPU result available at the time.
Sources
Cerebras
Announcement · Nov 18, 2024
Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference
