RandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 16 hours agoElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comexternal-linkmessage-square106fedilinkarrow-up1744arrow-down113
arrow-up1731arrow-down1external-linkElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comRandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 16 hours agomessage-square106fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·edit-23 hours agoYou should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete. I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.
You should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete.
I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.