Thanks to Micro Center for sponsoring this video! Shop Back to School Tech Deals: https://micro.center/a6ef91Sign up for a FREE 128 gig Flash Drive at Micro ...

*Western AI companies are terrified

Anyway, what do you all think of the news?

Video is 8 minutes and 30 seconds long if you’re interested.

Is it true that the DeepSeek models and that other AI model (I forgot the name already lol) can be run locally? How do they compete with Claude and OpenAI?

Also, what about that other AI thing, OpenClaw?

I am interested in AI but still very much skeptical.

I have only used Duck AI sometimes but try to avoid it.

  • Faux@lemmygrad.ml
    6·
    6 days ago

    You need 128GB of VRAM for aggressive quantization of DeepSeek Flash. 16GB might be enough for aggressive quantization of smaller models. In theory you can run Qwen3.6 27B in 2-bit quantization (or something like https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf ) and have some space for context but it’s a toy. You need at least 24GB of VRAM to use reasonable quant of Qwen3.6 27B (or 32GB of shared VRAM to handle Qwen3.6 35BA3B which is faster but takes more VRAM than 27B) in actual tasks.

    • jmo@lemmygrad.ml
      3·
      6 days ago

      I’ve had great luck running qwen2.5-coder:7b on my work laptop. It’s great for a wide range of dev and operational tasks with opencode.