Looking to spend between $300 to $800 max for a gpu that will run ai models efficiently. The 1080 TI has 11 GB of ram, but no tensor cores, so it seems like not the best choice.
Just got a new rig, with a 3080 super, which I thought would be good, but it only has 8 GB of ram, big bummer, so I want to replace it with something that will do a better job. What should I get?
If you have a good PSU( above 850w and recent ), a used 3090 is probably your best bet.
I got a membership on google colab and I think it’s really worth it compared to buying your own GPU. The GPUs that you can use there are so much faster than what you can buy with that budget.
I am not sure what you call “run ai models”
I train on GPU but then i move to prod and it runs on CPU. The inference is not near as computation intensive as training so CPU might be enough.
I converted my model to onnx format, wrote code in C# and packaged it like AWS lambda function. My AI model is not called constantly and is only a small part of my project. So my AWS bill is literally couple dollars a month.
Depends on what your requirements for AI models are, both now and in the future.