So, I figured there would be some very well informed opinions on here for this rather basic (but essential question). I’ve had a quick look and can’t find a similar question.
I appreciate the answer is likely to be “it depends”. So here is what I’m trying to work out.
Let’s say my main desire to have a “well balanced” GPU. Good for ML, Good for massively parallel custom written stuff (say game theory tree exploration), perhaps even (dare I say it) good for gaming.
I’ve been trying to make sense of FLOPS (SP and DP) per dollar. Is this a useful way to think about it? Why is the ratio between SP and DP usually so high? And what is it about the nvidia 1660 that they had a lower ratio?
What about the GPU on-board RAM, what sorts of tasks tax that? Should that form part of the decision?
Do ray tracing GPUs bring any useful features for GPGPU /computation.
Many thanks in advance for your thoughts, and if this has already been answer comprehensively elsewhere, please accept my apologies and point me the right way.
Good questions. First you need to look at the ‘compute capability’ of the GPU
This tells you which levels of CUDA the GPU is compatible with.
Next the choice is between data centre grade GPUS like V100 and A100
and the consumer grade - these being Titan cards. Titan Black cards were very popular a year or two ago for deep learning.
I know I am not answering your questions directly. I work for Dell as an engineer and really should know better here… in my defence I deal with datacentre projects and have just done my Nvidia A100 training - so I would say the A100 features of a new integer data type and partitionign the GPU are the cat’s pyjamas. I would guess you are not in this market!
Please ping me your email address and I can put you in touch with one of our product specialists.
thanks for the rapid response… I’m also constrained by cost… so much as I’d love a multi thousand dollar card, I’m looking to find a “sweet spot” in the hundreds…
Nvidia seem to push the Titan RTX. Compute capability 7.5 and they say it is a pretty good gaming card too. I will then second this. With the disclaimer that a latest generation card will cost $$$
2850 UK pounds on Amazon UK - wooowoiieee…
yeah, that would be lovely! But money can always buy an awesome solution. I sadly don’t have that sort of budget…hence the question (sorry for not making that clear)
Thanks Oscar, I guess I thought that by going to that forum I’d be asking the people with the domain expertise (and getting there attention since I was in the the "right place for GPUs)… anyway… thanks for the guidance!
Used to be, always go for the 70 series. 970, 1070, etc., until the recent one, where 2060 is the bang for the buck king. That’s quite powerful but still a good value. 2070 is only a bit faster but a lot more, and 2080 is only a little bit faster and a lot lot more, and even the 2080Ti is not too much faster but a ton more. Meanwhile 1660 is a lot slower.
thanks so much for the suggestions so far. having accidentally taken us off down the rabbit-hole of forum decorum - I wondered if anyone had any thoughts on my original questions?
Specifically:
How do you think about choosing a GPU (on a budget!) - ie, less of the what (which is gratefully received!) but more of the how…
Why is the ratio between SP and DP usually so high? And what is it about the nvidia 1660 that they had a lower ratio? (I think I found the answer to this and it’s to do with not having Tensor cores and introducing SP cores instead…)
What about the GPU on-board RAM, what sorts of tasks tax that? Should that form part of the decision?
Do ray tracing GPUs bring any useful features for GPGPU /computation.
I think it is much more simple than that, although I don’t do any GPU computing currently (so my opinion might be useless). Get the best GPU that your budget allows for. And if your purpose is mainly compute rather than gaming, then get Nvidia because that infrastructure is much more built-out (especially in Julia).
The bulk of GPU computing right now is machine learning types of workloads. So if you are doing that, it is worth following the advice above. Right now, I don’t even have a dedicated GPU and I do only CPU compute – it all depends on your workloads.
To sell more expensive cards. Literally, at one point it wasn’t so bad, but then that cut into the data center GPUs, so Nvidia stopped people from using it for FP64 computing through a driver setting. Everyone found out and disabled that to get back the FP64 compute, so they started destroying that part of the cheaper GPUs and sells the same chips with a really high markup for the FP64 (yes, same exact GPU chip in both the $5000 card and the $500 card). Then people started using FP32 a bunch in ML, so they had to ban those from data centers using licensing. Seriously though, if you didn’t live through this story, it’s mind boggling.
