“Pro” programmer since 1977, mostly with C and “Zim DBMS” (now defunct.)
I want to play with neural networks now. Base hardware ultimate is Jetson Nano or Orin, for robot/drone tinkering and LIDAR image processing and recognition, including the use of color-match matching from the stills/videos captured while doing the LIDAR gathering tasks. Pref in live time but during post if need be.
Been studying neural nets from the academic viewpoint for a couple decades and now wish to get down in the weeds and start making actual working programs that DO stuff. 
One more piece of relevant background: currently, my strongest GPU is a 2070 unless you can include some 1980’s/1990’s SGI gear.
Now, to the substance of my questions. All my video cards are based around nVidia because it was the best supported of the neural net tools and is still true with Julia (but getting better all the time – THANKS, guys!) If I can find a good bargain in say, an AMD 7900 version or is it safer to continue with nVidia? Both companies have lost their minds WRT pricing with nVidia being worse by far. However, before I even seriously consider $1700 for an $800 graphic card, I need to be sure I am not totally wasting my meager resources (and I do mean meager!) Even the $800 AMD card is gonna take a couple years to pay off 
.
Since all my AI usage prior to now has been purely theoretical, will switching over my to focus on AMD/ROCm going to add too much extra to my task? Or are the programming algorithms/implementations close enough now that that will be a minor inconvenience? (I hope!)
Secondly, can you guys recommend any literature that can help me along the way? I’m not beyond lifting/copying someone elses code even if I have to (grudgingly) give credit and mention?
Thanks.
If buying a new card would break the bank, then I wouldn’t do it
NVidia cards should work great for many kinds of ML models, and are the best supported today. The AMD side of ML in Julia is getting much stronger this year, but isn’t yet in the position where I’d say you should take a big expense just to use AMD GPUs instead of NVidia’s.
Thankfully, any improvements in our GPU ecosystem tend to be portable to other vendor’s GPUs (namely NVidia, AMD, Intel, and Apple), so if you one day want to switch to a newer card by one of those vendors, it should work pretty well.
If you have a 2070 RTX, that should be strong enough for some experiments? Did you run into any problems yet? I also think that an extra speed-up with a more recent model will not change fundamentally what you can archive. At the end, the GPU memory is the hard limitation, but getting more of that is extremely expensive at some point.
Even though I love Julia, if you really want to get a new GPU at some point, I would keep in mind that many python tools (e.g. pytorch) have much better CUDA support. So, it would be good to keep that door open maybe just to be able to try some other tools.
Maybe Google Colab could be interesting…
A couple of thoughts, Steffen. My uneducated guess is that the sizes of 'nets needed for development can be met by running the training sessions with the 2070 Super (early name for their ti) along with all the other cuda cores spread around on the smaller GPU’s in my stash. Training sessions, of course, requiring much stronger hardware than fielded ones.
PyTorch, AIUI loses ground (speed and memory space) considerably during successive runs, as compared to Julia or C.