Hi all
I’ve been putting this off for about a year now as I managed to eke out some more time from my trusty daily driver. I’m going to be starting a new project that will load up about 20 - 30 gb ( minimum) data into memory and run analysis against it. I have been enjoying reading the book
“Julia high performance 2nd edition” by Avik Sengupta. It’s easy to read, more like a novel really, and lashings of excellent guides relating to getting the most out of Julia. I a particularly interested in the relationship of cpu, memory, gpu ( cuda) and getting to know more about it.
Using advice, over time, from the great people on this forum
I came up with an homebrew approach about $1200 - $1400 with enclosure, power and cooling etc. NOTE the HP ( great workstations in my experience) has 2 Xeon E5-2699 v3 therefore 2 * 18 core and 2 * 36 threads. The cooling is stock and awesome and the power is a beefy 1125w designed for the unit. I can see that the homebuilt has expansion capabilities and the relationship between the components would be faster. I also can see that the cpu’s are AVX2 not AVX512.
either system will have Linux mint 20.2 loaded
HOMEBREW $1500 with cooling, power,case,gpu etc
B550 ,
amd5900x,
64gb ram ( 2 sticks ),
m.2 500gb,
and an old gpu I have lying around until the current silliness abates and I can get a decent one.
VS
EBAY BOUGHT $1500 nothing to add for about a year
( no avx512 though)
https://www.ebay.com/itm/294444768541?hash=item448e469d1d:g:ATMAAOSwveNdyy68
2x Xeon E5-2699 v3 2.3ghz 45mb SmartCache 9.6 GT/s QPI (18 Core) CPUs
8x 16gb 2133p DDR4 Memory = 128gb
1x 500GB SSD Drive
Quadro 2000 1gb GDDR5 (1x DVI – 2x Display Port
So what should I be considering for julia development using core and threads. Thank you
theakson ( chicago)
3rd alternative is renting.
I’m going to be starting a new project that will load up about 2 - 3 gb data into memory
and run analysis against it.
in Europe from EUR 45 / month (+setup) - you have a good machine
or you can find a free service
- Cloud Free Tier | Oracle - always free
- 2 AMD based Compute VMs with 1/8 OCPU** and 1 GB memory each.
- 4 Arm-based Ampere A1 cores and 24 GB of memory usable as one VM or up to 4 VMs.
and later you can decide …
2 Likes
So I will select the Ryzen5950 …
- it is easier to sell later …
- more economy friendly ( less power )
- has a better / faster disk ( Gen4 )
1 Like
Hi there
I thought about renting BUT I sometimes throw my machine into the back of my “car” and go into the depths of Wisconsin to relax and go off grid. I tend to like to design for isolation AND distribution so nothing concentrates the mind like cutting the cord and seeing what breaks. I do have 5G but that gets turned off straight away when I have my first trip to the local diner. thank you for the suggestion though
2 Likes
I did look at the 5950x but couldn’t face paying $800 for just the processor. I tend to stick to $1500 for the whole machine. After that , as you suggested, comes renting.
BUT I sometimes throw my machine into the back of my “car” and go into the depths of
Wisconsin to relax and go off grid.
the other alternative is the Laptop + AND/OR renting
( you can wait for the next “black Friday” )
And buy a cheap (gaming) Laptop with
- nVidia RTX + ( AMD Ryzen™ 7 5800H or Latest Intel ))
IMHO: for learning it is perfect and If you don’t like it you can easily sell 1y later.
And If you need: you can buy a second desktop monitor for home.
example:
https://www.lenovo.com/us/en/laptops/subseries-results?visibleDatas=991:Legion
AMD Ryzen 7 5800H passmark 21621
I’m going to be starting a new project that will load up about 2 - 3 gb data into memory
and run analysis against it.
IMHO: 16GB RAM is fine for starting …
And If you have a bigger data … you can rent a bigger machine for a few hours …
- in the Google Cloud / AWS / …
EDIT:
- or you can buy gaming Notebook with Intel CPU ( + AVX512) + nVidia RTX
1 Like
all excellent suggestions BUT…
EVERY laptop I have ever owned ( except my fav x1 carbon gen 3) has been a disaster. I do see your point though.
I typed in the wrong numbers for the data load, you are NOT dealing with the top tier of intelligence here, I meant 20 - 30gb of data. No doubt this will, like a gas, expand to fill the available memory.
I’m not a huge fan of google, right from day one. I spent a while in the valley before they took off and spotted the change. I’ve seen it before in Yorktown Heights, Redmond etc. It’s not pretty.
thank you for giving me food for thought, exactly what I am looking for.
1 Like
other alternative: https://frame.work/ laptops
it is not nVidia CUDA; but you can test the Intel oneAPI solution
Sometimes I am processing a ~100GB compressed Wikidata JSON DUMP or OpenStreetMap dump on my laptop
( 24GB RAM + 1TB SSD + 8gen Intel CPU - Thinkpad T480s )
My method: extreme prefiltering → loading the minimal data to SQL database (PostgreSQL)
( in my case this is geodata - so PostGIS is a must )
IMHO: with sqlite/postgreSQL/mySQL you can handle data larger than fits into RAM
And with PostgreSQL - you can use Julia as an embedded - procedural language
in the next year: you can expect laptops with 12c - 16c Intel mobil CPU-s - so no easy decision.
+info: Dataframes.jl #1 requests: “handling data larger than fits into RAM”
2 Xeon E5-2699 v3 therefore 2 * 18 core and 2 * 36 threads.
other important: you can expect extremely low single-core performance;
and not all Julia package is optimized for multithreading.
first of all thank you for taking the time and giving me alternatives.
laptops I break them, usually in half. So that’s not something I want as a daily driver.
not really an Intel fan at the moment. I think AMD have better price performance.
I don’t really want to mess with the data if I can avoid it. I’ll have already cleansed it to the bare minimum before loading it up. There’s a LOT of it and I don’t really want to load it into a database.
can’t wait until next year as I’ve put this off long enough.
thank you so much for all your wonderful suggestions and I’ll take the time to investigate all the links you sent.
1 Like
