Let us say a MFLOP has an energetic cost of C. Then M * MFLOP * C is the training cost, and sqrt(M) * MFLOP * C is the query cost. As pointed out above, if the speed of the system serving the query is in the GFLOP/s range and it spends a few seconds on it, the square-root argument is not too outlandish, given that it took similar system days to train.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| AI models are too costly | 78 | 2935 | December 27, 2023 | |
| I Have a dream! a Green dream -- Does Julia save energy? | 21 | 3230 | December 18, 2021 | |
| Which is the best open model for [Julia] coding? And practical, and still reasonably good | 22 | 1579 | August 15, 2026 | |
| How much faster is GPU compare to CPU | 16 | 27698 | November 24, 2018 | |
| Proof of concept LLM chatbot built with Julia: KeemenaLM.jl | 36 | 1683 | August 12, 2026 |