# AI models are too costly

**URL:** <https://discourse.julialang.org/t/ai-models-are-too-costly/107983>\
**Category:** Offtopic\
**Created:** [December 23, 2023, 6:42pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983 "2023-12-23T18:42:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [December 23, 2023, 6:42pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/1 "2023-12-23T18:42:46Z")

</div>

Do we want to be part of the problem, or part of the solution?

> **[A world suffused with AI probably wouldn’t be good for us – or the planet |...](https://www.theguardian.com/commentisfree/2023/dec/23/ai-chat-gpt-environmental-impact-energy-carbon-intensive-technology)**
>
> Amid all the hysteria about ChatGPT and co, one thing is being missed: how energy-intensive the technology is

---

<div class="post-metadata">

**Author:** ![KZiemian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kziemian/32/9020_2.png) [@KZiemian](https://discourse.julialang.org/u/KZiemian)\
**Post date:** [December 23, 2023, 6:48pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/2 "2023-12-23T18:48:40Z")

</div>

Of course I want to be part of solution. But, it is easier said than done. I quite ignorant about AI and about managing CO2.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [December 24, 2023, 5:15pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/3 "2023-12-24T17:15:15Z")

</div>

Sign up at [https://scientistrebellion.org/](https://scientistrebellion.org/) … You will find many colleagues to discuss this topic and - perhaps - find a way to act…

We had a lot of nice actions in NL this year, see: [Past Actions and Events – Scientist Rebellion Netherlands](https://www.scientistrebellion.nl/index.php/schedule/)

And it’s fun!

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [December 24, 2023, 11:04pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/4 "2023-12-24T23:04:52Z")

</div>

I hope publishing a link to [https://clintel.org](https://clintel.org) and to [Critique of the AR6 WG1 Summary for Policymakers (SPM)](https://clintel.org/wp-content/uploads/2021/10/Critique-of-AR6-Clintel-ICSF.pdf) will not be considered inappropriate here.

I must say I am not a climate scientist, but I am at least physicist and chemist enough to understand how complicated the matter is.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [December 24, 2023, 11:28pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/5 "2023-12-24T23:28:39Z")

</div>

I was perplexed to find professor emeritus of geophysics Guus Berkhout, winner of several [industry awards](https://wiki.seg.org/wiki/Augustinus_Berkhout), at the head of a climate-skeptic organization. Whatever our impact on the climate, that on biodiversity is undeniable.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [December 25, 2023, 9:46am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/6 "2023-12-25T09:46:35Z")

</div>

The UN Secretary General António Guterres said:

“We are miles from the goals of the Paris Agreement – and minutes to midnight for the 1.5-degree limit,” Guterres went on. “But it is not too late.”

There are many small things we can do: For example if we need a new laptop, buy one  
that can be easily extended and repaired like the Framework 13: [Framework | Framework Marketplace | Framework Laptops](https://frame.work/nl/en/marketplace/laptops)

Or join a climate fresk, which is a card game for 6-30 people where you learn all about the reasons and consequences of climate change, see: [https://climatefresk.org/](https://climatefresk.org/)

In Delft, NL we are trying to organize a climate fresk for all of our students.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [December 25, 2023, 11:42am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/7 "2023-12-25T11:42:03Z")

</div>

Well, first I’d say “climate-skeptic” (in German they even say “Klimaleugner”, climate denialist) is too much of a shortcut for a somebody skeptical about the aptitude of the current climate models for the medium- to long-term climate forecast.

Second, there are a lot of things, simple and obvious to a layman (Sun movement about the flat Earth e.g. 😜), but less so to an expert.

Then, even if our (i.e. humanity) impact on the climate and on the biodiversity is undeniable, there are many open questions. How reliable are the predictions? What kind of impact is still acceptable? What can be done to alleviate it? What is the (environmental, economical, political) price? For example huge areas used for biofuel crops are probably detrimental for biodiversity. The wind wheels, destroying birds and insects en masse, either.

Now to the _emeritus_ point. I am/was taking part in several projects “to save the climate”, mostly on governmental grants, but one from a large governmentally controlled industrial company. Some of them IMO could have a chance for a technically and economically sound use. For some I do not see how the energetic effciency can grow above a fraction of a per cent (there exist alternative well established processes yielding better than at least 10%), however my boss - who is not a physicist - believes there are chances. But with one of the last projects, as he presented it to the group, he started with “please don’t ask me any questions about efficiency!” This is a multi-million project, of which we have but a small part, and the problems must be equally obvious to all other partners, which are mostly competent scientists and engineers.

But what would you do if you are not only want to keep your job, but also responsible for the jobs of a dozen of colleagues, some of them on short-term contracts?! Also, that kind of projects are not only ones in our department, there is also sound (environmentally relevant) science and engineering, so even from the point of “saving the planet” you may want to keep it running. So, take the money and participate.

Being an emeritus professor may be the only realistic chance to be heard (because renowned) while not jeopardizing anybody’s jobs (being emeritus) if you are against the political mainstream.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [December 25, 2023, 11:50am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/8 "2023-12-25T11:50:31Z")

</div>

> [@ufechner7](#):
>
> And it’s fun!

I do hope this potentially explosive thread will be closed soon without removing any posts.

Still, I’d like to note that the organization, advertised by @ufechner7 , is by their own proclamation engaged in activities which are most probably illegal, like stopping a train.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [December 25, 2023, 12:02pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/9 "2023-12-25T12:02:56Z")

</div>

> [@Eben60](#):
>
> activities which are most probably illegal, like stopping a train.

Well, we are involved in [Civil disobedience - Wikipedia](https://en.wikipedia.org/wiki/Civil_disobedience) , yes.

Without civil disobedience we would still have “Racial segregation” in the US…

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [December 25, 2023, 6:28pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/10 "2023-12-25T18:28:41Z")

</div>

I’m not entirely against AI, even the costly generative models, if they can find some academic purposes (approach these news with healthy skepticism, it tends to be overhyped through the news cycle). But I wonder how much demand there would be for generative AI if we culturally and legally determine that the generated content and any significantly derivative work cannot be credited to, owned by, or sold by any particular human or organization. Personally I think if it’s obviously wrong for students to generate and edit essays to save time on schoolwork, it’s just as wrong for AI “artists” to occupy the market with products made cheap by never compensating the creators of the training data.

> [@ufechner7](#):
>
> Without civil disobedience we would still have “Racial segregation” in the US…

It’s well-substantiated that it still does, it just must be legally indirect.

---

<div class="post-metadata">

**Author:** ![Jake](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jake/32/46007_2.png) [@Jake](https://discourse.julialang.org/u/Jake)\
**Post date:** [December 25, 2023, 9:05pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/11 "2023-12-25T21:05:52Z")

</div>

As someone with training in heat transfer but not in climatology and a general interest in the world around me, I find the global warming topic intriguing. If we take [Edgar Andrew’s hypothesis](http://whomadegod.org/) that the God of the Bible has better predictive power than the other models explaining (as opposed to describing it) the beginning and maintenance of the world around us, as I do, then it may be time that we look at what He has to say about this subject. This can be found in Gen 6:5-7, Duet 28:12, and many more.

From what I understand the most significant green house gas is H2O. This is probably somewhere around 95% of the total greenhouse gas effect. However the more water in the atmosphere the more clouds there are which is great to enhance the greenhouse effect, and also great to reflect sunlight. So it seems that these effects dominate the absorption of radiation from the sun and the emission of radiation from the earth to space and tend to cancel each other out. And it is interesting to remember that radiation follows the 4th power to help keep the earth’s temperature stable.

Merry Christmas to all!

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [December 25, 2023, 11:27pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/12 "2023-12-25T23:27:23Z")

</div>

This is all very interesting, but what about the topic of the starter post? AI models are too expensive to be practical, yet everyone seems to be eager to jump on the band wagon and cry hurrah. Just in my field (computational mechanics) there are suddenly hundreds of papers. Never mind that the proposed solutions are often ridiculously expensive and do not _solve_ anything. Just mention AI or ML somewhere in the paper, and you will get published. That is a problem, isn’t it?

---

<div class="post-metadata">

**Author:** ![Jose\_Diaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jose_diaz/32/43750_2.png) [@Jose\_Diaz](https://discourse.julialang.org/u/Jose_Diaz)\
**Post date:** [December 26, 2023, 12:18am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/13 "2023-12-26T00:18:44Z")

</div>

From the inference point of view, I think Julia’s KernelAbstractions.jl is one of the best things to help this issue. There is a ton of AI chip startups building new AI accelerators. That package can make things easier to integrate new, more efficient, hardware

I think it’s cool it already supports AMD, since NVIDIAs 4000 series are too intense in power usage.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [December 26, 2023, 6:05am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/14 "2023-12-26T06:05:06Z")

</div>

> [@PetrKryslUCSD](#):
>
> Just mention AI or ML somewhere in the paper, and you will get published. That is a problem, isn’t it?

Classic issue of journals prioritizing popular topics over less exciting results. Still, it’s worth mentioning that the paper is comparing classes of AI, finding that classification is cheap and that multitask and generative AI is expensive. It is not comparing it to other computationally heavy tasks, like physical simulations or a video game server. That would be important information for forming ethical lines for energy use; an academic project could weigh training 1 multitask model versus multiple single-task models versus untrained algorithms.

I personally am already ready to draw the line with image generation, given how much more expensive inference is and how it tends to be used: selfies and skipping art classes.

---

<div class="post-metadata">

**Author:** ![sob](https://avatars.discourse-cdn.com/v4/letter/s/65b543/32.png) [@sob](https://discourse.julialang.org/u/sob)\
**Post date:** [December 26, 2023, 10:27am UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/15 "2023-12-26T10:27:29Z")

</div>

How costly is it to query a locally run model? Say, for the sake of example, if I want to build a service that does local chi-chat with a ChatGPT like model, run locally, is it the kind of thing that could host, 1e3, 1e4, 1e5 concurrent convos on mid range consumer grade 2023 hardware?

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [December 26, 2023, 12:24pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/16 "2023-12-26T12:24:42Z")

</div>

To answer your original question: Using AI is like taking a plane,  
bad for the environment and sometimes unavoidable.

So try to apply similar rules:

- do it only if it really solves a practical problem of mankind  
that cannot be solved otherwise
- as a reviewer, always ask the people who write papers using AI  
if they considered the environmental footprint and why they  
think it is justified

What we (in the western world) need is degrowth, less and smaller  
cars, less, smaller and and longer living computers etc…  
More health and happiness, less usage of resources…

---

<div class="post-metadata">

**Author:** ![sob](https://avatars.discourse-cdn.com/v4/letter/s/65b543/32.png) [@sob](https://discourse.julialang.org/u/sob)\
**Post date:** [December 26, 2023, 1:24pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/17 "2023-12-26T13:24:37Z")

</div>

I disagree with this premise, and I strongly disagree with the actions of the organization you provided a link to. I think it’s best to keep this topic at the energy consumption of so-called “AI” and not veer into the political realm.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [December 26, 2023, 1:59pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/18 "2023-12-26T13:59:28Z")

</div>

> [@sob](#):
>
> I think it’s best to keep this topic at the energy consumption of so-called “AI” and not veer into the political realm.

Well, ethics is important in the field of engineering. How do you want to decide the statement of the topic, what is “too costly”? That is an ethical question as long as Universities are concerned.

Luckily in NL attending an ethics course is mandatory for engineering students. See for example: [Engineering Ethics - Introduction](https://www.tutorialspoint.com/engineering_ethics/engineering_ethics_introduction.htm)

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [December 26, 2023, 2:08pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/19 "2023-12-26T14:08:08Z")

</div>

IMHO we are deeply in need of rigorous energy / carbon intensity metrics for computational algorithms in general.

Modelling papers should be required to specify computational power required for the analysis. This would be especially important in fields like genomics where the energy use of algorithms is a large fraction of total departmental energy use.

It would also clearly be good for Julia. Ecologists where I work reach for the cluster to run their R scripts when the same code in Julia runs in seconds on a laptop.

---

<div class="post-metadata">

**Author:** ![yvikhlya](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yvikhlya/32/3753_2.png) [@yvikhlya](https://discourse.julialang.org/u/yvikhlya)\
**Post date:** [December 26, 2023, 2:35pm UTC](https://discourse.julialang.org/t/ai-models-are-too-costly/107983/20 "2023-12-26T14:35:54Z")

</div>

> [@Eben60](#):
>
> medium- to long-term climate forecast

You probably mean `scenarios`, not `forecasts`. These are two very different things. We can not forecast anything beyond few seasons.

[Next page](https://discourse.julialang.org/t/ai-models-are-too-costly/107983.md?page=2)