AI models are too costly

Hey @Raf , out of curiosity, are there any sorts of metrics like this in existence? It feels like a good idea!

Should a material scientist report the energy consumption for their experiments to be able to publish a paper?

P.S. We have a few electric furnaces in the basement with nominal power ranging from 3 to 50 kW, if I remember it correctly. It gets quite warm there, especially in the summer.

Iā€™m not really sure. But Iā€™ve been thinking about writing a suite of model evaluation tools that includes this. Like comparing models should be done on multiple axes:

  • sized of gzipped significant code used
  • size of input data
  • run time
  • total energy intensity / CO2 use

Like you should be able to wrap your function calls in a call to the package and get these stats back at the end / have them written to file. And include that in your paper / appendix to demonstrate that your methods practically make sense as well as theoretically. I think its pretty big task to make that work over distributed machines but I think its possible.

2 Likes

It is interesting how people immediately link high energy usage to CO2 emissions, like if it is our biggest environmental problem. Although this problem does exist, it is actually not the only one and not even the worst one.

2 Likes

Probably, eventually, yes?

I guess mostly in fields where energy use is actually significant, Iā€™m not talking about being pedantic about everything.

@yvikhlya Iā€™m not sure what point youā€™re trying to make here. What do you think people should worry about with energy usage?

IMHO rapid land cover change is our biggest environmental problem, and thatā€™s what I personally work on the most. But climate is right up there and will only become more significant. Iā€™m struggling to think of other consequences of energy use that compare to it. (Please be specific?)

Nice!

Now, what if we buy ready reagents, where the energy consumption happened upstream? E.g. we use noble metals in some experiments. I donā€™t know it exactly, but I guess a large part of itā€™s price is energy consumption (another part being probably slave-like labour somewhere in Brazil or Russia).

Absolutely, detailed upstream energy accounting is needed basically everywhere in society.

Agreed, currently the west is hiding a big chunk of its resource use in the countries that actually make things and dig things up. Where I live currently, Denmark, gets accolades for its carbon footprint but when you look at things that arenā€™t counted like flights and upstream costs of consumer consumption its clearly a scam - someone else is getting blamed for Denmarks real environmental footprint.

So doing it just for materials science is a big ask when we arenā€™t applying it to much larger systems, and I personally feel bad putting these requirements on scientists when the rest of society mostly ignores them. Like stopping climate research because its energy intensive seems like a bad idea.

2 Likes

Each time energy topics comes out, everybody start talking about carbon emissions and reaching carbon neutrality. This is so ridiculous. Oh, letā€™s spend trillions to reach carbon neutrality and environment problem solved. No people, none of your problems is going to be solved, you just flash your trillions down the toilet in best case (probably create many more problems down the road). Energy is just one of scarce resources which we critically depend on and deplete very rapidly. Depleting of resources is #1 biggest problem (energy, land, water, forest, ores etc ). Extraction and usage of any of these resources is devastating for environment. While we consume precious resources, we produce so much waste which is so much worse than CO2. Think about toxic emissions which kill thousands or tones of plastic garbage for example. Oh yeah, lets talk about CO2 and forget everything else.

1 Like

Thatā€™s huge bureaucracy with zero efficiency. Sure, your bureaucrats will get all certificates they need from China, Brazil, Russia, not to mention African countries.

I mostly agree with you, but really most wastes are not worse than CO2. They are just other things that are also bad, just like pumping waste CO2 into the atmosphere is bad.

Solving climate change is not going to solve all of our problems, true. But saying Climate Change is actually not really a problem is equally wrong. The reality is climate change is a huge problem, but we have other equally bad and worse problems at the same time, as you say, like resource use.

@Eben60 what is your point, that we shouldnā€™t try and track or understand resource use at all because its hard? I donā€™t get what you want instead. @yvikhlya is saying resource depletion is the worse thing ever, and you are saying we shouldnā€™t try to account for it, but thanking them at the same time.

5 Likes

He spent his career developing technology to aid petroleum exploration and development. Climate deniers are common in that industry. I work among them myself.

2 Likes

Strongly disagree. Ok, CO2 contributes to greenhouse effect (among other agents), but it is not toxic at least. We pollute such a lot of really scary toxic crap into our environment. And some proposed methods of reaching ā€œcarbon neutralityā€, which come out in the media from time to time are really terrifying.

What a fantastic idea. I am not too well-versed in this space, but if ever you want to collaborate on something like this Raf, happy to work with you within JuliaHealth! Iā€™d be curious about these kinds of impacts in the sort of global health informatics studies I personally perform and explore. Especially as some studies I run take several hours or even days to execute properly.

1 Like

My main point here is actually that environment problem canā€™t be solved by focusing only on single aspect of it, like CO2 and reaching carbon neutrality. It should be considered as a whole, taking all factors into account together, because it is like with coding, you fix one thing and break ten others at the same time if you do it wrong.

And talking about climate change, we canā€™t stop it. Climate changes dramatically regardless of whether we exist or not. We should be really careful with how and how much we contribute to this change. This is something that is often misunderstood in public.

2 Likes

Great! yes it would be good to have a team across fields to do this to make it more generic. I donā€™t know when I will have the time to work on it, but we should talk about it some time, are you at JuliaCon next year?

Its interesting to me that the climate/resource benefits are often roughly aligned with practical utility - like proposing a new algorithm that takes weeks to get a result is both practically bad for research and also bad for CO2 emissions, so there are win/win outcomes for different interests that make these benchmarks more likely to be used.

@yvikhlya we can 100% agree that a holistic approach to environmental problems is needed. But a holistic approach also needs a fair appraisal of the relative importance of the various components of the problem. It looks a little suspicious when in practice it downplays a major part of the whole.

3 Likes

Tying everything together, in my little corner of the Eastern USA, in the past 5 years they have bulldozed 100s (1000s?) of acres of forests within 15 miles of my house. Some went towards new electric power transmission, while across the street a Cloud HQ data center was built. A new data center is scheduled for installation by clearing another ~15 acres of forest near a childrenā€™s sports complex and park.

It would seem that the appetite for data and data processing is insatiable. I used to live in a nice forested area, now instead of having acres of CO2 absorbing trees, I have a bunch of buildings making CO2. Since some think CO2 isnā€™t a problem, what about the millions of gallons of water being used daily? Data centers are water hogs, and what happens to the waste water? I hope its not contaminating my well-waterā€¦ Not to mention all the increased traffic and all the non-CO2 pollution (carbon monoxide, unburned hydrocarbons, etc.). And also not to mention the plant/animal habitat loss necessary for building these data centers.

So to the OP, yes, it is absolutely crucial to consider energy use with AI, or crytpto, or any computing that takes enormous resources.

And as the saying goes @yvikhlya we ā€œcannot let good be the enemy of perfect.ā€ Will reducing CO2 save every problem? Of course not, but if Google+Amazon+Microsoft all agreed to cut their energy use for the sole purpose of reducing emissions, it would ABSOLUTELY have a wider impact. If they reduced their energy use, then maybe data centers wouldnā€™t be built near my house, that would stop the deforestation in my area, then the water in our local aquifer would be left for drinking instead of cooling servers, I and my kids wouldnā€™t be breathing in quite as much smog.

So yeah, itā€™s easy to dismiss ā€œcut energy consumptionā€ as just about CO2 emission, but there are myriad other benefits.

3 Likes

Good idea. Here is a private opinion: CO2 emissions alone is bad criteria.

I want to be there! Mostly a matter of finding funding to get myself out there ā€“ I think that would be an excellent occasion for a chat and hacking session! The good news is is that we over in JuliaHealth land have a full end-to-end sample pipeline that we could use to prototype all the complexities of this pipeline.

Exactly, this is something that Iā€™ve run into and want to be addressing. Itā€™s tough finding the right balance of not getting in the way of scientists wanting to do science and also being conscious that the tools we use may not be the best environmentally speaking.

1 Like

Again, ā€œenergy consumptionā€ and ā€œCO2 emissionsā€ are related, but different problems, which should not substitute each other. Tell big corporations to cut emissions and they will, but the cost may be much more harmful for environment than CO2.