Will there be any need for this forum? (It is not about this forum. Read on.)

[harvestdeparture729]

@PetrKryslUCSD

Hmm, I am not a terrorist and I can think of at least 10 different ways of causing a global calamity employing autonomous agents. One: AI is in control of the data that the human operator of a nuclear weapon sees. Then, AI manipulates the data to convince the human that it is high time to push the button.

Your example is a different version of what I posted immediately above it.

Also, just curious, do you think you’re the first to think of nuclear weapon dangers in the context of social engineering, hacking, sabotage, etc.? “Humans are stupid and can’t be trusted with armageddon deployment” is the entire basis of the need for total nuclear disarmament. Unless there is something about the interaction of chatbots with specific nuclear weapons control/deployment systems in the military, there is also no real point to be made here. Yes, stupid humans should not control nuclear weapons, and the nuclear commanders should make sure to avoid phishing scams.

@rafael.guerra

This is certainly not the case for Geoffrey Hinton, the godfather of AI, whose opinion should carry some weight.

If you think Hinton “quit” and does constant press as well as speeches, lectures, etc. for free as a Nobel laureate, I have a bridge to sell you.

In addition, he literally has stated he thinks ChatGPT is conscious at the start of 2025 (https://www.youtube.com/watch?v=vxkBE23zDmQ). I do not even remotely care what he has to say on “AI safety” unless he’s talking about some material reality with evidence, not whatever fairies he’s made up in his head.

@PetrKryslUCSD

Another possibility to cause global havoc: Anthropic blocks possible attempt to use AI to make biological weapons

“Anthropic said a thing”, truly groundbreaking stenography by the BBC. By the same logic, search engines can cause global havoc, as information to do very dangerous things has long been available in certain places. Now, if AI has been trained on classified/restricted information that makes illegal information about weapons manufacturing accessible, we call that “a crime that should be prosecuted”. It has little to do with LLM’s, other than the fact they are lossy zip files capable of outputting large amounts of source material, including dangerous things.

Notice how we are nowhere near talking about how “there will be no need for humans anymore” and instead trying to retreat to specific cases of harm which could hypothetically be material, but for which we have no strong substantiating evidence of even potential harm at this time?

@kellertuer

But I am really tired of this, so maybe I stop here and if the forum here – as it seems – turns into only having these discussions why the should all Hail the AI gods and that is the only way to continue, then you are right, I am probably wrong here.

I am still confident that this forum some day can return to actually talk about using Julia and nice package developments again. And if not, then I am maybe just a boring small mathematician sitting in my office coding and writing small papers by myself without AI – and teach students math topics I like.

I really cannot agree more. Activity has dropped off drastically on this forum (Announcement frequency on Discourse over time - #19 by nilshg), and it seems like a plurality of it is just discussing AI.

I used to learn a lot reading on here, and now it is talking about AI god rendering humanity obsolete. The lack of skepticism towards companies trying to market their capabilities and “experts” relying on their credentials rather than any modicum of tangible evidence is just bewildering.