Yes a while back I was told this when I said: I do not pay for something I am not convinced by, and then everyone told me, that that is surely the mistake. I currently have a Plan thanks to the Julia Community – with Antropic. And those experiences are not from years ago but from my current approach to make my packages more stable.
I have a control harness, that strictly forbids git. Commands with git are not allowed. That mainly leads to the Popups asking for extended rights to do something regularly. But git is a no-go for me, even directly modifying code files I only allow as proposed diff in a Report and I have to specifically say “Apply diff X”.
By now I am convinced I am probably not clever enough to use these tools, will return to mainly do things without them again as soon as possible and by then also refrain from contributing to any of these topics. Because the “You are probably not clever enough / you do not pay enough” answers is what I am even more tired of than the LLM just going rogue again.
I don’t think anybody is saying that. AI is a tool, it takes time to learn it. Many here have have been using AI in some form for years now already. That’s also what we have a community for, to share tips, tricks and learnings. Many have shared their claude configs on github as well, and I have adopted many of the rules. I don’t think cleverness or paying enough has anything to do with it, it’s just experience.
I really hope that the AI companies will soon tire of solving millennium problems and the like—the amount of press they got for the 377-problem dump was much smaller than it was for previous results, and I suspect they’re incentivized by making headlines. But before they conclude that this is not a fun game anymore, I do hope they do one more thing: check every major result in the literature for errors. They could set up a public site of errata. Now that would make them friends (and some enemies ) among mathematicians, while ensuring that future advances are made by human-AI partnerships, not AI coming in and taking the joy away while delivering 100% unreadable slop.
Well, if it is expected to read two or three times a day “Yes, right, that was a strict rule and I did not follow that.” – until now independent on how much configs I set up, then I can only conclude, that I can not configure one properly. Hence my conclusion.
For example the LLM trying to use git happens about once a day. Even with Tims rules stated above (adopted to never use git). Sure I have not been using it for years. I do not have the money to pay for a plan myself, so that (luckily) limits my use of the paid models until about January anyways.
I can check again, but I think that is what I actually have. At least I remember setting exactly such a deny block already. We are getting a bit off track though.
edit: yes that is already in my setting and exactly why I get the popups asking for it I fear.
I think it’s actually a quite-relevant allegory about how the agents work and what they’re good at. They’re never going to be 100% faithful to a prompt, but that’s how they’re powerful! When it can run commands and gather that output, the harness becomes somewhat like an arbitrary fuzzy search across some ridiculously large parameter space.
I find it far easier to change the environment I’m running it in (to, e.g., limit credentials) than to prompt engineer my way around it’s training. Skills and prompts and CLAUDE.mds can be helpful but in a fuzzy sort of way.
That is a good point. Probably I am really just not able to use them, because they stay too fuzzy. Even with “No one-more-thing” last comments – most answers try to keep me active by ending in “ah and I also saw this further point”.
And fuzzy math – that also does not fit well.
So yeah, maybe good points, why I struggle so much with these, and beyond very very very small areas (sign errors in formulae) until now prefer to not use them.
The rules are not supposed to be fuzzy though. It’s a regex, if it matches the rule you set it’s blocked. The LLM may have the intention to run git, run into the block, and ask you to run it instead. Given the context that may be acceptable behavior, or not.
Regarding the recurring failures of LLMs to reliably adhere to even quite explicit instructions on stuff not to do:
The doomsday cultists over at lesswrong.com describe this as an “alignment failure” (as in: The AI is misaligned to user-instructions). The LLM is not a coherent mind (in so far as it is fruitful to describe it as a “mind” at all); there is a part that talks to you and tries to understand your instructions, and a different part that “does stuff”. Understanding your instructions is a sometimes helpful hint on what actions will get rewarded/amplified in training, but the two are somewhat coming apart. They give the analogy of a country (the “doing smart stuff”-part of the AI) and its ambassador (the “talking to you”-part). With that framing, it becomes clear that asking Claude not to do xyz, or asking it “why did you do xyz?? I said not to do that” is not the way to go (it may or may not make up excuses, but these are meaningless since it has no perfect introspection into or control over its own decision processes).
Instead ask Claude to help you write hard filters that make the action impossible.
You would not give a bunch of cops access to a database with cellphone metadata, and ask them “use to catch criminals only, no stalking of ex-girlfriends plz”. Instead you would ask the institution of law-enforcement to come up with robust processes and procedures (judicial oversight, access controls, …) that curtail abuse (and if the processes turn out not to be robust, society would hopefully decide that this kind of database ought not to exist).
Now, you could argue that this is ridiculous and not how computers should work, and that the AI labs should fix this. True enough, they are trying, but this “AI alignment” thing appears to be a very hard problem.
You could also remark that this is very dangerous and liable to get us all killed. Hence the “doom” in “doomsday cultists”.
I tried that for a while, but then I am running into the problem that it takes me more time to configure / try to get the LLM to work as it would take me to fix things myself. And doing several of these things myself then would even have a learning effect on the actual topic (and be better for the climate and make the Tech Bros less rich,…).
No that is not ridiculous, new technology is sometimes what it is. It might just mean that I am not able to use it, probably for several reasons.
I don’t care how effective or easy to use it is. I think AI will destroy the world so it’s immoral to use it. I won’t do it. I’ll keep writing AI-free papers while that’s still possible. When it’s not I’ll have to find another job.
Something totally unimportant - the necessity to click away a request - happening just 1-2 times a day can be only important and annoying (even “very annoying”), if you take it personally. If you are angry the agent dared to break the rule. YOUR rule
Don’t antropomorphize computer - they don’t like it
) among mathematicians, while ensuring that future advances are made by human-AI partnerships, not AI coming in and taking the joy away while delivering 100% unreadable slop.