Guillaume Rossolini@GuillaumeRossolini@infosec.exchange
@lerg@infosec.exchange @jerry@infosec.exchange re: the LLM deleted files
There are many ways an LLM might “understand” a prompt. We know it isn’t repeatable in any exact way. We know there is some amount of randomness involved.
The LLM might assess that the non existent target folder evaluates to empty, so the operation becomes “move the files to nothing” 🤐
Jerry 🦙
💝
🦙@jerry@infosec.exchange
@GuillaumeRossolini@infosec.exchange @lerg@infosec.exchange true enough. In this instance, I think the issue stemmed from the fact that it shows an invalid command to create the directory. The person asking for help didn’t think to ask it to ensure the directory creation completed successfully before moving the files, and therefore it went off the rails. I think it’s pretty easy to replicate things like this, I’ve had ChatGPT help me with plenty of shell scripts and unless I’m really clear on everything it needs to do. I’ll end up with a lot of unhandled failure cases where I think we have a problem is that people see that it knows how to do 80% of the work 80% of the time and they make an assumption that it really knows how to do 100% of the work all of the time, and because of their own ignorance (not in the pejorative sense) of the subject matter, they don’t catch the problem. I think the exact error as a point out, won’t be very repeatable, but I do think that many different type are quite repeatable conditioned on the right level of ambiguity in the prompt. At least that’s my observation so far.