Brutkey

Jerry πŸ¦™πŸ¦™πŸ’πŸ’πŸ¦™πŸ¦™
@jerry@infosec.exchange

@GuillaumeRossolini@infosec.exchange @lerg@infosec.exchange true enough. In this instance, I think the issue stemmed from the fact that it shows an invalid command to create the directory. The person asking for help didn’t think to ask it to ensure the directory creation completed successfully before moving the files, and therefore it went off the rails. I think it’s pretty easy to replicate things like this, I’ve had ChatGPT help me with plenty of shell scripts and unless I’m really clear on everything it needs to do. I’ll end up with a lot of unhandled failure cases where I think we have a problem is that people see that it knows how to do 80% of the work 80% of the time and they make an assumption that it really knows how to do 100% of the work all of the time, and because of their own ignorance (not in the pejorative sense) of the subject matter, they don’t catch the problem. I think the exact error as a point out, won’t be very repeatable, but I do think that many different type are quite repeatable conditioned on the right level of ambiguity in the prompt. At least that’s my observation so far.

Guillaume Rossolini
@GuillaumeRossolini@infosec.exchange

@jerry@infosec.exchange @lerg@infosec.exchange also, assuming this is bash or equivalent, these shells tend not to default to set -e if memory serves? Meaning β€œexit as soon as you get a failed command”: this isn’t the default.

Another flag I discovered recently is
set -o pipefail which has the same effect for piped commands, because that isn’t covered by -e for some reason.

So a failed directory creation would let the remainder of the script run, like you said, without extra checks (or these flags)


Guillaume Rossolini
@GuillaumeRossolini@infosec.exchange

@jerry@infosec.exchange @lerg@infosec.exchange still another failing of these tools, they aren’t really absorbing the boring tasks (the quality checks) that were hidden in the prompt