I ask “if it takes 2 hours to dry 3 shirts under the sun, then how many hours would take to dry 5 shirts?” And AIs answer batshit crazy stuff. Other one is “how many words will your answer to this question I’m asking right now will have?”. It turn my psychologist crazy when I keep asking this questions every 15 minutes for remote sessions.
I agree partly with you but…
If the wiretapper releases that conversation and it appears on every TV in the World then it is public and the first thing you are going to say is “but that was a private conversation in a private room and was not meant to be public”. There is expectation of privacy. There is none in a DM. It is a direct message/contact between you, other person and potentially 99999 auditors, and the rest of the world. This is by design, not an exceptional situation.
Well, not if you preface the prompt with some bullshit roleplay thing. There are a lot of not censored AIs, but the ones that are censored only answer the generic “I’m an ai” because you are asking a direct question and not telling it to pretend to be a fictional person and answer accordingly.