If it's a received pm, you can leave the conversation and refuse to accept future/new messages in the conversation. There's a box to check. I'll PM you and we'll get it straight,
After you said you were trying it now, I sent a reply. If you checked the "Ignore future messages" box, you should not have gotten a reply alert. ETA: That doesn't permanently delete a message sent by you to someone else unless they permanently leave the conversation too.