Oh, I think I might have met the old woman (or man, depending on char) when I was younger, but I can't remember it too well.

That sort of thing basically stops the conversation going any further into your new character's knowledge of the old one.