Where you calling out or were people calling in? I do not know much about cell phones, but I know you can have numbers forwarded and also 3way calling. So maybe he got ahold of your cellphone at sometime? Also with cloning of phones(probably retrotech and doesn't apply anymore), maybe he had access to all the calls you did?
I dunno, where are the phreakers?