Is he really just forgetting?

I feel like my boyfriend isn't giving me the full truth. In the beginning of our relationship he was messaging one of his high school ex-girlfriends on myspace (he is 23, so that was a while back) and asked if he could take her out on a date. She gave him her number and then after that is part of the questions. A few months after I found out, I recognized the number when he got a text out of the blue. I asked how she got his number, after he had sworn up and down that he had never contacted her after that, and he said he must have forgotten talking to her. Now, that is the tip of ice burg. Any time that I have a question about how faithful he is to me, he always answers my questions with an "I don't remember". Now, he has had seizures in his past which have taken a toll on his mental capacity, but he only doesn't remember things that are convenient for him to forget. He has hidden things from me before, like his rifle getting stolen from our place to "save my feelings". Now with him bringing up marriage and children, he is sending me so many mixed messages... I just want to get an outside perspective on this.