Is your memory your own?

The other day I was talking to someone about brain transplants. We were discussing if someone, one day, could have a brain transplant, what would happen to their memories, and would they take on the other person's memories and knowledge as their own? For instance, if a man got a woman's brain, or an english speaker got the brain of a spanish speaker, if a child got the brain of an elderly person, etc. what would be transferred and what would be forgotten? Or perhaps would it be wiped clean with no recollection of anything including a transplant. Obviously, brain transplants are not possible with today's technology and medicine, but one day, it would be interesting to see how it turns out.