when we first met he wanted to start dating immediately and I thought it was way too fast
Dating is a way of getting to know someone to see if they are a perfect partner so I don't see why you think it is too fast to or too soon to start dating someone...seems to me right from the start you were on edge/uneasy/suspicious or had your doubts about this man and with your constant arguing probably over trivial things you've proved this to him without doubt!
It also sounds to me that he has pick up on this very fast and has made the choice to end the relationship. You won't change because you started out this way for whatever reason so he knows it will be pointless to carry on.
Next time around with any man try not to bring previous relationships/hangups into new ones. Trust each person until you get to know them for who they are.