Go on with your life as before you guys hooked up and stop waiting around for him period!
If he makes a plan with you, tell him that if he's still keen on whatever day it is he can call then and confirm. Sounds like he's lazy & boring if all he wants to do is hang out doing nothing... Do you want someone like that because he won't change!
The only good thing so far is that he actually talks and listens.
I think you should really assess whether you want to be with this guy or not...can you see yourself with him in 20 years and what sort of life can you see? Sometimes we can be blinded by what we think we want or having someone to love us so we feel more fulfilled.