Fight or flight

I'm feeling a little confused and could use some guidance. A little history with my Libra man:

I was seeing him casually at a consistent basis for a month straight in June. I use casually loosely because we weren't seeing anyone else and I was over at his place and went out with him almost every day/night. I met his friends and his family knew about me. I thought things were going well. I went on a two week vacation to Greece and he barely contacted me during my vacation. To make a long story short, I came home and he was even more distant, treating me more like a buddy than an intimate partner. Granted, he had a lot going on with moving to a new place, some family issues, as well as some money issues. I finally demanded an explanation and he told me he was feeling it at that moment and didn't want to lead me on. So I left.

Cut to a couple months later, we started hanging out again. We didn't have a conversation about what had happened, basically just decided to start a clean slate. It's been almost two months of us hanging out and we're no longer just hanging out. We don't see each other every single day like we used to, but it's still consistent. He's been taking me out, bringing me around his friends and roommates even more frequently, checks up on me, etc. Just the other day, we spent a good 48 hours together, running errands and hanging out with friends. He makes no secret of us, he's affectionate with me in public. Last night, I accidentally called him "baby" during sex and when I left his place this morning to go to work, I called him "baby" again. It just slipped out. He didn't really react much to it, while I'm freaking out on the inside.

I'm afraid, I really am. Any slight difference in the way he holds me at night, I freak out on the inside. Any slight difference in the way he talks to me, I keep wanting to just run away. I feel myself opening up emotionally to him again and I'm having a difficult time with it. The last time we go this close, he left me. And I never got a definite answer as to why he bolted. Getting him to be vulnerable is like pulling teeth. We have our differences. I'm very much on track in what I wanna do in life, and he know where he wants to go in life. He's irresponsible, I'm very responsible. He's financially unstable, I'm staying afloat. So I don't know if maybe it's our differences that made him feel insecure or if I wasn't entertaining enough... At this point, does it seem pointless to bring it up? Will he leave again?

i would recommend getting away from asap. not because he is a bastard (and he probably is) but because it doesn't sound like you are strong enough to handle his superficial, temperamental ways. why try and get close to a guy who doesn't want to be close to you? why allow yourself to emotionally invest in someone who shows no reciprocity?

he keeps you around for now and when he becomes tired with you or a shiny new toy comes along, you will be left behind. don't accept this as good enough. it is not. you deserve better and certainly so much more than a guy who make you question your entertainment value.

don't wait for him to leave, you leave. find someone better.