Thinking about dumping my Gemini

First off I'm a Sag, he's a Gem, we have amazing chemistry. We are both Aquarius moons so we have a close bond, and really love each other deeply because we were friends before we started dating. BUT, lately he's been pissing me off quite a lot. He's lazy within the relationship a lot of times, never really tells me how he feels about me even if I ask him to he says he has to be "in the mood to talk about it" and I don't ask for much honestly I'm not high maintenance. I just want a little bit of reassurance every so often, it's nice to hear how much they care about you, ya know. Also he plays video games a lot, and by a lot I mean almost all day every day. I honestly wouldn't care, as long as I get a little bit of undivided attention at some point, but he's just selfish and doesn't see it as important. You might think by all of this that he doesn't care about me or he's playing me, but I know for a fact that's not true, because he honestly needs me more than I need him, he has a lot of emotional issues and I'm the only person in his entire life he's fully opened up to, so when he's upset, he comes to me. Another thing is he says things that he shouldn't say to me, like backhanded complements almost, or something rude, which I usually understand that he didn't mean it that way, but I'm getting tired of it. I mean it's nice that we can talk about everything, but there's some things you should just keep to yourself. For instance earlier he got drunk, and I was driving us back home, I asked him in the car how drunk he was and he said it was wearing off, so being drunk shouldn't be an excuse. Anyway, we're laying in bed at home and he starts talking about these girls he remembers from when he was in like 2nd grade and they were in 8th and they thought he was a cute little kid so they would come up to him all the time and stuff. Then he was saying how he could probably talk to one of the teachers at his old school and try to figure out who they were. Then he said "what do you think they'd do if they saw me now?" And I said "idk they probably wouldn't know it was you" and then he said "do you think I could get some?" I was pretty pissed when he said this, I mean why the treetrunk would you think that's a good question to ask your girlfriend? I understand that guys think with their dicks, but it's called having a filter. His excuse was "it's all hypothetical, it's never going to actually happen" but no it gets worse. He then has the audacity to say he's been really horny all day and wanted to have sex when we got home, ofcourse I turn him down, and say he doesn't deserve it. Then he was like "are you really that mad?" I responded "I'm not mad, I'm just annoyed. If you really wanted to have sex then you should've been loving instead of rude." And then I tried to keep talking to him about it and he said he was tired and he was trying to go to sleep. Then I was even more annoyed because he wasn't too tired to try and have sex with me, but doesn't wanna talk to me. I've been contemplating breaking up with him for about a week but the past few days he had been doing a lot better being sweet and stuff, so I was like maybe I'm just overreacting a little bit. He has always been somewhat up and down with his affection/emotions I think it's the 2 different sides of him (Gemini), one being sweet and the other being standoffish and stupid. I've dealt with it for a long time and usually just get over things after a while but right now my patience is running very thin. One of my friends told me I should give him an ultimatum and say basically if he doesn't start treating me the way I deserve then things need to end. I don't know if this would work though. I mean I don't like being controlled and I know neither does he, but at this point just talking to him about it does nothing. He just acts like it shouldn't be a big deal, and then tries to make excuses. I don't want to give up on our relationship, I love him with all my heart, but I'm not going to let someone treat me like butter half the time, and only try when they feel like it. Don't know what to do...

My first question is, do he has a job? Second, I was with a Gemini man for years, and when they start asking questions like that,that means that they have got other women on their minds.Oh and they will cheat, given the right opportunity. So follow your heart. I personally don't think that A Sag and Gem can make it over the long haul... They are too fickle...