Scorpio Female living with Male Taurus Roommate

I am a Scorpio lady living in a house with another female and a Taurus guy.

To make story short. I met Taurus guy online a year ago, saw each other for a few months and then we kinda stopped.. but still remained friends.

A room became available in his house 3 months ago, at the time it worked in my favor to take offer and rent from him.

Other female roommate I have perviously met when I dated him. They have been close friends since public school. They have a platonic relationship. (I know this cause they have same circle of friends and they are here every weekend)

ok, so..

3 months here so far, at the beginning we agreed nothing would happen for I was seeing someone when I moved in.

Agreement failed. I broke things off with other guy.. not to try and be with Taurus, but because things weren't serious with the other guy anyway and I certainly wouldn't sleep with both!

So we have slept together 3 times..
He is affectionate towards me sometimes, but says very mean things other times (in his brutally honest ways with the apology afterwards lol)
He says we would never be anything more, but he gets jealous in weird ways..
He said we had sex cause it was available to him.
He is not a man whore, and does not go out, or bring chicks over. If he is not at work he is playing video games/watching movies/shows.
He is kind and generous.
We do have many laughs and great moments together.

I don't take what is happening seriously, cause he did say we would never be anything more. So it is interfering with my social life of potentially finding a romantic partner. But sometimes when he throws his jealousy at me it says other wise, and the nice things he does for me. I sense he is holding back or something.

Mine:
November 4th
Sun Scorpio
Moon Cancer
Venus Sagittarius
Mars Scorpio
Mercury Scorpio

His:
May 15th
Sun Taurus
Moon Taurus
Venus Gemini
Mars Aquarius
Mercury Taurus

What do you think is happening?

I meant to say I try and not take this seriously, but yes it seems I may be.. lol

wow that was well said, thank you so much!