Help with Taurus Man

I'm new to this so I'm just gonna place some signs in my birth chart and the taurus man Im talking about. I have sun in Aries, rising in Libra, venus in Aries, and virgo mars. His is sun in Taurus, rising in Scorpio, venus in Gemini, and mars in Taurus.

I have no idea where I stand with this man. I've known him for 2 years at the most. My best friends boyfriend is really close with the guy so whenever they would all hang out they'd bring me along and thats how I've been seeing him a lot. This past year whenever we were all hanging he was super sweet, always next or around me, and hed always mess with me playfully. My best friend told me he liked me cause he's always asking for me whenever he hangs out with them or mentions me when Im not around. I took the initiative and started to message him first our conversations are great but then he'd stop replying? The only time he messages first is whenever he wanted to hang out. Ive spent time with him but not alone we always had a friend around and we'd have a great time. Recently I had the house to myself and he somehow found out and asked me if I wanted to hang out I said yeah sure why not. So that night when he came over we smoked, watched tv, played games, ordered pizza, smoke some more, and also drank. It was only just us two in the house this time. We were having fun nothing intimate happened. Some of my friends would roll through to hang for a bit then leave. Once it started to get late I decided to move the wii to my room so we could smoke and play games in there also watch movies because I was getting tired and wanted to lay down. So we went in there played a few games. He mentioned sleeping on the couch but I felt bad so I offered him my bed. I put on a movie and we were watching it then he started tickling me and playing with me. I hugged him and asked if he liked me? He said yes and he asked me if I liked him I said yes. We kissed. Then we were kissing and making out. Before anything happened I told him I didnt want to be moving too fast he said we werent doing anything and we continued to make out. But lies! We ended up having sex but before we did I asked him if hes sure? He said whats the matter? As long as we like each other it shouldnt be a problem. So i trusted him. Afterwards he was caressing me and we were cuddling. The next morning we kissed, cuddled and ended up doing it again. Before he left my house we kissed and he told me hed see me later. Since we planned inviting my best friend and her bf. That night happened but he didnt spend the night. I didnt know how to act infront of his friends also since they were there. It wasnt awkward. If anything I felt more comfortable with him. ANYWAYS now that im done with the backstory. Lol. This is my actual question.

The other day I messaged him if we could talk about what happened when he spent the night. He said alright. So I basically asked if he meant what he said that night since he was high or drunk maybe both? He said "no i mean i do like you although i was pretty faded.dont trip though i think youre cool but i dont want to be in a relationship atm." well i totally understand cause i feel like i dont even know him well enough to jump into a relationship like that what was he thinking. I just messaged back saying well im glad were on the same page. But i also dont think we should be doing anything like we did that night. He said i feel you. I said okay thats all i had to say? he said do you hate me? I told him how i feel and said i like you but i just wish we didny do anything that night but you probably hate me though and all he said was nah lol were good. And i just ended it at that. Im stuck and Idk what to do. Im thinking of just dropping him. Does anyone know what this taurus man is thinking?! Im not sure what to do in this situation to make it better or else Im just gonna have to stop talking to him.