Help please!

I have a complex situation. I met a Taurus through work, I am 36 and he is 26-- I'm a cancer. It's was instant between us. We flirted a little bit and then it was an onslaught of passion and feelings. He had just been broken up with and I going through a separation. Because of his breakup he had decided to do a traveling job so we were only able to have a few weeks. From that point on we talked effortlessly day and night. I went out to see him and it was amazing. After I left I kinda freaked out about us doing this because we are going through tough times. It turned into a pretty nasty argument on both sides. A week or so passed and I texted him because I was late and I was scared. He was supportive and caring about it. We both decided that maybe it's better to remain friends though I did say I know myself well enough that I know I'll continue having feelings. We texted pretty regularly, he told me he was sleeping with some one and I became upset and told him I still have feelings and that I don't want to talk about that stuff with him. He backed off texting me for about a week though we still texted occasionally. He wanted to FaceTime but I wasn't available. I'm going back out because of school and he says he wants to see me. He told me today that he's been hanging out with a girl and is considering staying there though not for her. He made special plans for when I come out, we are going up into the mountains for a special trip and plan on hanging out all day. He keeps saying that he can't wait to hang out and drink beers and even tossed in the "bro" word. I know that I should let him go, but I also know he still has feelings for me (he said it a few weeks ago). I am absolutely in love with him. I'm a patient person and I'm giving it until I see him to decide which direction I should go in. Is there any hope with this? Everything I hear is that if a Taurus is over you then he withdrawals completely especially after what we have been through. Should I just give up?