I don't know how to take you?

Can someone please shed some light on this for me?

I have been told more than once in the last few months that "I don't know how to take you"?

I find this interesting because I will be the first one to tell you what I think about something if you ask me and yeah sometimes I will tell you even if you don't ask me.

I joke at work like everyone else, however, it seems like when I joke I am taken very seriously and at times gone back to the person and said "hey was only kidding".

I have heard this more than once from the virguy I am good friends with and he is as "frank" about things as I am.

maybe you're too stoic in your expressions (personally I like this when humour is used) ,or too dry for people to pick up on—

Maybe that is it>>> I am pretty stoic when I joke and there is usually no inflection in my statement or reply. My intention is not to come across intimidating.