He has the advantage over you, cheating boy. A lot of the cave kids are here, and he probably knows all of them. Dex, the eyepatch boy, no doubt they care for you, but you aren't really 'theirs'. I'm sure you don't want to completely go your own way, but having them be your priority when you aren't going to be theirs will just be painful, won't it?

So. Meet new people. Make new friends. Find a place that isn't right on top of his.