But it's not Tom's place to say whether Joe can stay or not, is it?
Whatever they think they've agreed, it's not their house and it's you and John who make the rules
Given the amount of headaches they've been causing you I'd be tempted to just say no, no discussions, you can go and have the meal out as planned, or friends here for a Chinese, but it's Tom's birthday not Joe's. Apart from anything else his friends being there as well would make for quite a different party, and Tom being responsible for him

ETA - and it does rather sound as though only Joe was aware of this 'agreement'
