Don't be afraid to let him experience the consequences. I may be reading more into your post but, from the sound of it, it seems like your question is, "how can I get him to pick up his toys so he won't miss the party and won't lose his toys?" The answer may be that that's exactly what needs to happen.
If you've set him up for success and you're certain you're not asking more than he can manage, then yes, he misses the party. Yes, his toys go in to a donation bag and get dropped off. That's okay. If he doesn't care,
then he has too many toys (most kids do, including mine).
I do let friends be disappointed if that's what needs to happen. We've been on the other end of that and I am understanding. "We can't meet you today after all, my kids won't do their chores so we have to stay home until they're done and we've past the point where we can leave and still make it."