5 stars and 10 points for advice on final decision?

It is okay to say NO if that is really what you want to do. However, unless what "came up" is extremely important, since you may have led him on and not made it super clear about you being undecided, then I think you should go. If he is a friend and you want a clear conscious about this, do it this time and learn from this experience. We all get trapped from time to time, so do not worry about it, do what you think is right. Enjoy the day, whatever you do.