I find the following website fairly organised and, from what I can tell, satisfactorily accurate:
Who To Tip: Tipping Around The World
Good service in Japan is taken as a standard in Japan. There is a difference between phenomenal service and good service, but the Japanese are cultured to be hospitable.
A "tip" offer in Japan is sometimes seen as a bribe, or an indication that someone needs to be "encouraged" or paid more in order to do a satisfactory job. Quite the opposite of the "intent" of tips in tip-heavy cultures. This is why it is perceived as rude. The "not smiling" staff at the end of your experience is likely because they had stopped trying to smile and perhaps exhibiting a disposition of personal guilt or shame. It is not inconceivable that if their employer saw them accepting a tip (unwittingly or otherwise), they could lose their job.
Having grown up in Australia (and especially as a child in an era where tipping was certainly not normal, unlike now where it's not necessarily normal but certainly a social discretionary trend), it's difficult to get one's head around tipping and the nuances of it. Even though I've been to the USA a handful of times now, I still feel like a fish out of water when I need to tip (as well as feeling a bit awkward as I try to find a way to break lots of $1 and $5 notes to tip with once I arrive in the USA). Not only that, but growing up in a non-tipping culture can make you very confused and defensive about what all this tipping business is about (why do we need to do it in the first place).
But as I say, when you go to another country, it's quite important to try and familiarise one's self with the social and cultural (and business, if necessary) norms.