In 空恥ずかしい (sorahazukashii: feeling ashamed, embarrassed, or
bashful without knowing why, futile + shame), the “shame” kanji 恥 breaks down as “ear” (耳) + “heart” (心)!
In A Guide to Remembering Japanese Characters, Kenneth Henshall suggests that a shrinking heart represents shame, which just about sums up the experience of shame, I would say. Henshall also views the “ear” here as “acting phonetically to express shrink but of unclear semantic role.” That’s hard to follow, but I at least grasp that he’s referring to the sound of the kanji. In Chinese, the characters for both “shame” and “ear” originally had the same sound: jhi. Is that the sound of shame? Strikes me more as the sound of silence.