not many free wi-fi spots in japan as far as i know...
a lot of cellphone providers sell wi-fi subscriptions, and you can also subscribe to services at the big electronics shops like Bic Camera. but it's pretty expensive (they charge by packet, so you'll be paying like ¥3000-¥6000 a month.
mcdonald's seems pretty consistent with free wi-fi.
pronto (a cafe chain) also has free wi-fi.
www.freespot.com lists some free spots around japan, but i've found that what is listed there isn't always accurate.
maybe we should all compile a list of places in the forum here, i bet it would be helpful for a lot of people.