Hi everybody! Firstly, learning Japanese via JPod is BY FAR the best way I've ever found. I found it after listening to ChinesePod - for the record I found ChinesePod really good too
The question as to hardest foreign language depends on what your native language is.
My native language is English. I don't think English is particularly hard... native speakers often think it is, but all my European friends say it's the easiest foreign language they ever learnt. Probably different for native speakers of Asian languages...
I teach high school German and French in Melbourne, Australia. My second language is German (I lived in Germany as a teenager and went to school there), French is my third. I speak German waaaay better than French, but I actually think French is easier from a beginner's point of view. From experience, my students make more progress in German to begin with, but after about 6 months the French students overtake them and get more fluency a lot quicker.
I learnt Spanish at uni, found it easier than both French and German (in theory). It's my fourth language in actual ability to use though.
I have also learnt some Russian, Mandarin and Japanese. Russian is complex and I can't actually say very much even after three years of class, but I learnt it in the most old-fashioned, dry way possible. A classroom setting straight out the 1950s or maybe even crustier.
Mandarin I learnt first through conversation with Chinese-speaking friends. Then I later took evening classes, which were pretty lame from a language acquisition point of view, but I did learn to read a lot of characters. If I hadn't already learn some conversation though, I would have got basically zero from the classes. I didn't find it that hard, but I like to think I've got a good ear for tone
Definitely not as hard as people told me it was.
Japanese I find to be about the same level of difficulty as Mandarin. Japanese grammar is harder, but with Mandarin the different consonants and getting used to the tones is harder.
BUT
I have to mention that at the school where I teach, we get lots of students from overseas coming for their last two or three years of high school. The Japanese teachers both tell me that students from Korea find Japanese really, really easy in comparison to the anglophone Australian students. They seem to learn Japanese about four times as fast as the Aussies. They come as absolute beginners, go into a class with Australians who've been learning for three or four years already, and by the end of the year they're the best in the class
Apparently some of that is because a) the languages are somewhat similar and b) they already know some kanji because they mostly still learn hanja (the Korean version of Chinese characters) at school. But that's just what the Japanese teacher tells me... I don't know much about the Korean language so I can't really judge!!