Chinese or Japanese, but I might be biased. Latin isn't easy, either. Let me outline my reasons:
Chinese
--Thousands of characters must be memorized (and must pick one of two kinds: traditional or simplified, both of which are still used depending on the country or place). I'm somewhat familiar with traditional. Not at all with simplified, even if simplified is more common now, with mainland China using it.
--Tonal language, has 4 (or 5, if including unstressed) tones. Each tone imparts a different meaning on the same basic sound. Quite a bit different from the use of tones in English. It's easy to mistake or miss the differences in tones, if one doesn't listen carefully. Both my father and my brother have a lot of trouble. I can hear them clearly, because I used to speak fluently at one point.
--Thankfully, grammar isn't too hard, but syntax matters, like in English.
--Can speak and understand some, but no longer fluent. I spoke it when I was young.
Japanese
--Thousands of characters must be memorized, and in addition, must learn two syllabaries (hiragana and katakana), and writing consists of a mixture of all three systems.
--Kind of atonal, so it makes it easier than Chinese in that respect...
--But grammar is harder, because verbs, differing forms of respect and address (for humility, politeness, close or distal relationships), etc. require more learning and mastering.
--My Japanese is poor at this point, but I hope it will get better with more study.
Latin
--Alphabet helps out a lot.
--Learning how to decline and modify things is a challenge, but also provides tremendous flexibility in expression and meaning. I've only learned a little about this, but it reminds me of French and Russian a lot, which I studied when I was younger.
--Dead language.
I'm not a linguist. Just like to learn languages when I have spare time.
Re: The Hardest Language 
