Go for it.
If you learn the kanji in grade order you'll be learning a lot of kanji like:
1st grade- grass, insect, bamboo, string/thread.
2nd grade- fur/hair, boulder, bow (for shooting arrows), sword.
I rarely read or write about these things. Sure you might encounter some of these words and you might want to use their kanji. but maybe you'd prefer to discuss history (grade 4), English ( a grade 4 and a grade 2), North Korea (has one grade 8), S. Korea (un-graded), going to the onsen (sen is grade 6). etc.
I just find that the vocabulary at the early levels isn't all that meaningful for me. Though definitely you want to learn the thread kanji and the bamboo kanji, since they're both radicals. I don't know of any textbooks that teach the kanji in grade order.
If you're younger than h.s. you're more likely to discuss bugs and rocks than nuclear proliferation in Korea. Though, I did have a class of 5th and 6th graders asking me where Kitachosen was on the map. (I'd forgotten the word for N. Korea, so they changed the sentence to the place where the bad people with bombs live).
But, you get the point?