An exercise ball is an excellent chair, for your height you are probably looking at about $45.

Kneeling chairs are actually a lot better than you think. Instead of encouraging slouching (as you imply) they actually prevent slouching. They also don't do any knee damage (if my physiotherapist suggested one with my knees they have to be okay!!) There are different types out there, and the best is one with an angle and shape of pad that puts zero pressure on your knees (the ones my PT recommended to me have pads on the shins instead of knees).

I'm going to be replacing my exercise ball soon, so if you want to borrow it (even if it is a tad small for you) feel free!