And a lot of the ones who don't feed on humans directly feed off of something from humans, usually their fear or magic or belief.

Honestly, what exactly most youkai feed on is left vague in Touhou canon. The Human Village, depending on interpretation, is anything from a bleak human farm to a luxurious human farm to a suspicious fortress patrolled by hunters to an ordinary town with wacky not-quite-human neighbors who occasionally stop by and cause mischief.