Neutral is just that..... neutral. It isn't positive, but it doesn't reflect as poorly as a negative either.
Personally, I have only left neutral once, and left an explanation as to why I chose neutral. If you leave a comment explaining it, then at least they understand what they could have done differently (in this case, making sure they understand the listing and ask any questions before purchasing).
I don't really believe in leaving negative for a customer, and neutral is kind of pushing it for me. I mean, as the customer, their only real job is to make sure they've paid and asked any questions they need to - and hopefully be polite in the process.
Though I do understand that, YES, there are some customers out there who can be difficult to work with. I haven't had any yet that have been tooooo tough, but in those cases I probably wouldn't leave any feedback.
Basically, what I'm getting at is, keep in mind how the feedback you leave may reflect upon yourself. As long as you keep that in mind, and are reasonable about it, leave whatever you feel is right. :)
What I've said is just my own opinion/feelings toward it in how I run my shop - it's totally different for everyone. :)