If you cancel the order before you ship it, then there's no way to leave feedback. If you cancel the order, after it's been marked as shipped, it's still ''open' and the customer may be able to leave a review, depending on the timing.
Over the years I've gotten better at recognizing the warning signs of a no-win situation and have elected to cancel or refuse orders before they can become a bad situation. There are some (now former) customers whom I have flagged in my mind and will cancel should they make another purchase.
If there's a case of fraud against you, I imagine you could contact Etsy about that. One of the original intentions behind shops leaving feedback on customers was to let other shop owners know if a customer didn't pay, etc. That feature is no longer available through Etsy's feedback system.
