I want to quote an Etsy admin here from the discussion forums, because I think it addresses a huge concerns for people:
"Exactly what recourse do shop owners have for being left unfair bad feedback?
Will there be some type of "review" process that will be implemented to protect the shop owners?
______
This is on a lot of minds! Reviewing buyers proved to not be an effective way of protection against unfair reviews. Buyer feedback scores were rarely checked, and most buyers we spoke to didn’t care about their “score”, and many didn’t even realize they were being reviewed.
Although we’re eliminating buyer reviews, Etsy will continue to take action on buyers who are in violation of our policies. In cases of non-payment we have our Seller Protection Policy, and if buyers show a record of non-payment issues, we close their accounts. Note that since we’ve implemented Direct Checkout, and riskier forms of payment have declined (such as checks and money orders), occurrences of non-payment have reduced significantly. In the case of a buyer using profane language, or disclosing personal information, the seller can flag a review for violating our policies. For “not-as-described” cases, we review the information provided in the original listing as well as the perspective of the buyer and seller, to resolve the case.
If we see buyers who show patterns in non-payment or in leaving violating reviews, their membership will be terminated. For not-as-described, the cases are dealt with on a case-by-case basis.
The question of “unfair” feedback is extremely subjective. Unlike the above areas (non-payment, policy violation, and not-as-described), Etsy can’t create standards around buyer satisfaction and their perception of quality, what they like, etc. This is really a human issue - there are some you just can’t please. Buyers get this, and if they see one lower review amongst a sea of high reviews, they will understand it’s not the norm. Keep in mind that low ratings will drop off your shop score after 12 months, so they don’t weigh as heavily as the past “negatives” did.
We added editing to reviews so that you can work with the buyer resolve the issue, as much as makes sense within your shop’s approach to customer service. This is much better than KAMU, is easier to find and clearer to use, so we believe resolving poor feedback will be a much smoother process going forward."