I had my badge pulled on June 1 for message response time. I pulled the log and it was 1 message that counted for 8%. The problem is I answered the message within minutes.
The message came through a day after dealing with a customer on an item that USPS had destroyed. I facilitated their return and refund and the next day a redundant message related to this came through. I responded to it within minutes and reiterated the steps that were taken for the customer. It seems to have been hung up on Etsy's end for a day before it came through.
Having auto replies on woulf not have prevented this as it would have answered the message when it came through, as I did.
Etsy wants to "Keep commerce human" but is willing to let this system be cut and dry with no recourse for mistakes or errors on their end. Run it with accurate and correct information...
In addition I am now showing 97% on ship time due to one package that was shipped on time and USPS tracking verifies that is the case.
It is ridiculous to have to wait until the first of next month fpr this program to recalculate and get its act together.
You can download the CSV file to see which message that they're saying that you didn't respond to in time. Then find the message and mark it as spam, your score will go up to 100% within 24 hours or so.