From Mohd at Customer Support:
Messages that are replied to are not counting as replied to, saying it never got a response and impacting message response rate.
I know this forum isn't going to do anything about it.
Go to Star Seller in your shop manager and download the spreadsheet saying which ones were not answered. There has been a bug the past few weeks where messages were going straight to spam.
Check the CSV file. Pay attention to message and response times.
I’ve also had ones dinged when I answered in the app rather than a browser. They just disappeared.
@BadwolfForge: As suggested, check the applicable CSV file available on your Star Seller page to confirm which new Message threads Etsy thinks were unanswered or answered late. When you look at the file remember that that the dates / times are in UTC (not your time zone), only the first message of a new incoming thread is included (all that counts for Star Seller), and usually a new incoming thread is not included until after the 24 hour reply window has passed regardless of when you replied. Star Seller stats (and the CSV file) seem to be updated only once per day so there may be an additional day's delay. There is also a link to the actual message thread so you can verify if the data is correct or not.
As an aside, You need to include the licensing information for all of the trademarked and copyrighted names and images in your shop in your listing, otherwise buyers (and Etsy) may conclude they are counterfeits. And a buyer could file a not as described case against you and both get their money back and keep the item. Including the licensing information will show that you are not selling those items illegally. 'Inspired By' is not an exemption to intellectual property infringement.
If you are answering via a message in the order that is not counted if it is in your message file..say they asked a question in your message file and on the order and you answered via the order..that will not count. Also if you answer on the refund page...not counted.