Customs fees are a real pain. I've only had to pay one so far, and I knew nothing about it until the card arrived through my door. What irks me the most is the handling fee on top of the customs fee. Still, I quickly discovered that it is always the CUSTOMER'S responsibility, so I swallowed my irritation and payed the fee. It was certainly worth it, either way, because the item was beautiful and something I'd been after for quite some time. It's not made me wary about purchasing from overseas, though, I just make sure that I can afford the item atleast twice over, just in case.
While I'm sure it was appreciated, you shouldn't have refunded any money. The customs fee is there for legal reasons, and you've just lost out on a sale as far as finances go.