I do for ONE day if I'm going to be doing a craft fair, because you never know when you might sell something and my stuff is all fairly unique and one-of-a-kind.
For five days I would too, because my shipping policy is 'usually within 2 days, unless it's over a weekend or holiday'.
If you can put a note in saying people can order, but only if they're willing to wait for shipping until you get back, I'd say it's fine to leave it open. I don't think I'd do it for more than five days though.