Speaking as the mother of a highly allergic child (both wheat and nuts are deadly to her and she has a long list of other allergens as well), I would encourage you to clearly state what you use for filling in your listing, on your packaging, and on the label. Although I'd think it would be good to avoid the major allergens (wheat is one of the major 8), you can be certain that whatever it is, someone is likely to be allergic to it!! Some people are very allergic to rice. So the best practice is to accurately label things (and a permanent label about the filling on the product will protect you, too, in case an allergic baby chews on the product, you've protected yourself by providing fair warning to users).
I'd agree that buckwheat hulls are a safer bet allergen-wise than some other fillings.