Fusible interfacing can give the bag structure. If you not want the bag to have any structure then you shouldn't use it.
How big is the purse that you are creating? Also, are you using a kisslock mechanism to close it or are you creating holes in it for your hands, or are you doing the fold over style clutch?
I made a clutch a few months ago and I used two kinds of interfacing to lend structure.