I don't believe that they are showing up on the search on Etsy, but it is possible for them to show up elsewhere, like on Google search. Also, as Eva said, people tend to look at your sold products as well as your "currently for sale" stuff.
I do it as a buyer myself. When I'm shopping I like to get a "whole picture" of the store that I'm looking through, and see what they have offered in the past. For me it is curiosity.
But I know that I have had many people favorite sold items, and there has been several occasions when I have relisted another piece of something that I sold several weeks prior, and it sells within a week or two at most.
So definitely keep an eye on your sold favorites. If something seems to be popular, and you can create another one (and don't have any ethical issue with recreating it), then try listing a second one and see how it goes!