That's one big problem with tumblr; you can't. You can keep them from seeing your posts on their dashboard (the ignore feature) but people can visit your blog directly and follow you as they please. I've had a similar problem on my personal account and there really is nothing that can be done. For now at least. This has been an issue for years, since tumblr began, and it seems like they just don't want to address it. :/