I use Fimo, and have exactly the same problem. Black, white, and the metallic colours are very easy to work with. All the plain colours take a lot of work to get them maliable enough to work with.
I usually resort to mixing with a soft colour, or just tough it out until I get it soft enough. I really need to buy a pasta machine to do all this for me!