Well, you do not need every factory to be working. I am not sure I agree that people need to go to work at the factory, I have seen nothing in game to suggest it is true at any rate. You need to have a station by the factory and a way to deliver goods, for sure. But that can be train track or road. NOTE: I will have to relook, the goods may need a road to get to the station.
You seem like you know what you are doing at any rate. Just embrace your micro management more. Constantly check the cities for goods delivered. With goods being delivered, and intra city network to move people, the cities will not stop growing!! It gets quite insane.
