In the late game, the unloading and loading take more time than the frequency upon which you like to operate the line. Unloading and loading can take up more then 60 seconds. The frequency of the line is dropping because of that.
Secondly, the load and unload time are presumably not in the equation of the travel time simulator. The 20-minute rule is not taking in account all loading and unloading times lost during all stops between start and arrival. People jump off the train because they did not arrive in time at their destination. Or, they decided to take the wrong (slow) route because they did not see the time lost during all stops on the slow route.