First the trees are logged and then the stumps removed to make a level grade for the steam locomotives. The steeper the grade, the more dangerous it would be for the locomotive and crew. In those days, locomotives had limited power for pulling freight and poor braking systems.
The construction crews consisted of hundreds of men. Remember, at the time (1860's), America was in the worst war in its history, the American Civil War. Most able-bodied men were off fighting for the North or South.
Some of the first men working on the new railroad line were the teamsters, men with a pair of mules or oxen, grading the line.