LONG ANSWER: (see below for short answer)
The pilot trucks served to help guide the driving axles and keep the locomotive from derailing (especially on rough track and tight turns) and support the front end of the locomotive. Trailing trucks were usually added to designs to allow for a larger boiler and/or firebox. The exact number of pilot and trailing axles was determined by how much power the engine could produce and where the driving axles needed to be placed for the best traction. A well balanced locomotive design typically had the driving wheels near the middle, with enough pilot and trailing axles (if any) to support the remaining weight and keep the engine from derailing.
As I understand it, pilot trucks made it a little more difficult to cross the points on switches. Yard engines typically had no pilot or trailing axles. (Although some mainline engines were retired to yard service.)
Locomotive manufacturers were constantly trying to figure out how to make better locomotives. As technology improved, and railroads became more widespread and specialized, builders produced a wider variety of locomotives to fit their customer's specific needs. For example, Shays and Climaxes are really good at pulling heavy loads over poor track along steep grades; but 4-8-4 Northern and 2-8-2 Mikado locomotives are better for fast freight service on good mainline roads.
A general rule of thumb is that in order to make a steam locomotive more powerful, you have to give it a bigger boiler (or boilers). Technological advances like superheating and better firebox designs improved engine efficiency, but there was always a need for something bigger and better. Builders tended to create new designs out of older ones, establishing a logical progression. For example:
4-4-0 American - start here...
4-6-0 Ten Wheeler - bigger boiler, added extra drive axle to get more power
2-6-0 Mogul - Added extra drive axle to get better traction for steep grades and curves, took away extra pilot truck (engine wasn't much bigger overall)
2-8-0 Consolidation - Improvement over 0-8-0 with more power than typical 2-6-0 and 4-6-0 designs.
2-8-2 Mikado - Bigger firebox than 2-8-0, more powerful.
4-8-4 Northern - Huge engine with even more power than a 2-8-2.
...and so on.
(Manufacturers couldn't just jump to a larger engine size without figuring out first how to feed it enough fuel and keep enough steam coming to make the engine effective. Later technological improvements allowed for bigger engines...)
So a railroad might go to a manufacturer and say, "We need a locomotive than can do such-and-such, and nothing exists that can do it." The manufacturer might then build a new model based on an old design (extend the boiler, add a drive axle, add extra pilot wheels and trailing axles to support the weight) and sell it to the requesting railroad. The new design then usually got named after the first railroad to request/buy it. Other railroads usually bought a particular design, too, if it fulfilled a need. ATSF most likely bought a lot of Mikados because it fulfilled their specific needs for power vs. cost over certain parts of their system. Other railroads that ran over different terrain and/or had different practices for moving freight bought other designs that suited their purposes better.
And, one wheel configuration used by one railroad might differ from another railroad's due to modifications made to suit their specific needs. (A 2-8-0 owned by one railroad might differ slighly than a 2-8-0 used by another.)
Keep in mind, too, that long haul freight trains often had several locomotives involved with getting the train from point A to point B. For example, a train that left a city in the flatlands might be pulled by an engine that was good at running fast on relatively flat track. When it got to the mountains, an engine that was more appropriate for mountain travel might be coupled on or replace the other locomotive.
SHORT ANSWER:
It all came down to adhesion and how fast they wanted to move...
-Rory
(Sorry for the long reply!)
