Nickel-silver track (or any track) has a certain amount of resistance to it. The farther your locomotive is from the feeder wires, the more resistance the electricity has to overcome before it reaches the motor in the locomotive. So if you only have one set of feeder wires from the power pack to the track, your loco receives the least juice at the point farthest from the power--which is, I bet, where your locomotive runs the slowest. If your track pieces are only held together with rail joiners, rather than solder, this adds to the problem of resistance. If rail joiners are loose this can cause intermittent losses of power.
The solutions: First, add more track feeders at various points around the track, and make sure track is securely glued or nailed down. Second, solder your rail joints, at least in between the points where you have track feeders, to ensure good electrical connection.