Short distances can be travelled either on foot, or by taxi-velo bicycle taxi. Taxi-velos are widespread, and are relatively inexpensive but not allowed in urban areas. A taxi-velo driver will cycle, and the passenger will sit rather precariously on the back.
Motorcycle taxis taxi-moto are also popular, especially in Kigali, a normal journey will cost up to $2. If you look like a foreigner and are walking on the main road, drivers will probably come up to you to offer a ride. Most of the drivers speak basic English or French, some do not.
Taxis are less common, and are best found at taxi stations, by waiting at the taxi sign at bus stops, or by calling them. They are significantly more expensive, even short rides cost 2000 francs, almost $4, and longer rides can be 5,000 or more almost $10.
Slightly longer distances, indeed the whole country, can be travelled by Matatu or Twegerane, literaly let's get closer. These white minibuses are found throughout East Africa, and are crammed full of adults, children, and anything else you can think of bags, chickens.