Uganda is not a terribly touristy country. Jinja is one of the only places that sells 'African souvenirs'. A day of shopping is best started by The Source Café and the surrounding streets. It is possible to buy drums, Ugandan guitars, paintings, and many other things ideal as gifts. Remember that -- apart from a few locations in Kampala -- you are unlikely to find shops like this elsewhere in Uganda.
Tourist shops also sprung up around the NRE camp in Bujagali, but they'll be a little more expensive than in Jinja.
