Balaklava Naval Museum
The great views of the Black Sea from the remains of the Genoese fortress.
There is a World War Two German fortress further up in the hills by the sea: just follow the paths that lead up from the Genoese fortress, taking ones that lead upwards, and aim for the Orthodox cross that you can see on the top of one of the hills. There are various trenches, there's a cast-iron sentry's tower which overhangs the cliff, and lots of huge concrete fortifications which might be exciting to explore if you have a friend and several flashlights. No signage at all.
If you take a path that contours round the cliff beyond the Genoese fortress, you can get down to a small sandy beach.