Cafe Argo
Does quite nice georgian food pretty cheaply 50uah a head for two courses with beer; the menu is in russian only and an english translation would probably just give the equally-incomprehensible georgian names of the dishes, but the food was good, and locals will give you good advice if you appear baffled and ask them what to eat.
There are various places along the waterfront which will serve you coffee and cake during the day, and beer and small food at night
There is a quite expensive fish restaurant next to the 'Golden Symbol' yacht club. The food however, is excellent and the atmosphere is very welcoming, plus lovely views of the harbor.
The 'piratical pub', in a yacht club on the side of the bay with the submarine base, has cheap beer 5UAH for a half-litre and a range of reasonable pub food.