Everything! Truly, the most impressive thing about Portofino is the lifestyle, so running off and "seeing the sights" probably won't be your number one priority when you plan your trip. There are some notable attractions that might be able to lure you from the beaches, but the view from the harbor or any of the surrounding beaches is unforgettable and one could spend the entire day marveling at it.
Castello Brown is a 16th century castle/fort, once used for the area's defense, but now primarily a museum with a fantastic view of the harbor and the Mediterranean Sea.
The church of St. Martin Divo Martino is around the corner from the harbor and is a quaint, stylish little chiesa from the 11th century. It's definitely worth taking a casual stroll around it.