At the peak of the hill there is a large building with a tower. The Olive Palace, as it's known, is the commercial centre for the Olive Gardens and lookout point. Even on a rainy day it has marvelous 360 degree views of the coast line and the surrounding olive farm. It's a great spot to check out if you're in need of a coffee and some local edibles (particularly if you've got a thing for olives).
The tower gives amazing views down into the valley and has a two tiered setup that gives 360 degrees of panoramic visibility out to the islands and settlements. Directly below the hill you can see the town hug its way along the coast's bays and hummocks, accompanied by forests and terrace farms.
The olive trees are everywhere on the hilltop, and the space has a Mediterranean feel. The garden makes for a nice wander with plenty of benches and a number of meandering pathways. It's a dry open space, I'd think of it more as a park than a garden, but regardless it's nice enough. Most of the environment around Ushimado is tropical forest or farming land, so the olive garden makes for an interesting change.
The café and shop on the hill have a selection of local produce, including olive oil, olive soap, crackers, dried fish and tourist goodies. The olive ice cream makes for an interesting treat (although I preferred the ice cream from Copio gelato just a bit down the hill). The shops and the adjoining olive farm are run by the local Nippon Olive organization which produces olive oils that go into a variety of commercial products. The olives in this farm aren't the type that you chew on, though, so don't expect jars of the fruit.
Relaxing with a coffee in front of the marvelous views is a delight in the well-kept café, as the seating stretches along a large windowed wall looking straight out over the city and Maejima. The windows can be opened, giving you an unobstructed breeze and the sweet wisps of olive trees fill the place. The café menu is sparse, including only coffee (ice, hot, and specialty types) and orange juice. The air conditioning keeps it cool on hot days whilst cold days can be quite comfortable.
There's outside seating on the grass amongst the trees and on a hot day the breeze and a juice can be quite refreshing, since this area gets steamy in the summer months. If you're looking for food the shop provides a whole variety of local produce, ranging from dried fish to crackers to ice cream. Occasionally a bus load of domestic tourists will stop by and the Olive Palace bursts into a flurry of activity as the shop, café and tower are flooded with people. Outside of these occasions the place is peaceful and is a nice spot to sit and relax in an interesting environment.