Assuming you arrived in Port Townsend by boat, the first person you'll see is Martha. Martha is an ogre who sells tickets for people wanting to travel by boat to Farport.
To the east of the docks is a common weaponsmith, and to the west is a storage room...with a guard.
The buildings along the western edge of town are all barracks. The Anama live in barracks, so there aren't many houses or apartments outside of Shayder.
If you follow the road to the north you'll find the eating or living area for Port Townsend to the west, and the Anama temple to the east. Father Rice runs the temple here in Port Townsend. If you ask him about his belief he'll ask you a yes or no question. You want to agree with him, and in fact you want to agree with all the Anama priests, particularly if you hope to join the Anama. Even if you don't, it's a good idea to humor them anyway, since it doesn't affect your ability to do anything else.
That's it for Port Townsend, so now head east along the road to Bavner. Before entering Bavner go a little bit further east and talk to the nephil sitting by the side of the road. He'll give you a quest to recover his necklace from the Innkeeper in Bavner, and gives you a Blessed Bow as a reward.