I usually walk around the walls before entering the city itself, so that I end up going through the Eastern Gate.
The first building through the Eastern Gate is a temple for the Church of the Divine Lucre. Kilborn the priest can sell you some Priest spells, but you can get the same spells for much cheaper from the temple in the Tower of Magi. There's another priest in Silvar, named Gardner, who wanders around outside. He can't teach you any spells, but he does provide healing.
The long building in the northeastern section of town is the Inn, which is run by Kelda. She'll talk to you about Colchis, and if you've completed the Slime Quest will be thankful that you've solved the slime problem.
Sitting at a table in the Inn is an Empire Dervish named Vorgar. You can antagonize him into a fight, which is actually a good idea, since he drops a decent suit of Plate Mail and a couple other items, along with some coins.
In the southwest corner of town is a house. If you haven't completed the Slime Quest you'll find some slimes in the bedroom, but if you have completed the quest they won't be there. There's a Lightning Spray scroll in the bedroom dresser, and a Bolt of Fire scroll on one of the bookshelves in the main living area.
The two buildings in the center of town are a Common Provisioner and a Common Weaponsmith. Just to the north of the Provisioner is Merry's Shop. Merry has many siblings who also operate shops across Valorim, and they're all named Merry. Apparently the parents weren't very imaginative.
The buildings near the gates are all guardhouses, and the one remaining building is two apartments. The one to the west doesn't have anything in it, and the one to the east contains a mother and child who will scream at you to get out if you try to enter.
