Start by searching outside the walls of the Portal Fortress. In the Southeast corner there's a Cursed Large Shield. You'd be best off just leaving it there. In the Northeast corner are some rats and a Stone Dagger. The Northwest corner has a secret passage. It's on the dark section of wall, just push against all the sections until you find it. The secret passage leads down to the Southwest. Whatever you do, DON'T PUSH THE BUTTON. Pushing the button results in instant death and the end of the game. There isn't anything in the chests there, either.
Now enter the Portal Fortress through the Northern Gate. If you to the west you'll enter a small area with four rooms. The first room to the north is empty. The first room to the south lets you rest.
The second room on the south has Carol in it. Carol sells you the First Aid skill up to Level 5. I personally find this to be a pretty useless skill, since I've never played without a Priest in my party, and Healing Potions work better anyway, but if you want to get it, this is the place. Carol will also heal your characters and uncurse cursed items. Uncursing items lets you unequip any cursed items you might have accidentally equipped, and it also lets you sell cursed items in your inventory.
The second room to the North has Denise in it. She'll give you a quest to get her amulet that she forgot in her room in the Tower of the Magi.
Go back into the courtyard and enter the building to the Northeast. Assuming you've entered from near the North Gate, the first room the north is the library. Check out the bookshelves, and one of them will have a scroll on it. Try to read the scroll; you won't be able to read it yet, but you need to have tried before you can learn how. The next room to the north has Walner in it. Walner is a sage and can identify your items for a small fee. Ask him about the library now. First mention the scroll you couldn't read, and he'll tell you how to read it. Also ask him about needing more books and he'll give you a quest to bring him any Research Tomes you across. You get a small amount of money and a small amount of experience for each book you bring him. Now go back to the library and read the scroll again to get Light Level 3.
Now go to the southwest corner of the building and you'll see a small platform and a lever. Go talk to Kelner in the southeast room and if your reputation is high enough he'll let you use it to augment one of your items. Talk to him again after you've used the platform and he'll reset it for you once.
Across the courtyard from this building is the storage area for the Portal Fortress. There's some decent items in some of the side rooms, but unfortunately you can't steal any of them, since the rooms are all guarded.
The last room leading off the main courtyard is the barracks for the Portal Fortress. There's a couple soldiers in here, and nothing much else. Now go into the Portal Chamber. Standing by the stairs up to the Portal is Seles. Once you've completed the first of the surface quests she'll let you use the portal to return to the Tower of Magi for some magic training.
The room leading off of the Portal Chamber to the southeast is a crystal shaping chamber. There's two storage rooms leading off here. The first one requires Unlock Doors Lv. 3 to get through, and once inside you need either Dispel Barrier Lv. 2 or a Piercing Crystal. There's an Unshackling Crystal and the Healing Spellshard guarded by two Guardians in there. Guardians are invisible, so it's best to go in fighting. They start in the corner beside the bench. One in the nook beside the bench itself, and one right beside the first. The second storage area only needs Unlock Doors Lv. 1, but you still need Dispel Barrier Lv. 2 or a Piercing Crystal. There's a Recall Crystal in here, guarded by one Guardian.
The only thing left to see in the Portal Fortress is the water reservoir in the far southeast corner.