When you first enter the Golem Factory on the main floor, the first thing you'll have to do is fight the welcoming committee at the entrance.
Once you've killed all of these Golems you want to make your way to the southwest corner of the map where you'll find a set of stairs leading up to the catwalks that lead around the room. There's more Golems waiting for you at the stairs, so kill them and then make your way along the walkway to the middle of the northern wall, where there's yet more Golems waiting for you.
Take the stairs into the northeast quadrant of the map and move towards the center where you'll find a set of stairs leading down to Under Golem Factory. This will be the first of two trips to that level, so just follow the walkthrough for that section this time around.
Once you've finished your first trip to Under Golem Factory it's time to make a trip to the Upper Golem Factory.
Make your way back to the middle section of the northern wall, and then over to around the middle of the Northwest quadrant of the map. There's more Golems standing here waiting for you, guarding a room up on a platform. Kill the Golems and make your way to the door of this room. The key that you just got downstairs will unlock the door allowing you access to the Upper Golem Factory, for which the walkthrough is at the bottom of the page.
Once you're done with the Upper Golem Factory make your way back to the stairs leading down to Under Golem Factory and out the passage to the fourth Golem Spire.
Once the fourth Golem Spire is disabled make your way back up to the Upper Golem Factory, and from there to the Golem Control Spire at the very top of the Factory.
Once you've finished the Golem Control Spire you've completed the Golem Quest. Don't forget to return to Tevrono to talk to General Baziron, to the Drake Aerie to talk to Dalakros, and then return to Fort Emergence to talk to Berra, Anaximander and Levy. You can then go to the Tower of Magi to get your new spell from X.