Obtain the from an Arcane Container inside The Steamvault at Coilfang Reservoir and the from an Arcane Container inside Arcatraz at Tempest Keep. Return to Khadgar in Shattrath City after you've completed this task.
Strategy and Tips
This quest is now harder to solo (as of Patch 2.0.7), through Stealth or Invisibility, due to the container no longer containing anything - it triggers an event which spawns an elite guardian which must be killed in order to obtain the fragment. The Guardian only drops the number of keys needed by the group. Upon leaving the instance, your copy of the key is deleted. This prevents you ungrouping and allowing others to join and loot the key. It should also be noted that if you spawn the Guardian Mob, and consequently wipe, then it is likely that you may not be able to loot the key fragment.
If you bring a party to obtain the key fragment and can defeat the Bog Overlords, getting to the key is a pretty quick process. Kill the two Bog Lords on the right side as you enter and skirt that edge. Following the Bog Lords, there is a pull of a 4-group of Naga, the 2-group at the front edge of the ramp (watch for the patrollers and pull them back) and then you can jump on to the "ramp" of rock-geometry just out of aggro range for the four group at the base of the ramp. Jump in the water and then get your key!
It is possible to get the fragment with 2 players and at least 2 each. This method requires the first group to be pulled and a melee/spellcaster patrol to be pulled. After the two groups are dead, careful timing is required where each party member takes turns to run into the room and drink the potion the moment before they may pull aggro. Making the run seperately can prevent a full-party wipe. Once Invisible, run straight to the pond and aim for the second column to hide on the backside directly against it, away from the "ramp". When the potion wears off, you should be standing evenly between a large Naga group and two Mermidon guards next to the ramp. Wait for your 10 minute cooldown on the potions to wear off. Afterward, drink and proceed to the pond. Priests can use their Mind Vision spell to locate the "rest stop" and guide party members. Avoid Bog Overlords at all costs, as they are able to detect you while Invisible. s
The Third Key Fragment is a little more of a challenge. You first of all need to enter Arcatraz (which requires at least one party member with a lockpicking skill of 350, or the obtained from starting the  quest-chain) and work your way to the first boss, Zereketh the Unbound. If your group is only interested in obtaining the Third Key Fragment then you can skip Zereketh the Unbound and go to the stair case, which is located directly in front of you as you enter Zereketh the Unbound's room.
Upon arrival at the top of the staircase you'll need to remove a few guard mobs, until you reach the first room - the Third Key Fragment is located in the corner, on the right as you enter the room.
Much like the Second Key Fragment, this too is now a little harder to solo through Stealth or Invisibility - as looting the container for the Third Key Fragment now spawns an elite Guardian mob that must be killed in order to obtain the fragment.
I hid the second fragment at the bottom of Serpent Lake, long before it was called that. The giant drain installed there after the naga took over has likely pulled the container into the depths of Coilfang Reservoir.
Bring me these remaining fragments, <Name>. Much depends on your success.
How goes your search for the key fragments, <name>?
You have the fragments. This is most impressive, <name>. You're certainly worthy of holding the key to Karazhan.
A small problem remains.
Karazhan quest progression
This is part of the Karazhan Quest Progression quest chain.