Normalization refers to an Attack Power adjustment made to instant attacks in patch 1.8. Hunter ranged instant attacks were also normalized in patch 1.10.
Background
Before normalization occurred in patch 1.8, the damage of instant attacks included the weapon speed as a factor to attack power, thus greatly benefitting slow weapons with high swing times. Because of this, slow weapons with lower weapon dps could lead to a higher total damage per second than fast weapons with nominal higher weapon dps, which is counter-intuitive.
One notable example is the 
[Barman Shanker], which was at this time considered by many as the best rare dagger due to its exceptional high swing time, although there were other rare daggers with higher nominal weapon dps (or item level) available.
Formulas
The normalized weapon damage for instant attacks is:
- normalized_damage = base_weapon_damage + (X * Attack Power / 14)
where X is:
- 1.7 for daggers
- 2.4 for other one-handed weapons
- 3.3 for two-handed weapons
- 2.8 for ranged weapons
- weapon speed for non-normalized attacks (see below)
Prior to patch 1.8, the weapon damage used for instant attacks was:
- damage = base_weapon_damage + (weapon_speed * Attack Power / 14)
Note that ...
- base_weapon_damage + weapon_speed * Attack Power / 14
is simply the formula for weapon (white) hit damage, as 14 AP = 1 DPS. The nomalized formula treats your weapon as if its swing time was a certain normal value for its type (1.7 sec for daggers, 3.3 sec for two-handed weapons, etc).
Nearly all instant attacks based upon weapon damage are affected by normalization; only a few are not.
Examples of normalized instant attacks
...
Non-normalized instant attacks
- Druid
- All weapon damage based attacks, since druids in bear and cat forms have fixed weapon speeds.
- Rogue
- Shaman
- Warrior
[Cleave]- [Heroic Strike]
- [Retaliation]
- [Rend]
- [Slam] (technically, this is not always instant)
- Hunter
- All weapon damage based pet attacks, since each pet type has a fixed weapon speed.
Examples of instant attacks that aren't based on weapon damage and thus aren't normalized
- Rogue
- Warrior
- Hunter
- [Arcane Shot]
- [Concussive Shot]
- all stings
- [Counterattack]
Note: Druid feral attacks were not normalized. They are unaffected by weapon speed. Mages, warlocks, and priests have no instant melee or ranged abilities.
Tips
With normalization the weapon speed is now less influential on the total damage per second of characters, although weapon speed still has a small influence because the base weapon damage of instant attacks is higher for slow weapons of same weapon dps.
Nowadays weapons with higher weapon dps are generally preferred over ones with lower weapon dps (regardless of weapon speed), although there are exceptions like 
[Windfury Weapon].