Sticking in my two cents here; I admit to being somewhat confused about this, and am very grateful to Drs. Fischer and Gupta for their clarifications concerning the significant vowel. I remember, on the occasion of Elizabeth II's visit to the US in 1976, seeing something in the newspaper in which a reporter (American) asked a British official how the Queen should be addressed, and was astounded upon receiving the answer which he heard as `Mom'. When he requested clarification, the Brit amplified: `M-A-apostrophe-A-M, "Ma'am"' [pronounced `Mom', apparently, in Standard British English].
On the other hand, I note that in the movie _The King's Speech_, at one point Queen Elizabeth (the person known to my generation as the Queen Mother, played by Helena Bonham Carter) explains to Mrs. Logue, `It's "Your Majesty" the first time [you address me]; after that, it's "Ma'am" -- that's "Ma'am" as in "ham", not "Mom" as in "palm"', meaning, it's supposed to rhyme with `ham', not `palm' -- and, the way Ms. Carter pronounces these words, the vowels are quite distinct (to an American ear). She may have been deliberately exaggerating (from the British point of view) the difference, perhaps precisely out of consideration for American viewers.
But yeah, basically, there are really two different words here, one (often spelled `mum' in a British-English context) being used as an affectionate form of address to one's own mother or, by extension, to some other woman whom one regards in a maternal light, and one (usually spelled `ma'am') used to address a woman in a position of authority or whom one wants to treat with respect. And, at least in Standard British English, these are pronounced *very much* alike; hence your confusion.