An interesting article from The Los Angeles Times on why overhearing cell phone conversations is annoying:
Cellphones have made phone conversations ubiquitous. But many people confess to feeling a bit startled, then irritated, when they hear speech, think someone is talking to them and then realize the person nearby is talking to someone else on the phone. It turns out that our brains just don't like this phenomenon. Researchers at Cornell University conducted a series of tests to gauge people's reactions when exposed to four background noise settings: silence, a monologue, a conversation between two people and half a conversation (called a halfalogue). The study participants were seated at computers and asked to perform various cognitive tests while exposed to one of the three sounds or silence.
The study showed that hearing the halfalogue was the only background noise that distracted the study participants and lowered their scores on the cognitive tests. For some reason, our brains are unable to tune out half a conversation. Researchers believe this is because we can't predict the speech pattern of a halfalogue the way we can with a monologue or two-way conversation -- making it harder to ignore.