I'm sorry I missed this earlier: James Lileks looks at 100 mystery films. An excerpt:
The ending of the movie is superb, and brings “Casablanca” to mind as well - but you’re waiting, waiting for Charles Boyer to say the line before the movie ends. Say it! Say it! Because, you see, the part of town where he lives is the Casbah. The line, which summed up mysterious alluring dangerous romance, was, of course, “Come with me to the Casbah."But just as you wait in “Casablanca” for Rick to say “Play it again, Sam,” you wait in vain for Boyer to say the famous phrase. Apparently it was in the trailers only, and that was enough to make it a catch phrase.