Wow!  Aren't we a bunch of insensitive provincials!  At this point 76% of us would apparently wish a Jew, Muslim, or a Buddhist a "Merry Christmas".  Race is not a determinant of religion so it is not possible to tell someone's religion or  (in the case of atheists) their lack of it, simply by looking at them.  If you know someone and you know they are Christian, then, by all means, wish them a Merry Christmas.  However, in our cosmopolitan and largely anonymous modern society one cannot blithely assume one is talking to a Christian.  With strangers or persons known not to be Christians or persons having a high probability of not being Christian (eg. their name is Moishe or Avram or Mohammad), go with "Happy Holiday" or if it seems appropriate ask them "Do you celebrate Christmas?" and depending on their answer wish them a Happy Diwali, or a Merry Christmas or a  Happy Chanukka or an enjoyable holiday season or whatever is most appropriate.  In that way, your wish for them will convey your sincerity  and mean more to them than an obviously impersonal, slightly imperious and ill-fitting "Merry Christmas" ever would.  

We are no longer a homogeneous rural society where everyone knows everyone else.  We are for the most part an urbanized and cosmopolitan society and we need to act with the requisite sophistication and sensitivity to keep that going.