I've been training in martial arts my whole life (four or five different ones, at various times, starting when I was 10, I'm 41 now) and have many many years experience teaching martial arts to children and adults.
Here's my opinion: four is WAY too young. They just don't have the physical coordination or the mental focus (or judgment). They are certainly not learning "self defense" of any value, though it's possible that they may pick up other helpful values and skills (but no different than any other structured physical activity). Anybody teaching 4-year-olds is basically babysitting, "playing" martial arts at best, having your kids do dangerous things at worst.
There's no particular harm in this kind of play (though I think it's a waste of money, since they won't retain anything useful), but I do have a rather serious thought: martial arts can be a wonderful, life-long pursuit with many benefits, not the least of which are being great form of exercise to keep fit and possibly saving your life someday in a self-defense situation. But IMHO as an instructor with 30 years experience (on and off), the chances of a kid under 10 getting any real training or developing a lifelong interest is very very small. Chances are much higher for tweens and teens.
So in short, if you want babysitting now that happens to include kicking and punching, by all means go for it. If you want a lifelong gift, wait 10 years for martial arts, and in the mean time try some yoga or other physical activity. (Tai Chi, as one other poster suggested, is also fine, but having tried it, I would be shocked if it weren't too slow to keep a 4-year-old interested.)