This is a question that comes down to personal belief rather than science, mainly because the Science swings both ways and the facts are mixed in with opinion.
Looking at the facts, we know that humans are the only species that consumes milk from another animal, and also the only species that continue to drink milk after weaning during infancy.
However, we also know that dairy is an excellent source of calcium, protein and fat-soluble nutrients such as vitamins K, D and A. This is especially true of organic and grass-fed dairy, which offers abundant nutrients for development, health and wellness.
Bringing science and history into the equation, we discover that cultures that consume the least amount of dairy also have the lowest osteoporosis and bone fracture rates. Studies show us that dairy does not strengthen bones as advertising leads us to believe, and the saturated fat levels in dairy may also be harmful to cardiovascular health. This does depend heavily on the quality of the dairy, and whether it has been heavily processed or not.
If you do enjoy consuming dairy, the healthiest option is to go with small amounts of fermented grass-fed dairy occasionally. This includes yoghurt, kefir and fermented cheeses. The lactose in these products is broken down by the bacteria, so they are generally quite well tolerated. Ghee is also a nutrient-rich option that is very well tolerated because it has virtually zero lactose, as mentioned above.