For a long time, I could consumer goats dairy products (and sheep) but not cows dairy. From my early 20's through to around 34-35yrs old.
My understanding of that is that it is the different proteins (or shape of the same protein?) that I was able to get away with. I was also aware of a small number of types of cows that I could consume without issues as well (Jarlseberg cheese was one, as was Comte and Emmental, though I was less tolerant of those than Jarlseberg.
Curiously, I'm reading online that Jarlseberg is actually lactose free. Intriguing.
The lactose levels are not that different though (4.2% compared to 5% in cows' milk).
Goat milk versus cow milk: A comparison