My experience having a flock of 20-40 chooks at anyone time is that it is down to the individual chook. Some drink too much water, some don't eat enough stones (stones help them digest what they eat). Sometimes chooks that are just starting to lay show this issue in their eggs, other times it's the much older retired birds. Some of my girls I know we simply can not peel their eggs at all, so I don't even try anymore. Their eggs are for cracking open raw and cooking out of the shell.
It really seems, as you say, to be a lottery.
The teaspoon technique does help but you still won't get it 'perfect'. But it doesn't leave quite such large chunks missing. Older eggs where the air sac is larger also tend to shell better as mentioned above. This is because the membrane has pulled away from the shell as the water content of the white has reduced. Shells are porous as I'm sure you know.
Otherwise all you can do is change your supplier when you know you want hard boiled eggs. We just put up with it and use the teaspoon method simply because we connect a dozen or so eggs everyday so buying them is pointless. I'm just very careful with which eggs we use from which of the chooks. I can identify about 75% if our eggs down to a specific bird. The rest I can only say down to a specific breed.