Think about it: You have muscles everywhere, from your forehead to your pinkie toes, but you don't see too many people walking around with excessively built thumbs, do you? Of course not. Smaller muscles, by their very nature, aren't designed to "get big," the way larger muscle groups are.
This difference in "buildability" is due to a surprisingly long list of factors, including, but quite possibly not limited to, the following:
- Anatomical cross-sectional area: Often measured as the widest point of a muscle along the muscle's length
- Muscle structure: The length and angle of muscle fascicles (basically just a bundle of muscle) in relation to the line of action of the muscle
- Individual anatomical differences: Specifically, moment arm length, which affects torque at different joints
- Neural drive: How effectively your brain recruits motor units (muscle groups) and how frequently these motor units fire
- Muscle fiber characteristics: Distribution of muscle fiber type, based on type I ("slow twitch") fibers and type II ("fast twitch") fibers within a given muscle, as well as presence and physiological predisposition to activate satellite cells
Whew. There's a reason it's called exercise science.
Very generally speaking, you can assume that larger muscle groups, particularly those with a larger cross-sectional area, such as the glutes, will be easier to get bigger, faster. It's also a fairly safe assumption that muscles that happen to have a higher percentage of fast-twitch fibers, including the pecs, lats, triceps, biceps, and quads, will also experience faster, more pronounced growth.