I can absolutely see magic not being able to correct genetic or congenital conditions. It can make sense for developmental delays aswell. But something like missing a limb from a traumatic injurie or blindness due to macular degeneration… There is no reason a mid level adventurer or powerfull character would not just use magic to heal or fix it.
Maybe an injurie by a powerful lich, or since kinda of cursed weapon that makes it impossible to fully heal with anything short of a wish spell…
Poor people on the other hand, should absolutely have debilitating injuries and disabilities that will never be fully fixed due to magic being expensive.