Children of prisoners are often negatively affected by their parents’ incarceration, which raises issues of justice. A common view is that the many negative effects associated with parental imprisonment are unjust, simply because children of prisoners are impermissibly harmed or unjustly punished by their parents’ incarceration. We argue that proposals of this kind have problems with accounting for cases where it is intuitive that prison might create social injustices for children of prisoners. Therefore, we suggest that in addition to the question of whether children of prisoners are impermissibly harmed, we should ask whether the inequalities that these children endure because of their parent’s incarceration are objectionable from a social justice perspective. To answer this latter question, we examine the negative effects associated with parental imprisonment from the perspective of luck egalitarianism. We develop a luck egalitarian account that incorporates insights from the philosophy of childhood. On our account, children of prisoners might endure two different types of objectionable inequalities, since they are often deprived of resources that are important for ensuring fair equality of opportunity in adulthood, but also because they are likely to suffer inequalities in terms of childhood welfare. After defending this account, we explore its implications for policy.