This study examines the effects of neighborhood racial in-group size, economic deprivation and the prevalence of crime on neighborhood cohesion among U.S. whites. We explore to what extent residents’ perceptions of their neighborhood mediate these macro-micro relationships. We use a recent individual-level data set, the American Social Fabric Study (2012/2013), enriched with contextual-level data from the U.S. Census Bureau (2010) and employ multi-level structural equation models. We show that the racial in-group size is positively related to neighborhood cohesion and that neighborhood cohesion is lower in communities with a high crime rate. Individuals’ perceptions of the racial in-group size partly mediate the relationship between the objective racial in-group size and neighborhood cohesion. Residents’ perceptions of unsafety from crime also appear to be a mediating factor, not only for the objective crime rate but also for the objective racial in-group size. This is in line with our idea that racial stereotypes link racial minorities to crime whereby neighborhoods with a large non-white population are perceived to be more unsafe. Residents of the same neighborhood differ in how they perceive the degree of economic decay of the neighborhood and this causes them to evaluate neighborhood cohesion differently, however perceptions of neighborhood economic decay do not explain the link between the objective neighborhood context and neighborhood cohesion.