Advances in neighborhood research have been constrained by the lack of neighborhood data for many geographical areas. We utilize millions of Google Street View images across the United States and leverage convolutional neural networks to automatically label each image. We implemented regression models to estimate associations between built environments and health outcomes at various levels including county, census tract and individual level. At these various levels, we consistently find that built environment (walkability, physical disorder, urban development) have significant impacts on health behaviors and health outcomes. GSV images represent an underutilized resource for building national data on neighborhoods and examining the influence of built environments on community health outcomes across the United States.