Multiscale modeling of large self-organizing systems with applications to pedestrian and opinion dynamics