In visual environments, selective attention must be employed to focus on task-relevant stimuli. A key question here concerns the extent to which other stimuli within the visual field influence target processing. In this study we ask whether face identity matching is subject to similar effects from irrelevant stimuli in the visual field, specifically task-irrelevant people. While most previous studies rely on highly controlled face and body stimuli presented in isolation, here we use a more realistic environment. Participants take the role of passport officers and must match a person’s face to their photo-ID while other people appear in the background, waiting to be processed. Presenting an interactive virtual environment on screen (Experiment 1 and 2) or in immersive VR (Experiment 3), we generally found no evidence for distraction from background people on face-matching accuracy. However, when immersed in VR, an angry crowd in the background increased matching speed, while not affecting accuracy. We discuss the theoretical implications of these results and their potential importance in practical settings.