Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but ‘familiarity’ covers a wide range—from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.