Weak values are quantities accessed through quantum experiments involving weak measurements and postselection. It has been shown that “anomalous” weak values (those lying beyond the eigenvalue range of the corresponding operator) defy classical explanation in the sense of requiring contextuality [M. F. Pusey, Phys. Rev. Lett. 113, 200401 (2014)]. Here we elaborate on and extend that result in several directions. First, the original theorem requires certain perfect correlations that can never be realized in any actual experiment. Hence, we provide theorems that allow for a noise-robust experimental verification of contextuality from anomalous weak values, and compare with a recent experiment. Second, the original theorem connects the anomaly to contextuality only in the presence of a whole set of extra operational constraints. Here we clarify the debate surrounding anomalous weak values by showing that these conditions are tight: if any one of them is dropped, the anomaly can be reproduced classically. Third, whereas the original result required the real part of the weak value to be anomalous, we also give a version for any weak value with nonzero imaginary part. Finally, we show that similar results hold if the weak measurement is performed through qubit pointers, rather than the traditional continuous system. In summary, we provide inequalities for witnessing nonclassicality using experimentally realistic measurements of any anomalous weak value, and clarify what ingredients of the quantum experiment must be missing in any classical model that can reproduce the anomaly.