The modification of nucleocytoplasmic proteins with O-linked N-Acetylglucosamine (O-GlcNAc) plays diverse roles in multicellular organisms. Inhibitors of O-GlcNAc hydrolase (OGA), the enzyme that removes O-GlcNAc from proteins, lead to increased O-GlcNAc levels in cells and are seeing widespread adoption in the field as a research tool used in cells and in vivo. Here we synthesize and study a series of tight binding carbohydrate-based inhibitors of human OGA (hOGA). The most potent of these 2′-Aminothiazolines binds with a sub-nanomolar Ki value to hOGA (510 ± 50 pM) and the most selective has greater than 1800000-fold selectivity for hOGA over mechanistically related human lysosomal β-hexosaminidase. Structural data of inhibitors in complex with an hOGA homologue reveals the basis for variation in binding among these compounds. Using linear free energy analyses, we show binding of these 2′-Aminothiazoline inhibitors depends on the pKa of the aminothiazoline ring system, revealing the protonation state of the inhibitor is a key driver of binding. Using series of inhibitors and synthetic substrates, we show that 2′-Aminothiazoline inhibitors are transition state analogues of hOGA that bind to the enzyme up to 1-million fold more tightly than the substrate. These collective data support an oxazoline, rather than a protonated oxazolinium ion, intermediate being formed along the reaction pathway. Inhibitors from this series will prove generally useful tools for the study of O-GlcNAc. The new insights gained here, into the catalytic mechanism of hOGA and the fundamental drivers of potency and selectivity of OGA inhibitors, should enable tuning of hOGA inhibitors with desirable properties.