Atmospheric pressure plasmas have been ground-breaking for plasma science and technologies, due to their significant application potential in many fields, including medicinal, biological, and environmental applications. This is predominantly due to their efficient production and delivery of chemically reactive species under ambient conditions. One of the challenges in progressing the field is comparing plasma sources and results across the community and the literature. To address this a reference plasma source was established during the `Biomedical Applications of Atmospheric Pressure Plasmas' EU COST Action MP1101. It is crucial that reference sources are reproducible. Here, we present the reproducibility and variance across multiple sources through examining various characteristics, including: absolute atomic oxygen densities, absolute ozone densities, electrical characteristics, optical emission spectroscopy, temperature measurements, and bactericidal activity. The measurements demonstrate that the tested COST jets are mainly reproducible within the intrinsic uncertainty of each measurement technique.