The accuracy of the signs and tests that clinicians use to diagnose ventilator-associated pneumonia (VAP) and initiate antibiotic treatment has not been well characterized. We sought to characterize and compare the accuracy of physical examination, chest radiography, endotracheal aspirate (ETA), bronchoscopic sampling cultures (protected specimen brush [PSB] and bronchoalveolar lavage [BAL]), and CPIS > 6 to diagnose VAP. We searched six databases from inception through September 2019 and selected English-language studies investigating accuracy of any of the above tests for VAP diagnosis. Reference standard was histopathological analysis. Two reviewers independently extracted data and assessed study quality. We included 25 studies (1639 patients). The pooled sensitivity and specificity of physical examination findings for VAP were poor: fever (66.4% [95% confidence interval [CI]: 40.7-85.0], 53.9% [95% CI 34.5-72.2]) and purulent secretions (77.0% [95% CI 64.7-85.9], 39.0% [95% CI 25.8-54.0]). Any infiltrate on chest radiography had a sensitivity of 88.9% (95% CI 73.9-95.8) and specificity of 26.1% (95% CI 15.1-41.4). ETA had a sensitivity of 75.7% (95% CI 51.5-90.1) and specificity of 67.9% (95% CI 40.5-86.8). Among bronchoscopic sampling methods, PSB had a sensitivity of 61.4% [95% CI 43.7-76.5] and specificity of 76.5% [95% CI 64.2-85.6]; while BAL had a sensitivity of 71.1% [95% CI 49.9-85.9] and specificity of 79.6% [95% CI 66.2-85.9]. CPIS > 6 had a sensitivity of 73.8% (95% CI 50.6-88.5) and specificity of 66.4% (95% CI 43.9-83.3). Classic clinical indicators had poor accuracy for diagnosis of VAP. Reliance upon these indicators in isolation may result in misdiagnosis and potentially unnecessary antimicrobial use.