The emerging coronavirus disease 2019 (COVID-19), first identified in December 2019, spread worldwide and was declared a public health emergency of international concern and a pandemic in February and March 2020, respectively, by the World Health Organization (WHO).1,2 COVID-19 is caused by a novel, enveloped RNA betacoronavirus that has a phylogenetic similarity to severe acute respiratory syndrome coronavirus (SARS-Co-V) and has been formally named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).3 Given the high transmissibility and the significant burden this pandemic caused on healthcare systems, timely and accurate diagnosis is considered key in its management. Early detection can assist in both appropriate management of cases and prompt application of infection control measures to reduce its transmission in the community.4 Nucleic acid amplification tests (NAATs), such as real-time reverse transcription polymerase chain reaction (rRT-PCR) assays based on the detection of SARS-CoV-2 genetic material, have been widely used and recommended for diagnosing COVID-19.5,6 However, several challenges were faced with NAATs due to the increased burden of testing, including the high cost, need for human resources, and most importantly the delay in results.7 Rapid antigen tests are designed to detect viral particles from samples such as the throat or nasopharyngeal swabs in a dramatically shorter time than rRT-PCR. Tests are now available from several manufacturers, many of which are Food and Drug Administration-approved.8 However, the main concern of these tests is the variably lower sensitivity compared to rRT-PCR, with the potential risk of missing active cases.9 This study evaluates the performance of four different rapid antigen tests compared to rRT-PCR to determine the feasibility of integrating these tests into the diagnostic algorithm in clinical settings.

Methods

We conducted a prospective diagnostic test accuracy study of four rapid SARS-CoV-2 antigen detection tests compared to rRT-PCR betweem June and July 2020. It was conducted in Central Public Health Laboratories (CPHL), Muscat, which run rRT-PCR testing of COVID-19, in addition to two centers of disease control and prevention.

All clinically suspected COVID-19 patients who attended Medical Fitness Center (Darsait and A’Seeb) with acute respiratory symptoms or pneumonia during the evaluation period were included in the study regardless of severity or onset of symptoms. Asymptomatic patients were excluded. Nasopharyngeal and/or throat swabs were collected in viral transport media (VTM) from patients and tested via rRT-PCR and antigen assay according to manufacturers’ instructions. When indicated by the antigen kit provider, additional nasopharyngeal and/or throat samples (using swabs provided by the manufacturer) were collected for antigen tests in addition to routine VTM swabs for rRT-PCR.

We evaluated four COVID-19 rapid antigen tests to detect SARS-CoV-2 compared to rRT-PCR. The antigen fluorescent immunoassay (FIA) tests included the following:

• STANDARD™ Q COVID-19 Ag test (SD BIOSENSOR, Korea); a chromatographic immunoassay for the qualitative detection of specific antigens to SARS-CoV-2. Positive results are indicated by the visual appearance of a line in the designated window of the kit.
• PCL COVID-19 Ag Rapid FIA test (PCL, Korea); a FIA to detect SARS-CoV-2 nucleoprotein antigens in human nasopharyngeal swab specimen. Results are read automatically by an analyzer.
• BIOCREDIT COVID-19 Ag test (RapiGEN Inc., Korea); a lateral flow immunochromatographic assay. Positive results are shown by the appearance of a black line in the result window of the kit.
• Sofia SARS-CoV-2 antigen FIA test (Quidel, USA); The test uses immunofluorescence-based lateral flow technology in a sandwich design for the qualitative detection of nucleocapsid protein from SARS-CoV-2. Results are read by an analyzer and shown on a screen.

All samples were collected and tested according to manufacturer instructions.

RNA extraction was performed using a Liferiver extractor® (Shanghai ZJ Bio-Tech Co., Ltd.) or QIAmp viral RNA mini extraction kit (Qiagen).

Detection of SARS-CoV-2 was done by molecular assays using Liferiver Novel Coronavirus (2019-nCoV) Real-Time Multiplex RT-PCR Kit, Sansure Biotech COVID-19 Nucleic Acid Test Kit, or the Roche cobas® 6800 SARS-CoV-2 test. The SARS-CoV-2 genes targeted by the PCR assays are ORF, N, and E genes by Liferiver assay, ORF, and N genes by Sansure assay, ORF, and E genes by COBAS6800 assay.

Data were described using frequency and percentage. Diagnostic accuracy measures were calculated using MedCalc software (version 19.1.6). The correlation between rRT-PCR Ct value (ORF gene) and antigen test results was established using a point-biserial correlation coefficient. A p-value < 0.050 was considered statistically significant. According to epidemiological data, the positive predictive value (PPV) and negative predictive value (NPV) were calculated based on an assumed prevalence of COVID-19 in Oman of 9.2%.

Results

A total of 306 (nasopharyngeal, throat, or both) swabs were included in this study. Sixty-six samples were tested by the STANDARD™ Q COVID-19 Ag test, 87 samples by the PCL COVID-19 Ag Rapid FIA test, 75 samples by the BIOCREDIT COVID-19 Ag test, and 78 samples by the Sofia SARS-CoV-2 antigen FIA test. Results of antigen tests were compared with rRT-PCR. The sensitivity, specificity, accuracy, PPV, and NPV were calculated [Table 1]. All antigen tests demonstrated a specificity of 100% except for PCL, which has a lower specificity of 94.1%. However, sensitivities of the four antigen tests ranged between 64.0%
and 69.8%.

All assays showed a significant negative correlation between the reference rRT-PCR and Ag tests with statistically significant p-values [Figure 1 and Table 1]. Table 1 also shows each rapid antigen test’s sensitivity, PPV, and NPV after excluding samples with high Ct values (low viral loads). Sensitivity improved to ≥ 85% for the STANDARD™ Q COVID-19 Ag test, PCL COVID-19 Ag Rapid FIA test, and BIOCREDIT COVID-19 Ag test after excluding samples with PCR Ct values > 30. For the Sofia SARS-CoV-2 antigen FIA test, this data was not available. However, its sensitivity was 100% for samples with PCR Ct values < 20.

Discussion

As the COVID-19 pandemic continued to spread, the crucial role of diagnostic tests was proven. rRT-PCR assays have been used widely and played a vital role in many countries’ response to the disease by allowing epidemiologists to more effectively track the spread and determine infection rates in given geographical areas. However, rRT-PCR is costly, time-consuming, and labor-intensive. Thus, the need for rapid, affordable tests is necessary. Recently, rapid SARS-CoV-2 antigen tests have been developed. Therefore, this study was undertaken to assess four commercial antigen detection assays’ diagnostic accuracy via comparison to molecular-based tests to determine if a person has a current SARS-CoV-2 infection.

Our results showed test sensitivities in the range of 64.0–69.8%, with the PCL assay demonstrating a slightly higher sensitivity of 69.8% compared to the other three tests. However, the PCL assay specificity was 94.1%, compared to 100% for the other three assays. As shown in Table 1, sensitivities increase when the CT values are low, which occurs early in infection and probably in the first few days of symptoms.

Generally, although rRT-PCR assays detect SARS-CoV-2 at an average sensitivity of 95.2% and specificity of 98.9%,10 the detection limits and the ability to differentiate between true negatives and positives at low RNA concentrations vary between assays. In individual laboratories, careful evaluation is required to determine Ct value cut-offs for differentiating between positives and negatives. It is possible that at high Ct values, genetic fragments of the virus are detected, which are not indicative of live virus and therefore not clinically meaningful.11

All assays showed a significant negative correlation between the reference rRT-PCR Ct values and rapid antigen test results. This indicates that rapid antigen tests are likely to perform better with high viral loads (lower Ct values). High viral loads usually occur in the pre-symptomatic phase (1–3 days before symptoms onset) and early symptomatic phase (within the first 5–7 days) of the illness, which are also the periods with the highest rate of infectivity.9

The WHO has set minimum performance requirements at ≥ 80% sensitivity and ≥ 97% specificity for COVID-19 assays, which was also agreed on by the European Center for Disease Prevention and Control.9,12 In our study, excluding samples with PCR Ct values > 30 and > 25 resulted in sensitivity improvements up to ≥ 85% and > 90%, respectively, for the STANDARD™ Q COVID-19 Ag test, the PCL COVID-19 Ag Rapid FIA test, and the BIOCREDIT COVID-19 Ag test. As the Sofia SARS-CoV-2 antigen FIA test assessment did not include samples in the PCR Ct value between 20–30 category, the evaluation of sensitivity at Ct < 30 or < 25 was not possible. However, it had 100% sensitivity for samples with PCR Ct value < 20.

Although the principal concern of antigen detection assays are the false-negative rates due to low viral load or high Ct values, the clinical significance of this limitation might be mitigated to some extent, knowing that infectivity of patients with Ct > 24 and duration of symptoms over eight days may be low.13 In addition, setting a diagnostic algorithm to confirm negative antigen test results when clinically relevant may further mitigate the risk of missing active cases.

All four assays were easy to perform in the clinical laboratory within < 30 min, with the BIOCREDIT assay demonstrating the shortest timeframe (5–8 min). However, one limitation of the BIOCREDIT and STANDARD™ assays is that the assay’s result is determined by the visual presence or absence of a line, which is recorded by an operator, making it prone to operator subjectivity. In contrast, the fluorescence readout of the PCL and Sofia assays are generated on automated analyzers, preventing operator bias.

This study has some limitations. Clinical information, including symptoms and duration of symptoms at sampling time, were not available to correlate with PCR and antigen results. Another limitation is that different PCR assays were used to compare antigen results. Finally, for most patients, PCR and antigen tests were performed from two swabs taken at the same visit. While this might be considered a strength, it is well-known that sampling quality might affect the results, and maintaining the same level of quality for each pair of swabs cannot be guaranteed.

Conclusion

The accurate and rapid diagnosis of people infected with the SARS-CoV-2 virus is essential to address the global spread of COVID-19. Given the simplicity, rapidity, low cost, and high specificity of antigen tests, we speculate that integrating antigen tests into the clinical diagnostic algorithms would help contain the outbreak if correctly performed and interpreted. The limitation of the low sensitivity of rapid antigen tests is probably overestimated since the missed cases are likely to have low viral loads and are less infectious. However, confirming negative cases (where clinically relevant) by repeating the test or using a more sensitive assay like PCR is recommended to decrease the chance of missing active cases.

Disclosure

The authors declared no conflicts of interest. No funding was received for this study.

Acknowledgements

We would like to thank all the personnel working at Central Public Health Laboratories, and Darsait and Seeb Medical Fitness Examination Centers for their technical assistance and great work during the current COVID-19 pandemic.

references

1. 1. World Health Organization. COVID-19 Public Health Emergency of International Concern (PHEIC). 2020 [cited 2020 August 10]. Available from: https://www.who.int/publications/m/item/covid-19-public-health-emergency-of-international-concern-(pheic)-global-research-and-innovation-forum.
2. 2. World Health Organization. WHO announces COVID-19 outbreak a pandemic. 2020 [cited 2020 August 10]. Available from: https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/3/who-announces-covid-19-outbreak-a-pandemic.
3. 3. Pal M, Berhanu G, Desalegn C, Kandi V. Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2): an update. Cureus 2020 Mar;12(3):e7423.
4. 4. Cheng MP, Papenburg J, Desjardins M, Kanjilal S, Quach C, Libman M, et al. Diagnostic testing for severe acute respiratory syndrome–related coronavirus-2: a narrative review. Ann Intern Med 2020 Jun;172(11):726-734.
5. 5. Centers for Disease Control and Prevention. Overview of testing for SARS-CoV (COVID-19). 2020 [cited 2020 August 10]. Available from. https://www.cdc.gov/coronavirus/2019-ncov/hcp/testing-overview.html.
6. 6. Hanson KE, Caliendo AM, Arias CA, Hayden MK, Englund JA, Lee MJ, et al. The infectious diseases society of America guidelines on the diagnosis of COVID-19. Molecular Diagnostic Testing 2020 [cited 2020 September 12]. Available from: https://www.idsociety.org/practice-guideline/covid-19-guideline-diagnostics/.
7. 7. Younes N, Al-Sadeq DW, Al-Jighefee H, Younes S, Al-Jamal O, Daas HI, et al. Challenges in laboratory diagnosis of the novel coronavirus SARS-CoV-2. Viruses 2020 May;12(6):582.
8. 8. Food And Drug Administration. In vitro diagnostics EUAs: individual EUAs for antigen diagnostic tests for SARS-CoV-2. 2021 [cited 2021 January 27]. Available from: https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitro-diagnostics-euas-antigen-diagnostic-tests-sars-cov-2.
9. 9. World Health Organization. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. Interim guidance 2020 [cited 2020 November 12]. Available from: https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays.
10. 10. Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, et al; Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2020 Aug;8(8):CD013705.
11. 11. Zheng S, Fan J, Yu F, Feng B, Lou B, Zou Q, et al. Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China, January-March 2020: retrospective cohort study. BMJ 2020 Apr;369(Apr):m1443.
12. 12. European Center for Disease Prevention and Control. Options for the use of rapid antigen tests for COVID-19 in the EU/EEA and the UK. Technical Report 2020 [cited 2020 November 25]. Available from: https://www.ecdc.europa.eu/en/publications-data/options-use-rapid-antigen-tests-covid-19-eueea-and-uk.
13. 13. Bullard J, Dust K, Funk D, Strong JE, Alexander D, Garnett L, et al. Predicting infectious severe acute respiratory syndrome coronavirus 2 from diagnostic samples. Clin Infect Dis 2020 Dec;71(10):2663-2666.