The relationship between N-antigen concentration and viral load within and across different specimens guides the clinical performance of rapid diagnostic tests (RDT) in different uses. A prospective study was conducted in Porto Velho, Brazil, to investigate RDT performance in different specimen types as a function of the correlation between antigen concentration and viral load. The study included 214 close contacts with recent exposures to confirmed cases, aged 12 years and older and with various levels of vaccination. Antigen concentration was measured in nasopharyngeal swab (NPS), anterior nares swab (ANS), and saliva specimens. Reverse transcriptase (RT)–PCR was conducted on the NPS and saliva specimens, and two RDTs were conducted on ANS and one RDT on saliva. Antigen concentration correlated well with viral load when measured in the same specimen type but not across specimen types. Antigen levels were higher in symptomatic cases compared to asymptomatic/oligosymptomatic cases and lower in saliva compared to NPS and ANS samples. Discordant results between the RDTs conducted on ANS and the RT-PCR on NPS were resolved by antigen concentration values. The analytical limit of detection of RDTs can be used to predict the performance of the tests in populations for which the antigen concentration is known. The antigen dynamics across different sample types observed in SARS-CoV-2 disease progression support use of RDTs with nasal samples. Given lower antigen concentrations in saliva, rapid testing using saliva is expected to require improved RDT analytical sensitivity to achieve clinical sensitivity similar to rapid testing of nasal samples.