A better understanding of the immunological markers associated with long-lasting immune responses to SARS-CoV-2 infection is of paramount importance. In the present study, we characterized SARS-CoV-2-specific humoral responses in hospitalized (ICU and non-ICU) and non-hospitalized individuals at six months post-onset of symptoms (POS) (N = 95). We showed that the proportion of individuals with detectable anti-SARS-CoV-2 IgG or neutralizing (NAb) responses and the titers of antibodies were significantly reduced in non-hospitalized individuals, compared to ICU- or non-ICU-hospitalized individuals at 6 months POS. Interestingly, SARS-CoV-2-specific memory B cells persist at 6 months POS in both ICU and non-ICU patients and were enriched in cells harboring an activated and/or exhausted phenotype. The frequency/phenotype of SARS-CoV-2-specific memory B cells and the magnitude of IgG or NAb responses at 6 months POS correlated with the serum immune signature detected at patient admission. In particular, the serum levels of CXCL13, IL-1RA, and G-CSF directly correlated with the frequency of Spike-specific B cells and the magnitude of Spike-specific IgG or NAb, while the serum levels of CXCL12 showed an antagonizing effect. Our results indicate that the balance between CXCL12 and CXCL13 is an early marker associated with the magnitude and the quality of the SARS-CoV-2 humoral memory.