In the mouse, over the last 20 years, a set of cell-surface markers and activities have been identified, enabling the isolation of bone marrow (BM) populations highly enriched in hematopoietic stem cells (HSCs). These HSCs have the ability to generate multiple lineages and are capable of long-term self-renewal activity such that they are able to reconstitute and maintain a functional hematopoietic system after transplantation into lethally irradiated recipients. Using single-cell reconstitution assays, various marker combinations can be used to achieve a functional HSC purity of almost 50%. Here we have used the differential expression of six of these markers (Sca1, c-Kit, CD135, CD48, CD150, and CD34) on lineage-depleted BM to refine cell hierarchies within the HSC population. At the top of the hierarchy, we propose a dormant HSC population (Lin(-)Sca1(+)c-Kit(+) CD48(-)CD150(+)CD34(-)) that gives rise to an active self-renewing CD34(+) HSC population. HSC dormancy, as well as the balance between self-renewal and differentiation activity, is at least, in part, controlled by the stem cell niches individual HSCs are attached to. Here we review the current knowledge about HSC niches and propose that dormant HSCs are located in niches at the endosteum, whereas activated HSCs are in close contact to sinusoids of the BM microvasculature.