A central challenge in conservation genetics is revealing rare species` population genetic structure and demographic history through comparative studies. To illustrate this fundamental concept, the population genetic structure and demographic history of the locally rare Pygmy gerbil (Gerbillus henleyi) were studied alongside those of the more common Anderson`s gerbil (Gerbillus andersoni allenbyi). Gerbils were trapped in several locations in Israel`s western and inner Negev sandy areas. Two mitochondrial DNA sequences, the control region (CR) and the cytochrome-oxidase-2 gene (CO2), were amplified using DNA extracted from collected ear samples. A moderate-to-great genetic differentiation was detected among Anderson`s gerbil populations, which is more common but has a higher affinity to sandy habitats. This pattern was more pronounced between the western and inner Negev populations. No population genetic structure was evident in the rare Pygmy gerbil, which is known to exhibit broad habitat use. Analyses of the species` demographic history indicated that Anderson`s gerbil had undergone a substantial population expansion during the last 30 thousand years. This expansion fits well with known geomorphological data about the formation of the Negev sand dunes and paleontological data about this species` spread throughout the Levant. The same analyses for Pygmy gerbil indicated that its population size had not changed recently and is under an ongoing bottleneck. This bottleneck may result from the competition with the larger gerbil species. In conclusion, adopting a comparative approach can markedly improve our understanding of the causes and effects of rarity, allowing us to protect key biodiversity modalities better.