While timing and rhythm-related phenotypes are heritable, the human genome variations underlying these traits are not yet well-understood. We conducted a genome-wide association study to identify common genetic variants associated with a self-reported musical rhythm phenotype in 606,825 individuals. Rhythm exhibited a highly polygenic architecture with sixty-eight loci reaching genome-wide significance (p<5×10−8) and SNP-based heritability of 13%-16%. Polygenic scores for rhythm predicted the presence of musician-related keywords in the BioVU electronic health record biobank. Genetic associations with rhythm were enriched for genes expressed in brain tissues. Genetic correlation analyses revealed shared genetic architecture with several traits relevant to cognition, emotion, health, and circadian rhythms, paving the way to a better understanding of the neurobiological pathways of musicality.