The encoding of speech and music in the auditory brainstem is available at the human scalp via the auditory-evoked frequency following response. The FFR, primarily reflecting activity in the inferior colliculus, may be evoked by speech or music stimulation and represents the combined activity of sensorimotor, cognitive, and reward centers in the brain. Its response properties, like the inferior colliculus itself, are influenced by long-term experience with sound. The transparency, individual-level reliability, and ability to gauge neural plasticity provide the researcher and clinician a powerful probe of auditory processing in the human brainstem. With it, we have learned a great deal about how mechanisms of decline, deprivation, and enrichment affect the processing of complex signals such as music and speech in the human brainstem.