Although many patients with cluster headaches (CH) are disabled by their condition, few studies have examined this in detail. This cross-sectional, multicenter observational study prospectively collected demographic and clinical questionnaire data from 224 consecutive patients with CH. We assessed headache impact using the six-item Headache Impact Test (HIT-6) and evaluated the factors associated with the impact of CH. Participants with a HIT-6 score ≥ 60 were classified into a severe impact group. The majority (190, 84.8%) of the participants were classified into the severe impact group. These patients were characterized by younger age, earlier onset of CH, longer duration of each headache attack, higher pain intensity, more cranial autonomic symptoms, a higher proportion of depression or anxiety, higher score of stress, and lower score of quality of life. The anxiety (OR = 1.19, 95% CI: 1.08–1.31, p = 0.006), greater pain intensity (OR = 1.06, 95% CI: 1.02–1.10, p = 0.002), and age (OR = 0.99, 95% CI: 0.99–1.00, p = 0.008) were significant predictors for a severe impact of CH patients. According to the HIT-6 results, most of the CH patients were significantly affected by CH. As well as pain intensity, anxiety and age modulated CH’s impact on their lives.