Purpose: We sought to refine the clinical picture of primary adrenal lymphoma (PAL), a rare lymphoid malignancy with predominant adrenal manifestation and risk of adrenal insufficiency. Methods: Ninety-seven patients from 14 centers in Europe, Canada and the United States were included in this retrospective analysis between 1994 and 2017. Results: Of the 81 patients with imaging data, 19 (23%) had isolated adrenal involvement (iPAL), while 62 (77%) had additional extra-adrenal involvement (PAL+). Among patients who had both CT and PET scans, 18FDG-PET revealed extra-adrenal involvement not detected by CT scan in 9/18 cases (50%). The most common clinical manifestations were B symptoms (55%), fatigue (45%), and abdominal pain (35%). Endocrinological assessment was often inadequate. With a median follow-up of 41.6 months, 3-year progression-free (PFS) and overall (OS) survival rates in the entire cohort were 35.5% and 39.4%, respectively. The hazard ratios of iPAL for PFS and OS were 40.1 (95% CI: 2.63-613.7, P = 0.008) and 2.69 (95% CI: 0.61-11.89, P = 0.191), respectively. PFS was much shorter in iPAL vs PAL+ (median 4 months vs not reached, P = 0.006), and OS also appeared to be shorter (median 16 months vs not reached), but the difference did not reach statistical significance (P = 0.16). Isolated PAL was more frequent in females (OR = 3.81; P = 0.01) and less frequently associated with B symptoms (OR = 0.159; P = 0.004). Conclusion: We found unexpected heterogeneity in the clinical spectrum of PAL. Further studies are needed to clarify whether clinical distinction between iPAL and PAL+ is corroborated by differences in molecular biology.