Purpose. To examine the associations among intake of refined grains, whole grains and dietary fiber and aggressiveness of prostate cancer in African Americans (AA, n = 930) and European Americans (EA, n = 993) in a population-based, case-only study (The North Carolina-Louisiana Prostate Cancer Project, PCaP). Methods. Prostate cancer aggressiveness was categorized as high, intermediate or low based on Gleason grade, PSA level and clinical stage. Dietary intake was assessed utilizing the NCI Diet History Questionnaire. Logistic regression (comparing high to intermediate/low aggressive cancers) and polytomous regression with adjustment for potential confounders were used to determine odds of high prostate cancer aggressiveness with intake of refined grains, whole grains and dietary fiber from all sources. Results. An inverse association with aggressive prostate cancer was observed in the 2nd and 3rd tertiles of total fiber intake (OR = 0.70; 95% CI, 0.50-0.97 and OR = 0.61; 95% CI, 0.40-0.93, resp.) as compared to the lowest tertile of intake. In the race-stratified analyses, inverse associations were observed in the 3rd tertile of total fiber intake for EA (OR = 0.44; 95% CI, 0.23-0.87) and the 2nd tertile of intake for AA (OR = 0.57; 95% CI, 0.35-0.95). Conclusions. Dietary fiber intake was inversely associated with aggressive prostate cancer among both AA and EA men.