I'd even say it's cleaner than almost all of San Francisco and SF's worst parts (eg, the Mission, Bayview, Hunter's Points) are more violent than Durham's worst parts.
Every urban environment of any size is going to have its problems but there is a definite reality distortion field surrounding how violent and dirty people think Durham is.