Roy Jones was probably the best non-heavyweight I ever saw. He just had the bad luck to be fighting weak fighters in his prime, similar to Tyson. He didn't have the competition around that guys like Leonard, Hearns and Hagler did.
I would also put Duran up there with those guys but he was a little before my time so I really didn't see him in his prime. But from what I understand he was scary.
I just don't think you should penalize someone who came up at a time when competition was weak.