When Should I Refactor ?
If you have more than two screenfuls of code in a single method, then it's time to refactor.
If the call stack for one of your libraries is ever more than ten or twenty frames deep (excluding recursion), then it's time to refactor.
If you ever have an inheritance hierarchy more than five levels deep, it's definitely time to refactor. (Usually, its good practise to not have more than two levels of inheritance depth.)
If you have more than 25 classes in a single package/namespace, it's probably time to start thinking about refactoring and if you have more than fifty classes in a single package, then it's almost definitely time to refactor.