Every one of my job evaluations with my current employer has been done on the fly. Usually at the end of the day when my boss is on the road. We can afford to do this in a relaxed fashion since we are a small shop and give each other feedback pretty much every day as part of our workflow.
Larger companies with many employees could really benefit from software to keep track of employee evaluations. ManagerAssistant.com offers a total employee management package to track employee salary and benefits, activities, job performance reviews and attendance records.