I think you do have to take the fan + shroud off, if only to give your hands more room to work. Assuming you have the right tools, I think the most difficult part of the job is:

1. removing the plug wires from the cap.
2. accessing the bottom allen screw for the cap.

But most especially #1. The rest of it isn't bad.