This problem has been reported repeatedly in the past. I used to have it on my 85TD. Even bought a new starter in an attempt to solve it. I never installed it and the the problem has gone away. No idea what was causing it but i had precisely the same symptoms which got worse int he cold. Once it even failed to start at all around 0 degrees. No problems in over a year now.

__________________
1977 300d 70k--sold 08
1985 300TD 185k+
1984 307d 126k--sold 8/03
1985 409d 65k--sold 06
1984 300SD 315k--daughter's car
1979 300SD 122k--sold 2/11
1999 Fuso FG Expedition Camper
1993 GMC Sierra 6.5 TD 4x4
1982 Bluebird Wanderlodge CAT 3208--Sold 2/13