I am using Version 126.96.36.199 (x64) on Windows 7 Ultimate. I am figuring that it might just be C# not garbage collecting dangling references as I've seen this before, especially with the collection classes.
The simple workaround is to stop VidCoder every few days and just requeue the last file it was working on. The program saves the existing queue and can restart everything but the job which was currently working.
I'd grab the source and do a debug build with a memory analyzer if I weren't so busy right now.