When we last left Mike's Mail Client, our protagonist was left with a client with message integrity and a journal... that still stored its canonical message indexes in GDBM files.

Kicking an old habit can be hard. You're like "well, what if I'm doing something really wrong here? What if I need to go back?"

So at first, I set about trying to make the CSV index files primary and maintaining the GDBM files purely as backup - essentialy inverting the current arrangement for indexes the way I had for messages.

Of course, this isn't at all straightforward. Simply dumping an index maintained from my CSV files kind of defeats the purpose of maintaining an independant store. OTOH, maintaining parallel indexes and journals and attempting to reconcile the two is just a recipe for pain.

I like to avoid pain, so I had been successfully avoiding finishing this effort off until fate forced my hand: I once again exceeded the size limit on my inbox GDBM file. Free from the constraints of having to maintain a working system, and unable to get any new e-mail until I fixed the problem, I decided to simply jump in with both feet and cut over to the CSV indexes.

I'd already done most of the work, finishing off the fix mostly consisted of deleting the code that loads from GDBM.

The code that applies the new journal is a little funny. It ends up keeping track of the folders that a message was added to, deleted from and marked read in with some additional information about order. I think the intent of all of this was to keep track of messages which should be considered for garbage collection (deleted in one or more folders, added in none) though it seems like the code could be simplified, I'll have to take another look at it. For now, it works.

The upshot is: the new code is in operation. The GDBM folders are now bit-rotting on my hard drive. E-mail works again, and (considering that my inbox index is now only 115K) the new folders seem likely to degrade much more gracefully than the old ones.

I suspect I could now track every e-mail I've ever archived from a single index that fits comfortably in memory. And after this migration, that's something I can fairly easily test.