I have this issue in duplicacy utils related to the filters file.
My question is to the people who use Thunderbird:
is %profile%\AppData\Roaming\Thunderbird\Profiles\uulrl7mw.default\ImapMail
a folder safe to exclude?
Would you lose any functionality by excluding it?
My concern would be: what if there are accounts stored in Thunderbird just for history/archival purposes (eg. there’s no remote server – mails are moved by hand). Won’t they lose all the images and rest of attachments?
I use Thunderbird with Gmail (G Suite), some considerations:
If you really want to have a backup of your emails, you can not ignore the imapmail folder. Just think of the “3” from the 3-2-1 principle of backups, you should have 3 copies: one on Google, one on my machine and the third … on Duplicacy backup. Consider that: what if, for some disaster, Google excludes the account? What if a hacker discovers (using another way, like phishing, etc.) your password, download your e-mails and delete them? Not everyone has two-factor authentication enabled.
Another point: it is perfectly comfortable to do daily backups of the folder (I back up the entire profile folder), remember that they are just text files. However, some elements can be excluded: the index files and the database. They are recreated by Thunderbird itself.
Some email files (Google specific) can also be excluded and don’t need to be restored:
And obviously you don’t need the backup of:
My filters are like this (note that I configure one version in English and another in my language, Portuguese, just in case):
#=============== "UNWANTED" FILES ===============
e:(?i).*Todos os e-mails.*$
e:(?i).*Todos os e-mails.$
e:(?i).*All Mail.*$
e:(?i).*All Mail$
e:(?i).*Importante.*$
e:(?i).*Importante.$
e:(?i).*Important.*$
e:(?i).*Important.$
e:(?i).*Lixeira.*$
e:(?i).*Lixeira$
e:(?i).*Trash.*$
e:(?i).*Trash$
e:(?i).*Starred.*$
e:(?i).*Starred$
e:(?i).*Com estrela.*$
e:(?i).*Com estrela$
# ==== CACHE FOLDER AND ALL ITS FILES AND SUB-FOLDERS
e:Data/profile/cache2/
# ==== EXCLUDING DATABASE
e:(?i).*global-messages-db.sqlite*$
# ==== EXCLUDING INDEXES
e:(?i).*.msf*$
The user in GitHub post called the folder as “local imap cache”. I don’t think it’s a cache, it’s a local version of your emails. The real Thunderbird cache is in Data/profile/cache2/.
And finally, about the “only local” files, they are not in the imapmail folder, but in the local folders folder (which is also inside the profile folder), and obviously must be backed up.
I don’t understand exactly, but why wouldn’t you want to backup “All mail”?
As a gmail user i send to archive (all mail) all the mails which i’ve read but i might come back to sometime, so not backing up that folder sounds rather dangerous to me.
Is there anything i’m missing?
Also, from what you’re saying i understand that that particular folder should be kept in the backup.
The all mail folder in Gmail is where messages that have no labels remain. Note that even inbox is a label. All messages that you received or placed in a folder are also in all mail.
Thunderbird translates Gmail labels as folders (thus generating duplicity / copies). A message with inbox label has two copies in Thunderbird, one in inbox and another in all mail folder. A message that you move to the Project A folder will have Project A label and will have two copies in Thunderbird, one in the Project A folder and the other in the all mail folder. And so on.
So the all mail folder is just a large MBOX file with copies of the messages, and in my case it is not necessary to back it up.
I’d rather not use all mail folder as archive. Really “cold” messages I move to the local folders folder. And I usually move messages from current projects to their respective folders (synchronized by IMAP). My inbox has only pending issues that I have to resolve or respond to.
I also understand that the way people organize email messages varies greatly. If you use all mail as archive then you should back up this folder, and in this case you may actually have a backup size problem because it is a large and constantly changing MBOX file.
Thanks for the in-depth explanation,I understand better now.
From all these details i think it’s better that i just won’t exclude anything Thunderbird related.
Even if you don’t want to deal with folders and MBOX files, you can exclude without any problem:
msf files (indexes)Generally speaking, although there’s no harm in backing up the ImapMail folder, it isn’t necessary. And for many users, backing it up could lead to a false impression that all of their mail is safe because the folder is basically only a temporary cache.
For example, let’s pretend that you have exactly two messages in your Gmail inbox. You launch Thunderbird, go to the inbox, and click on message #1, ignoring message #2. At this point, there’s no guarantee that the ImapMail folder contains a copy of both messages #1 and #2 because IMAP is an on-demand protocol.
Some IMAP clients will automatically retrieve a few of the most recent messages with the assumption that they are most likely to be read while others might download more depending on the particular settings. But the important takeaway is that in Thunderbird, the ImapMail folder is not guaranteed to be a complete copy of every mail folder in an IMAP account.
“All Mail”…
In Gmail, “All Mail” is basically a catch-all containing every message regardless of its label. It is literally, all mail. The only time it’s handy is if you use the message archive option in either Google’s Gmail web interface or mobile app. Otherwise, to hide it in Thunderbird, under Server Settings -> Advanced, set “IMAP server directory” to “[Gmail]”.
For messages that need to be retained indefinitely, manually (or automatically) move them over to the “Local Folders” branch which maps to the sub-folder “Mail\Local Folders” under the Thunderbird profile base folder.
I completely agree, this is a good point, the client configuration makes all the difference. In Thunderbird I use:
Exactly what I do, as described. 
Yup, I’ve got the same sync settings in Thunderbird. 
One thing to keep in mind is that Thunderbird doesn’t immediately grab all of the messages. It starts with 5 or so of the newest unread messages, followed by the older ones when it gets a chance. So, if a backup does include the ImapMail sub-folder, depending on when a backup runs, the backup might not match exactly what’s shown in Thunderbird’s message summary panel.
I forgot to mention Thunderbird’s mailstore formats…
Several years ago Mozilla started reworking Thunderbird’s mail storage backend so that it’s modular. Up until the past year or so, changing the mailstore format required going into the advanced configuration, but now it’s available under Server Settings -> Message Store Type.
The caveat is that the GUI only allows an immutable choice – the mailstore format must be chosen before the first sync, and once a choice has been made, it can’t be changed (it technically can be done, but involves the use of external programs to convert between mbox and Maildir formats).
So, why use Maildir instead of mbox format? Well, unlike mbox, Maildir stores each message as a separate file while each corresponding mail folder is a sub-folder. For large mail folders, it greatly speeds up access (and causes a lot less wear on flash-based media).
For backup programs that support deduplication, it also helps because there are no large mbox files with perhaps only a few small changes in the middle.
I’ve been following this implementation over the years, but in my case (Windows) unfortunately it’s not yet reliable. 
So I still keep the good old MBOX.
(Nonetheless, much better than PST… 
)
PST… argh!, absolutely. Anything I’ve used before, am using now, and probably will use is a better choice. 
As far as the remaining issues in maildir support in Thunderbird…
About 2 years ago I ran into speed and scalability problems with mbox mail folders for the operations staff at work. They send and receive lots of email – a single mail folder easily has more than 25 to 30 thousand messages, and on a typical day upwards of 100 to 200 new messages are added to various mail folders. (One of our staff’s Thunderbird profile is currently almost 32GB.)
With roaming profiles in Windows 7 Pro, it got to the point of taking as long as 15 minutes for the login process to finish (and that’s with a gigabit private LAN). With each workstation equipped with 16 CPU cores and 128GB of RAM it was fairly easy to rule out lack of hardware as the culprit.
So (with nothing to lose, and everything to gain) we decided to bite the bullet and try switching from mbox to maildir. During the setup, one thing that I did was to ensure that there were no damaged mail folders on the IMAP server, and we didn’t even try to directly convert existing mbox files to Maildir. Instead, we moved any original mbox files and had Thunderbird import them so that they were converted to Maildir on-the-fly.
Other than minor hiccups during the initial rollout (e.g., At the time, Thunderbird’s account settings UI didn’t have the message store type menu option), it’s been smooth sailing ever since. Not a single lost email or crash related to Maildir (staff use an internal trouble ticket system to log issues that need to be addressed).
Looking over the past year’s worth of bug reports on Mozilla, it looks like most of them might be due to client side issues (e.g., corrupted mail folders). Granted, Thunderbird should handle all errors gracefully, so there’s still work to be done.
For me, I’m very happy to hear that the intention is to use Maildir by default for at some future Thunderbird release.
Hello, If a user is expecting a disconnected imap account to be a long term storage, using the cached copies of the messages, they are living very dangerously. I don’t think that is a use case intended by the developers, but maybe I’m wrong.
I would just worry about a new version of thunderbird deciding to reload the cache because of a local storage change of some sort.
If the user copied message from their imap account to their local folders, then I would have the expectation of the the data sticking around.
In my use case, I run the mail server, so I know that there are multiple backups of the server mail store already, so I’m not concerned about the mail server backups not being trustworthy. But I could see the logic in not trusting a free provider like gmail/yahoo mail to have an error.
Besides, I’m not sure how much you can actually do with a disconnected imap cache. It looks like you can copy/move messages from a disconnected imap cache to local folders. Thunderbird must keep a log of those actions, for the case when a disconnected imap cache gets re-connected and all the actions need to be applied to the server.
What if the exclude for the imap cache dir was included, but commented out, so it is easy for someone that wants to leave it out to exclude it.
The feeling i’m getting from all these examples is that i don’t wanna mess with anything Thunderbird profile related.
Thank you all for your input and opinions on this matter, but i have decided I won’t exclude anything and leave the user do as it pleases.
I’d much rather waste storage space than make the user lose data.
