How can I resolve problems writing e-mails to folders?
I am running Thunderbird on Windows 7. I have several e-mail accounts set up in Thunderbird, and I have rules set up to move all the inbound mail to the Inbox folder in a separate set of Local Folders. This has worked great for as long as I have been using Thunderbird. A few days ago, I noticed e-mail wasn't moving from the various account inboxes to the Local Folders inbox. This was presumably the first time this happened, and Thunderbird had been open for 4-6 hours (I generally let it run for about 9 hours a day). I was running 24.4.0, and closing and re-launching Thunderbird would solve the problem. The first thing I did to try to fix this was update Thunderbird, so now I am on version 31.1.2 (and have enabled automatic updating).
I experienced the same issue the next day, so I did some searching thinking maybe I needed to repair the database. The first suggestion I found recommended deleting global-messages-db.sqlite and then letting Thunderbird rebuild it, watching progress in the activity manager. I did this, and the next day the problem still occurred. The next suggestion I found was to delete all of the .msf files. I did this, and then went through all my folders to get them rebuilt. The problem has reoccurred since then as well, and I'm not sure what else to try.
For the record, the issue is occurring more frequently, and I am also seeing issues saving sent messages now (I have lost at least three sent messages because it sticks on the saving to Sent screen until I cancel and don't retry), so this isn't related specifically to the Inbox folder. In fact, once the issue starts, I can't move any e-mail until I restart Thunderbird. This includes automatic moves per rules as indicated at the beginning of this post and manual moves for the following:
-From account inboxes to Local Folders Inbox -From Local Folders Inbox to Local Folders subfolders -From compose screen to Local Folders Sent folder (my default sent save location for all accounts)
Please help me resolve this.
All Replies (20)
Wish I could but I've been using Thunderbird since early this year and have always had this problem. There are lots of threads about it. Titles vary as people first see it from different angles - hang on writing to sent, hang on writing to drafts, filters stop working, junk stays in inbox, drag and drop is ignored, etc. Some people experience it multiple times a day, some every few days, every few weeks etc. For me it's highly variable - I might have a problem within hours or over a month after starting Thunderbird.
Seems to get no attention from the developers - at least that I can tell. Or maybe they care and just can't track it down.
There's a tendency in the Thunderbird world to blame environmental stuff such as AVs but I've seen reports on Windows, OS X, and Linux with a variety of AVs and sometimes no AV.
In a couple of threads, I've tried to get people to talk about their configurations and extensions to attempt to find any commonality but haven't had much success at that. (I use Enigmail, AllowHTML Temp, and Quicktext myself.) People often say they fixed it by X where X is never the same twice, but usually they post again later that it came back.
I also have several accounts and I use a global inbox in my local folders. Maybe there is some relation there. Sometimes I wonder if all the options in Thunderbird are considered fully supported. Another thread said that his TB had been working until a few weeks after he switched to a global inbox so there might be something to that. But then, it seems like it was working for you for a long time that way??? Then again, like I said before some people get this fixed and it comes back.
Thanks for the response. I have been running Thunderbird since April and initially set it up with a global inbox. It seems like the problem is nearly global for me when it occurs. Fortunately, e-mail is being written to the account inboxes, but it isn't being written anywhere else that I have tried. I used ImportExportTools to run a backup before I tried deleting the .msf files, and that backup is ~780MiB. That includes e-mail all the way back to 1997, which was imported from an Outlook PST. Looks like it doesn't include Sent items, so oops. Fortunately, I haven't lost anything that I know of. I'm not sure what configuration information you are referring to, as the items you mention don't sound like addons.
The add-ons I use are: Enigmail, AllowHTML Temp, and Quicktext. Any commonality there?
Since you've always been running with a global inbox and haven't seen the problem until now, that probably clobbers another thread/poster's theory that switching to a global inbox was possibly the cause of the problem. (He had not seen the problem until he made the switch - I have had the problem all along but I have used a global mail box all along.) So for a while I was thinking that moving away from the global inbox could salve the issue but it sounds like that's another wrong guess now.
There are lots of other settings in TB, though it's not obvious where to even start. For example, I use POP, don't check mails on start up or automatically, have about 7 email filters. I also moved my profile directory from the default location. For awhile I was convinced that having the activity manager open solved the problem since I went for three weeks straight without an issue by always having that open. But I was fooled by the randomness of the issue again and it came back.
Oh - I agree - once the problem occurs - anything that involves moving messages, saving messages etc. fails - until you restart. I believe that's true for everyone. It's just that the thread titles vary widely because people tend to report the first symptom that they notice.
No, I wasn't using any of those addons. When the problem started, the addons I had installed were Clean Subject, Edit E-Mail Subject, Lightning, and Show All Body Parts. I had been using all of those addons all along, and none of them had been updated since April, either. Only Thunderbird and Lightning have been updated since the problem started. I also added EditSender and ImportExportTools, but they are not relevant since they were added after the fact.
For the record, each of my accounts has one filter to match all e-mail and move it to my Inbox folder in Local Folders. I'm not sure this is the same as your configuration since you use the word Global Folders like it might be a setting. IIRC, I manually set this all up because I never came across such a setting.
I use POP, DO check on startup automatically, and have one filter (as previously mentioned) for each of my 6 accounts. I have NOT moved my profile directory away from the default location. It certainly seems like almost nothing we have discussed this far matches except the symptoms. POP accounts match, but if only the Local Folders are affected, which may be the case since the POP accounts still download the e-mail and store it in their local Inboxes, then the account type should be irrelevant. If e-mail account folders could end up in the same scenario, then using IMAP might work around that in 2 ways. 1) Not using cached mode would mean Thunderbird might not need to write whatever is causing the issue. 2) Even using cached mode, the local copy may be stored differently. Regardless, who wants to use IMAP? I would if I had one really good e-mail account to use for everything, but I wouldn't want several IMAP accounts, and I wouldn't want to use IMAP with the likes of Google, which refuses to follow standards that work great (even using POP with Google is annoying because of their non-standard behavior, so I try to keep inbound gmail at a minimum).
This is the global inbox that I use:
http://kb.mozillazine.org/Global_Inbox
Yes, it does sound like it's hard the pin the problem on add-ons or any obvious collection of settings.
The end of the first paragraph on that page confirms it is not what I'm doing: Accounts that use the Global Inbox will not have their own folders displayed in the folders pane; instead, all you will see is the single set of folders in Local Folders.
All of my accounts and their inboxes are visible in the folders pane as I previously alluded to (e-mail goes to their inbox instead of my Local Folders inbox once this issue is triggered). So with your setup, does Thunderbird stop downloading mail when the problem occurs, or what? I certainly hope it doesn't download the mail but fail to write it and subsequently delete it from the server...
I have never seen anything like you describe. What happens is it downloads to the inbox, then all rules and junk mail filtering out of the inbox fail. So I stop TB, restart TB, run filters on the inbox and then run junk mail controls, and it's back to normal.
Apparently the code that writes to the inbox originally is unrelated to the code that writes to all the other folders ??? since it's only the latter that routinely stops working.
Well, no, the code that writes on receipt must be different than the code that writes moves. My folder pane looks like this (but with more accounts, all collapsed)): Account1 +Inbox +Junk +Trash Account2 +Inbox +Junk +Trash Local Folders -Inbox +Drafts +Sent +Archives +Trash +Outbox
When things are working right, e-mail is received to Account1\Inbox and then moved to Local Folders\Inbox by a catch-all filter. When the problem occurs, new e-mail remains in Account1\Inbox. IOW, mail cannot be written to Local Folders\Inbox in my case (when it would be moved from elsewhere by a filter), but can in your case (when it is placed there immediately upon receipt). This seems like something that would help narrow down the scope of the problem if we knew who to report it to. Presumably there would need to be a way to reproduce it in order to file a bug report, though, and so far, there is no obvious cause.
ETA: You may have basically said the same thing with different semantics. In that case, don't take my correction personally. Also, I should have mentioned this in the OP, but I have been compacting the inbox immediately after each and every time I delete any e-mails from it and still see the issue occur, so between doing that, deleting the MSF files, and consistently seeing the issue resolved by restarting the software, it seems unlikely that this is related to any type of corruption in the folders. However, I haven't tried the "Real fix" under "Problems when compacting" found here: http://kb.mozillazine.org/Compacting_folders
Have you?
Modified
"ETA: You may have basically said the same thing with different semantics. In that case, don't take my correction personally. "
Yes I think you're saying the same thing. As for the "real fix" that was recommended to me the first and second times I brought this up. It did it, but I also asked if TB is really that "fragile" that this generally needs to be done 20 days after installation and then roughly a week after the first time it was done. The person I was asking seemed to get kind of annoyed at me for not originally telling him that my TB installation was so new, but he didn't have anything else to suggest anyway.
Likewise I'm always getting told to compact whatever folder it happened on but sometimes the folder is zero bytes and zero messages - but I compact it anyway to just to cover all the bases. As far as I can tell "the real fix" works just as well as simply restarting TB for this issue - and probably because you restart TB while doing the "real fix". BTW I don't mean to imply that the "real fix" doesn't work for anything else. I just haven't seen it working on this one. Also the problem reoccurs too often for that process to be a practical solution in my opinion.
If they can't track down the bug, it would be nice to have a button or watch dog timer/detector type thing that:
Restarts TB Runs filters Runs Junk Controls
Then at least you could get back to business very quickly.
The "real fix" is for db corruption, and it sounds like this issue is not related to db corruption, although db corruption was my assumption before I tried anything, and it obviously seems like a good assumption to other people as well. Your ideal workaround sounds potentially usable (and perhaps like something an add-on could do), but for my situation, simply switching to Global Folders would at least make the issue a lot less of a nuisance. Regardless, if there were some instructions for gathering data for debugging, I'd be willing to give that a shot, but since there isn't actually a crash, I'm not sure how useful basic instructions would be since they would most likely be for gathering crashdump type information.
This recently started happening as soon as I start Thunderbird sometimes. I've got a theory that I don't want to test (because I don't want to disable automatic mail retrieval), but thought you might want to try (since you don't have automatic retrieval when you first start anyway). It seems to me like this is happening when more than 2 accounts are receiving mail at the same time. As I stated, I have 6 accounts and they all check when Thunderbird start. 4 of them almost never get e-mail, but one of those 4 has started getting e-mail recently. Since I don't use global folders, I see the account folders count up, and then the mail moves to the Local Folders Inbox. When I'm seeing 3 accounts count up in the morning, the first two to finish move, and the third one does not. It could be more difficult to reproduce, because the issue could actually be happening immediately after two accounts finish receiving mail at exactly the same time (maybe to the second?) instead of when 3 are downloading at once, or it could even just be a fluke.
So the other day, I hit send right when Thunderbird automatically checked for mail. The received mail was moved to the Local Folders Inbox at the same time the sent mail write was attempted to my Local Folders Sent folder. The sent mail was unable to be written. I thought each folder had its own database file, so it seems odd that it would matter that two different ones were being written at the same time, however, anytime I see this happen (including again this morning right after the software was started), it definitely looks like it is caused by multiple writes needing to happen at the exact same time. Considering that I am 90% certain what leads to the issue, and it's not something easy to reproduce on demand, is there somewhere else I should go to get help trying to get some debug information that might actually help developers fix this if they are so inclined?
Further evidence that the issue is related to trying to perform multiple writes at the same time:
Today, I hit send on an e-mail, and then proceeded to move an e-mail from the Local Folders Inbox to a subfolder of said Inbox without thinking to wait for the e-mail to send and write. The e-mail did send and write, but the e-mail I was trying to move from the Local Folders Inbox would not move. No new e-mail arrived when this happened (to the account inboxes or Local Folders Inbox), which also indicates that the receipt of new mail is not directly related to the issue (if receipt of new mail was the trigger, the sent item would have failed to save because it would have already been impossible to write due to a previous receipt).
Modified
Well, multiple writes is not the root cause. I lost my copy of a pretty big outbound e-mail today and decided to change all of the mail filters to manual in order to avoid multiple simultaneous writes to the Local Folders structure (because if I'm sending, I'm not running filters, and if I'm running filters manually, it's for one account at a time and I'm not sending). The issue occurred pretty soon after I made those changes (maybe a total of six actions [that would write to the Local Folders structure] later). Aside from these forums, which apparently aren't attracting the attention of anyone who can work with me on this, is there any other advisable way to get support for this issue?
I doubt it's the sqlite database; as far as I know, that's for the global search function. It's not a message store as such. Your messages are stored in mbox files and indexed by the accompanying .msf files. (e.g Inbox, Inbox.msf)
With POP, yes the code for getting mail via POP from the server will be quite separate from the filter/move stuff, since the latter is totally local. (IMAP would be quite a different story, since you're remotely managing the message store on a server somewhere in the cloud.)
I think all Thunderbird users occasionally see warnings about writes failing, usually because the folder is committed to something else, often compacting. But it's not usually a drop-dead problem.
How big are your profiles? Do you hive off old stuff into Archives, or have you got ginormous message stores? Are your profiles located anywhere with high or variable latency, such as a SAN drive? Are you trying to share profiles with another user or installation? What AV is in use, and is it allowed to scan TB's profile folders? How many accounts are you using?
I suspect that a large proportion of TB users are exclusively IMAP users these days; that's certainly the default protocol, and TB will choose it when available. The fact that you're both exclusively using POP may be significant.
Thanks for the response and comments / questions. Database may not be the right term, but I was referring to the Local Folders file structure (including mbox [presumably those are the files with no extension] and msf files) when I said database.
Regarding the comment on warnings about writes failing, there are not warnings, just failures with one exception. That having been said, writes only fail to create e-mail records, not modify them. To be more specific, once the problem starts, no mail can be written to ANY folder in the Local Folders "account". However, mail can be written to the folders in the various POP accounts, and mail can be marked read/unread and deleted in all folders in the Local Folders "account". The only time a warning is popped up is after sending mail. The sending progress bar finishes and switches to a saving progress bar that will remain indefinitely. Only hitting cancel or closing the e-mail will pop up a warning that there was a problem writing the e-mail. Also, received mail is written fine. As I write this I realize I may be wrong about this only affecting the Local Folders "account" since I am using filters to move mail instead of using Global Folders as another poster early on in this thread with similar problems is doing, the mail writes fine to the other account folders. However, the other user I mention states that his incoming mail writes fine to his Global Folders and he simply cannot move it. I have never actually tried manually creating/moving messages in the various account folders, so that may not work either, and the problem may be across all accounts vs. just the Local Folders "account". I will try to remember to test that out and update this thread.
Regarding the size of my profiles: C:\Users\<USERNAME>\AppData\Roaming\Thunderbird\Profiles\<Default>\Mail\Local Folders\ is currently 888MiB. However, the Inbox file is only 43.6 MiB. I don't use the Archives, but I do have a lot of subfolders that range from <1MiB to >300MiB. In case that isn't actually what you needed to know, the other accounts (each in C:\Users\<USERNAME>\AppData\Roaming\Thunderbird\Profiles\<Default>\Mail\) are <1MiB each since mail is received into them and then immediately moved out to the Local Folders "account". Finally, C:\Users\<USERNAME>\AppData\Roaming\Thunderbird\Profiles\<Default>\ is 1.04 GiB.
Profiles are stored locally as can be seen from the path. Locally is on a RAID10 SSD array.
Profiles are not shared in any way (not even to a separate OS in a dual boot environment).
AV is Comodo, and I did just exclude C:\Users\<USERNAME>\AppData\Roaming\Thunderbird\Profiles\<Default>\Mail\Local Folders\ to see if that will make a difference. I will update here if it does not, but may forget if no one else responds to this thread and it does solve the problem.
Counting Local Folders as an account, I am using 7 accounts. The other 6 are POP accounts.
Regarding IMAP vs POP, I will concur that POP usage could be significant since it is less popular, but that is why I am trying to offer to do anything I can to help get the underlying bug fixed if there is one. That is to say that IMAP messages are presumably stored locally as well, and the problem with POP messages being stored locally would more likely imply an unfixed problem in the POP programming due to less maintenance/use. If I am wrong regarding IMAP storing messages locally by default, then it could be a problem more core to the system that would be just as easily triggered in an IMAP environment where the setting is changed so the mail is stored locally, and if I am right, it could still potentially be triggered in an IMAP environment where mail is moved to a Local Folders "account" on a regular basis. At that point, the programming issue still might involve local storage procedures instead of POP specifically, but that area of the program may also experience much less maintenance/use (since receiving mail works even while the issue is present, the writing of downloaded mail is clearly not affected even though creation / moving of already local mail clearly does not work, and all in-account IMAP actions may go through the procedures for downloading mail vs local mail even when a copy of the data is stored locally).
I can't address many of your points right now, but I am almost exclusively an IMAP user, with a lot of stuff copied to Local Folders and a fairly deep/complex system of folders and subfolders (this is to provide me with a local cache of messages when offline or away from the office and thus disconnected form the company email server.) I certainly don't experience the serious kind of hang ups you describe, which kind of suggests that Local Folders per se aren't necessarily the problem area. :-S
Large folders can be problematic. A TB folder such as Inbox is implemented as a single mbox file, and so a big one can hit limits on file size or message indexing. This is partially why we urge users not to let messages accumulate in the Inbox. But none of your folders would appear to be worryingly large. Just for context, 2 GB and 4 GB have appeared as limiting sizes in the past due to underlying OS-set limits, but I'm told that these should no longer be relevant, at least in TB. I still think it's less than optimal to expect TB (or any program!) to manage large files as a matter of routine.
I've never had Thunderbird set up to check for new emails automatically, and yet I have always had this issue so I don't think that's it. It's nice to see someone trying to figure it out though.
As for large folders, I've had this problem within weeks of installing Thunderbird in a POP set up so most of my folders had tens of messages in them. On some occasions it has hung while saving to drafts when the drafts folder was zero bytes.
I haven't seen the issue in three days, but I still expect to see it come back. That having been said, while I noted that I excluded a specific folder in AV, I also compacted all of my POP account inboxes. One was 10MiB even though it was empty. I know we've already discussed compacting the inbox, but how frequently do you do it? I mean, if you only do it after the issue occurs, then maybe doing it every time you move or delete e-mails would prevent the issue (granted that neither elegant nor a solution). Also, since you use global folders, do you have invisible inboxes for your other accounts that cannot be compacted (because they cannot be seen)? I'm guessing you do not, but to be more specific, do you have any folders other than maybe "Global Folders" and "smart mailboxes" presumably named after accounts/servers in C:\Users\<USERNAME>\AppData\Roaming\Thunderbird\Profiles\<Default>\Mail\? If you do, and they contain inbox files, what is the last modified date on those files?
After I read all the open tickets and some other stuff, the stability of THunderbird is clearly suspect. I'm keeping it for some small accounts but I'm going elsewhere for my outlook.com. I see too many issues in messages that match up a lot of the problems I have as well as the sync issue!