is the account IMAP or POP?

The accounts are all POP3

re : The worst is sent mail where the first duplicate is a 2020/08 message and the oldest 2013/07 has 32 copies. Let's tackle this one first. The question is this: Do you really have - oldest 2013/07 has 32 copies or does the index think you have 32 copies of that email?

In Thunderbird

• Help > Troubleshooting Information
• Under 'Application Basics' section - about 8th in list is 'Profile Folder' - click on 'Open folder'

A new window opens showing you the contents of your 'profile name' folder.

• click on 'Mail' folder
• click on the pop account name folder for that account.

You will see a 'Sent' file - it has no extension as it is an mbox file. (Not the 'Sent.msf' file.)

• Open the 'Sent' mbox file using a text editor like Notepad or Notepad++

The emails are listed in the order they were stored, so the oldest will be at the top. Hence why I suggest you started checking for that old email. Look for the email dated 2013/7 which apparently has 32 copies. Carefully, check down through the document to see if there are duplicates of that email.

Then check to see if there are any duplicates for the newer email dated 2020/08, which will be nearer the bottom of the file.

If there are no duplicates:

• Exit Thunderbird now
• Close the 'Sent' file.
• Delete the 'Sent.msf' file
• Start Thunderbird.

Report on whether you still see the duplicates for emails in the 'Sent' folder.

There are indeed 32 copies of the message and they are spread though the folder. The first appears at line 42,192 and the last (presumably most recent) at 11,902,260.

I just looked through all of the folders to see if there was some sort of pattern but detected none. One folder has only the very first entry duplicated (2010/03/04), two have no duplicates at all and the first duplicates appear anywhere from 2021/01/03 (the latest entry) to 2017/02/14.

It's as if each folder is being replicated and written back into itself on an individual basis. This could be result of clumsy coding (don't ask how I know) but that hardly seems likely here.

I've heard of real duplicates, but usually it involved just the Inbox and a 'popstate.dat' file gone awol. This real duplicate problem as observed by you confirms that the index file is not wrong, so the 'repair folder' option is not going to work.

In the 'profile name' folder - pop account name folder for that account. Do you see just the one 'popstate.dat' file or are there additional ones with perhaps a number eg: popstate-1.dat ?

As you have a POP account, in Account Settings > Server Settings, have you selected the option to 'Leave messages on server' ? Do you have this duplicating problem in the 'Inbox' folder or is that the only folder that seems to have escaped the issue?

Do you have 'Message Filters' that auto sort newly downloaded emails into all your various folders?

Do you manually compact individual folders or is this something you have set up to auto do when prompted a threshold has been reached? Or do you only tend to manually compact the Inbox, Junk and Drafts ? How frequently do you compact folders? Is this compacting thing something you have never done ?

To understand your setup: As you use POP accounts can you confirm whether you see each individual pop account with it's respective folders in the left Folder Pane OR have you set up those POP accounts as deferred accounts that use a 'Global Inbox' which is usually the 'Local Folders' mail account, so all incoming mail from all pop accounts appears in 'Local Folders' mail account. This would mean you do not see all the various pop accounts as separate accounts in Folder Pane.

There is only one popstate file in each of two accounts.

Messages are not left on the server. The inbox is rarely more than one page and the oldest message 2020/09/04 has not been duplicated. Five folders have escaped, one dating back to 2012 but these folders have little to no activity

There is one sorting filter and one delete.

Compacting is done ad hoc, most often when hoping that it would now fix the duplicates. I just compacted the inbox but can't recall having ever done that before since it has so few entries. Drafts is empty and Junk gets emptied on an irregular basis along with Trash. I've never noticed (looked for?) duplicates in Junk but Trash does get some. They don't get compacted.

I only see the one account on the left. The second account is for my new ISP and is not used except for system notices. I have had it set up for other accounts in the past which each had an entry and folders.

Even though the Drafts and Junk folder look empty you would not know unless you had the 'Folder Pane columns' option enabled. Deleted or moved emails get a 'marked as deleted' tag and hidden. They may appear in other folders eg: Trash, but in reality the original copy would still lurk in the mbox file. Compacting removes those hidden and marked as deleted emails, thus cleaning up the file.

How to enable 'Folder Pane Columns': If you are using version 78*: Menu app icon > Customise > Layout > select 'Folder Pane Columns' See first image below

If you are using version 68* : Menu app icon > Options > Layout > select 'Folder Pane Columns' See second image below

Then you can select the 'Size' column. see third image below I've posted some images below so you can see what to do.

If 'Drafts' and 'Junk' have any size: Right click on folder and select 'Compact'.

I'll post additional message on what to do next.

As it is unknown at this point why you are getting duplicates, we have to assume the folder could be corrupted. So do the following to ensure you do not lose wanted emails. It is a bit of a lengthy process, so will take some time.

Starting with the 'Sent' folder:

• In the 'Local Folders' mail account, create a new folder and call it 'CopySent'

In the POP 'Sent' folder, delete all emails you do not want. You will be left with all emails you want to keep.

• Select first email
• Use 'Ctrl+A' to highlight all of the remaining good emails.
• Right click on highlighted emails and select 'Copy to' and choose the 'CopySent' folder in 'Local Folders' mail account.

When the copy has been done:

• In 'Sent' folder - make sure all emails are highlighted and click on 'Delete'.

As you have enabled the 'Folder Pane Columns' - Note the size of that 'Sent' folder.

• Right click on the 'Sent' folder and select 'Compact'

The size should say nothing and you will now have a new empty 'Sent' folder.

• Then Empty the Trash - right click on Trash/Deleted folder and select 'Empty Deleted'.

Now perform the same routine to get all the copied emails back in the POP 'Sent' folder - assuming this is desired. If you want to leave them in that 'CopySent' folder it is ok and you can now use the nice clean 'Sent' folder for new emails, so you can detect if weirdness is still occuring. If you wanto to put them back do this: In 'CopySent' folder :

• Use 'Ctrl+A' to highlight all of the emails.
• Right click on highlighted emails and select 'Copy to' and choose the 'Sent' folder in POP account.

When the copy has been done:

• In 'CopySent' folder - make sure all emails are highlighted and click on 'Delete'.

As you have enabled the 'Folder Pane Columns' - Note the size of that 'CopySent' folder.

• Right click on the 'CopySent' folder and select 'Compact'

The size should say nothing and you will now have a new empty 'CopySent' folder.

• Then Empty the Trash - right click on Trash/Deleted folder and select 'Empty Deleted'.

This same exact process then needs to be done on other folders containing duplicates. Obviously the POP folder name is different - but you can reuse the Local Folders' 'CopySent' folder as it has properly emptied. However, If you chose to leave all those copied 'Sent' emails in the 'CopySent' folder like an archive, you would need to create a new folder for the next batch and give it similar name 'CopyX', then use for the next lot of copying etc.

Thank you. I will be able to devote some time this evening and will report back.

Well it sort of worked. Not wanting to spend days manually deleting the copies I opted to copy the entire Sent mail to a new folder. It looked like it was set to take a couple of hours so I left it working. It succeeded in the copy but then crashed. It also deleted the entire content of Sent. The new folder was not created in my Profile E:\Mozilla\ThunderBird\Profiles\at66ajgl.olddefault\Mail\mail-in.kent-1.net\Sent E:\Mozilla\ThunderBird\Mail\Local Folders\fixsent

Now I can, as you suggested, use my squeaky clean Sent folder and wait to see if it screws up again and perhaps leaves some trace.

Does Tbird choke on external changes to the data? I've a really good text editor which could do a lot of the grunt work cleaning up the folders.

It should not have crashed, but I did wonder if you have an Anti-Virus product scanning opened files which may have contributed to the problem. https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues

It is often recommended that you should make the mbox files exempt from scanning. See pros and cons: http://kb.mozillazine.org/Email_scanning_-_pros_and_cons

You can delete/edit emails manually from the mbox files, but you would need to be careful, so that you do not edit incorrectly and end up with a corrupted file. You must exit Thunderbird first.

Typically, each email will start with these lines. Example:

• From - Tue Jan 22 11:27:18 2013
• X-Mozilla-Status: 0001
• X-Mozilla-Status2: 00800000

I bit the bullet and deleted the duplicates from the old sent file. Perhaps when my eyes un cross I'll get to the other folders. I'll try to monitor the Sent folder to see if the problem re-appears. Thanks for your help.