Friday, April 23, 2010

Can anyone explain this security failure in Domino?

I've never seen this problem before and I hope I never see it again.

A customer's branch office based in Sydney, Australia sends out a daily industry newsletter to their customers but for reasons specific to their industry they doesn't want any of the customers to realize who the other customers are. Consequently the author mails the newsletter to himself with all of the customer addresses in the BCC field.

Two days ago he mailed the newsletter out as normal but it bounced at one address (let's say for 'Mr Smith') because Mr Smith no longer worked at the target organization. After that all of the other 100+ customers listed in the BCC field received the Non-Delivery Report for Mr Smith addressed to themselves, and since they were given the original email encapsulated within the NDR then they could see the contents of the BCC field and thereby understand who all of the other 100+ customers were. So somehow the Router task had taken the contents of the BCC field and used that to address the NDR.

The customer had been getting random corruptions in their file on a monthly basis for over a year but Lotus support hadn't been able to determine the reason for this. Recently the corruptions had been hitting mail files also but Fixup never found a problem. I have a sneaking suspicion that the files weren't actually corrupt, and that a wayward Router Task is somehow to blame for all of this, but the server has been taken up and down more times than a Bride's nighty and there is still no end to the problem. The next step is to completely reinstall the server and patch it to R7.04 but that still doesn't answer the question of what happened.

The customer is standardized on Notes across the world and is unlikely to abandon the platform. They are looking at upgrading to R8.5 later this year, but for now they would like some reassurance that the problem won't occur.

Anyone seen this kind of problem before?

The server is an unclustered R7.02 FP3 running on Windows 2003 server. A PMR has been raised for this issue and if any Loti wants to investigate it further then I'm happy to give them the reference.


Henning said...

Could it be that you are saving the newsletter directly into the files (with an Agent for example)?
As a workaround you could produce individual messages. This would increase the mail volume but prevent that someone gets other email addresses and it has the advantage that you could further personalize your message.
The most common reason for mailbox corruptions are third party tasks like anti-virus, backup or spam protection tools but I am sure you already know this.
Have a nice week-end.

Graham Dodge said...

@Henning - The email was sent as a normal email and the logs recorded the delivery to the intended recipients. Everything was fine on the outward leg and the issue only occurred when the NDR was generated. The main question is how does the router mix addresses from the BCC field with the address of the original sender. Sure there are workarounds to confirm the problem can't happen again, but how the heck did it happen in the first place and what does it tell us about the security of the BCC field.

Anonymous said...

My guess is that all bcc recipients are somehow added to the original message as the 'From' address.
I would check the routing path of the ndr messages. If the ndr's are send to all bcc's by the ndr mail server, there is a problem with the outgoing message. If the domino server sends out the ndr to all bcc user, the problem starts at receiving the ndr. Good luck.

Henning said...

Thank you for the update. This is so weird that it is hard to believe that not a third party product is involved in the process. If it is a flaw in Domino then I keep wondering why it has not been discovered before.
I hope that the specialists at Lotus will have a solution for you very soon.

Lotus Evangelist said...

just a thought, check the configuration settings documents for domain * and the individual servers.
The routing/smtp has a subsection on what to do with returned email.
Suggest changing it to NOT send the message, rather than report back if there is an issue.
That way it stays in the file for an admin to clear at some point.
Just a guess.

Graham Dodge said...

Clarification: The server has been working for many years and has delivered probably hundreds (thousands?) of NDRs in the past with no problem. The customer has quite limited Notes skills and hasn't changed server settings for many moons

BP said...

By chance, did the sender of the mailout happen to suffer a corruption of their mail file around this time?

Palmi said...

Well That is why i do only recommend sending newsletter one at the time in to: field. using LS : Where this situvation can´t happen - Graham let me know if you like to go that route for your customer.

Jose Zaldivar said...

We do send individual emails 1000's of users every so often. Via LS works best as you can personalize emails. I also use the technique of putting the mail in the directly. Chris Toohey in the has a cool apps to send emails that way.