Malformed message id (I don't think it is a bug in MimeKit)

[Sicos1977]

I used the latest version.

We received an e-mail with what I think is a malformed message-id, because of that mimekit is returning null for the e-mail id.

From: XXXX | XXXXXXX <[email protected]>
Reply-To: Schade | XXXXXXXXXXXXXX<[email protected]>
Message-ID: <#D26928#O108924#S 20094.9b3890045df8d8ac0132ba357fd8651d@habithuis.fasterforwardelements.nl>
X-Elements: <#D26928#O108924#S 20094.9b3890045df8d8ac0132ba357fd8651d@habithuis.fasterforwardelements.nl>
DD-Elements: <7354ec2df7e0cafd72bf99f4d6353441@habithuis.fasterforwardelements.nl>

Is there a way to get the message id even when it is malformed?

[jstedfast]

Ugh, yea, that is invalid syntax :(

I guess the easiest solution would be to use message.Headers[HeaderId.MessageId].Trim () which should get you the value <#D26928#O108924#S 20094.9b3890045df8d8ac0132ba357fd8651d@habithuis.fasterforwardelements.nl>. If you want it in the same format as it would be from message.MessageId, just trim the <>'s.

I wonder how common this is... I tried to make the Message-ID parser more lenient but I guess it needs to be even more lenient - as in something ridiculous like:

int lessThan = header.Value.IndexOf ('<');
int greaterThan = header.Value.IndexOf ('>', lessThan + 1);
msgId = header.Value.Substring (lessThan + 1, greaterThan - (lessThan + 1));

[Sicos1977]

I use the message id as a unique key in a database to identify if I have seen an e-mail before so I'll try your solution with message.Headers[HeaderId.MessageId].Trim () and if that does not return anything just generate a random guid because I need something in the database because the field is not nullable.

[jstedfast]

A few points:

1. You are probably already thinking this, but just to make sure, you'll want to keep the format of the string in the database consistent, so if you've been using message.MessageId up until now, you'll want to trim the <>'s as well, for consistency.
2. The Message-Id header is not guaranteed to be globally unique and you'll want to be careful about making the assumption that they are. They can be a useful tool in helping to narrow down potential matches, but design your software such that if a hacker knows the Message-Id of an incoming important message, he/she can't just send you another message using that same Message-Id in order to deny that message from getting to the user (by tricking your software into thinking that it already has that message based on the Message-Id value).

I was going to try taking a closer look at the code that parses Message-Ids before I close this to see if there isn't a way I can improve the code to handle this case.

I am wondering if perhaps that space could have come from some folding logic in the sending software because the value is so long and if that is the case, it's arguable that MimeKit should better handle that.

If you look at the raw original header of the message, does it look like this?

Message-ID: <#D26928#O108924#S
 20094.9b3890045df8d8ac0132ba357fd8651d@habithuis.fasterforwardelements.nl>

(Either way, I think I'd like to try and handle this case because this scenario is at least plausible even if not true in your specific case)

[Sicos1977]

I know that the message id is not unieke enough because everyone can just generate something but it is unique enough for me. The primary key in my database is also build up with more then one field so just not only the message id. Because someone can sent the same e-mail to more then one e-mail adress and in that case all those e-mails have the same message id.

I mainly use the message id to track logging and things like that because I want to know if the e-mail was sent to more then one e-mail address. I also could just generate a random guid as an id but then the logic fails to match it with the same e-mail on another e-mail address. So in my case this was the best solution for me.

The header was copied from the raw headers. I just opened the EML file in notepad on windows and got it from there.

[jstedfast]

Okay, thanks for the info.

[rogatec]

Hey @jstedfast , I encountered a similar problem with a MessageId.

The MessageId was null, but when the customer checked the E-Mail in Outlook, it had a MessageId in the header, but with no closing > which seemed very weird.

Is there a way to get the (faulty?) MessageId - maybe by parsing the MimeMessage.Headers ? Is this way "more failsafe" when using the MessageId for logging - comparison features?

[jstedfast]

@rogatec

What you could try is something like this:

var messageId = message.Headers[HeaderId.MessageId]?.Trim()?.TrimStart('<')?.TrimEnd('>').Replace(" ", string.Empty).Replace("\t", string.Empty);

I have been hesitant about reducing MimeKit's Message-ID parser to something so brutish, but I'm seeing less and less possibility of having an actual Message-ID parser that actually works in all cases, no matter how lenient I make it, unless I reduce it to something like the above code snippet. It's depressing.

I think I'm going to have to do something like that because I'd really like the MimeMessage.MessageId property to be reliable.

[rogatec]

Hey @jstedfast thanks for your advice and thoughts. Another question comes to my mind, have you thought about making the MimeMessage object available for testing through an interface?

I dislike the idea to write a wrapper class with an own interface around it, because I wanted to test the rare case where the MessageId is null and the Setter id the property disallows a nullable setting.

[jstedfast]

@rogatec it is something I've thought about. Can you submit a new issue to track that? I agree it's worth doing.

I've been thinking of changing things to make use of Microsoft.DependencyInjection and interfaces would be a good step toward that.