NTLM AUTH failure causes subsequent PLAIN AUTH attempt

[jimjimbojimbob]

Hi,

Using 1.16.1 of Mailkit

This is the same issue as #265 but as we are using 1.16.1 I don't think this is fixed?

When connecting to a customers hosted Exchange 2013 server we see authentication errors. Error reported is

Server Unavailable. 15
at MailKit.Net.Imap.ImapClient.Authenticate(Encoding encoding, ICredentials credentials, CancellationToken cancellationToken)

I know absolutely the username/password are correct, and that the certificates on the server are correct as I have verified this from a Linux server using the 'openssl s_client' and 'mutt' commands.

Protocol log

Connected to imaps://hostedexchange.example.co.uk:993/
S: * OK The Microsoft Exchange IMAP4 service is ready.
C: A00000000 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=NTLM AUTH=GSSAPI UIDPLUS MOVE ID CHILDREN IDLE NAMESPACE LITERAL+
S: A00000000 OK CAPABILITY completed.
C: A00000001 AUTHENTICATE NTLM
S: +
C: TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA=
S: + TlRMTVNTUAACAAAAGgAaADgAAAAFAoEC/77SWeKNpNUAAAAAAAAAAO4A7gBSAAAABgOAJQAAAA9DAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAAIAGgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAAEAHgBDAEwATwBVAEQALQBFAFgAQwBIAEEATgBHAEUANAAEACYAQwBVAFMAVABBAFIARAAtAEMATABPAFUARAAuAEMATwAuAFUASwADAEYAQwBMAE8AVQBEAC0ARQBYAEMASABBAE4ARwBFADQALgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAC4AQwBPAC4AVQBLAAUAJgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAC4AQwBPAC4AVQBLAAcACAD1kEI5xfDSAQAAAAA=
C: TlRMTVNTUAADAAAAAAAAAGIAAAAaARoBYgAAAAAAAABIAAAAGgAaAEgAAAAAAAAAYgAAAAAAAAB8AQAAAYIAAgoAWikAAAAPUwBoAGEAcgBvAG4ALgBFAGwAaQBrAGUAcgCB21Flo4mziz0i8YV3BDvvAQEAAAAAAAAbEq+bzfDSAXyIXYeuaXhZAAAAAAIAGgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAAEAHgBDAEwATwBVAEQALQBFAFgAQwBIAEEATgBHAEUANAAEACYAQwBVAFMAVABBAFIARAAtAEMATABPAFUARAAuAEMATwAuAFUASwADAEYAQwBMAE8AVQBEAC0ARQBYAEMASABBAE4ARwBFADQALgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAC4AQwBPAC4AVQBLAAUAJgBDAFUAUwBUAEEAUgBEAC0AQwBMAE8AVQBEAC4AQwBPAC4AVQBLAAcACAD1kEI5xfDSAQAAAAA=
S: A00000001 NO AUTHENTICATE failed.
C: A00000002 AUTHENTICATE PLAIN
S: +
C: [base64 encoded username password string]
S: A00000002 NO AUTHENTICATE failed.
C: A00000003 LOGIN Username Password
S: A00000003 NO Server Unavailable. 15
S: * BYE Connection closed. 14

However as we already know if I exclude NTLM then it works

Connected to imaps://hostedexchange.example.co.uk:993/
S: * OK The Microsoft Exchange IMAP4 service is ready.
C: A00000000 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=NTLM AUTH=GSSAPI UIDPLUS MOVE ID CHILDREN IDLE NAMESPACE LITERAL+
S: A00000000 OK CAPABILITY completed.
C: A00000001 AUTHENTICATE PLAIN
S: +
C: [base64 encoded string here]
S: A00000001 OK AUTHENTICATE completed.

Hopefully have included all the necessary client/server responses as you asked for in #265?

Thank you.

[jstedfast]

Can you get a log of another client successfully authenticating? Without that, I don't think I'll ever be able to figure out the problem because I won't have enough information to solve it.

There's no way for me to reproduce this problem, obviously, and the unit tests for NTLM support all pass based on the examples provided in the NTLM documentation.

[jstedfast]

I'll probably also need your username and password so that I can actually test that whatever changes I make will result in the correct output.

If you have access to create accounts on your server, you could simply create a new (dummy) account and send me those credentials instead (along with a successful NTLM authentication log for said account).

[jstedfast]

I've just committed a patch that might fix the issue after closely auditing all of the code, but I honestly have no idea if it'll actually fix things or not.

Please download the source code, build it, and test it out for me. And please please please get back to me. No one ever seems to get back to me... :-\

[jimjimbojimbob]

Many thanks, I'm working on trying to get a working example for you also (but it's difficult to do so!), will revert asap.

[jimjimbojimbob]

Hi I've asked a developer to compile it up but he's getting the error:

Severity Code Description Project File Line Suppression State
Error CS0246 The type or namespace name 'X509Name' could not be found (are you missing a using directive or an assembly reference?) MimeKit.Net40 E:\development\MailKit\submodules\MimeKit\MimeKit\Cryptography\X509CertificateDatabase.cs 441 Active

[jstedfast]

He needs to recursively clone MailKit.

Or, do this:

git submodule update --init --recursive

[jimjimbojimbob]

Hi thanks for that. OK we've got it built but unfortunately it didn't resolve the problem. However we did get an error which I believe explains why the second attempt to authenticate using PLAIN fails (I appreciate it doesn't help with the reason of why the AUTH NTLM fails in the first place).

The error we got was:

System.IO.IOException occurred
HResult=-2146232800
Message=Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.
Source=System
StackTrace:
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at MailKit.Net.Imap.ImapStream.ReadAhead(Int32 atleast, CancellationToken cancellationToken) in E:\development\MailKit\MailKit\Net\Imap\ImapStream.cs:line 339
InnerException:
ErrorCode=10053
HResult=-2147467259
Message=An established connection was aborted by the software in your host machine
NativeErrorCode=10053
Source=System
StackTrace:
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
InnerException:

What we believe is happening is that after the AUTH NTLM attempt has failed the socket is getting torn down, but the AUTH PLAIN is still trying to stream using the same socket. Does that make sense?

[jstedfast]

Yea, the server is dropping the connection on a failed authenticate command.

[jstedfast]

Do you have a domain in your username?

[jimjimbojimbob]

No it's just:

forename.surname

[jstedfast]

Okay, thanks. I was wondering if maybe the code wasn't handling domains in usernames properly or something. Guess not.

[jstedfast]

FWIW, you can work around this by doing client.AuthenticationMechanisms.Remove ("NTLM") and that may end up being the only solution.

[jimjimbojimbob]

Yes thank you, we have used that as a work around, I just thought that though the NTLM auth fails it would still then try PLAIN auth and succeed there, and so it wouldn't require the workaround.

[jstedfast]

Ideally that would be the case, and that normally works, but it seems Exchange doesn't allow fallback. Or at least not if NTLM is tried...

[jstedfast]

I've disabled NTLM auth.