Fixing Pass The Hash and 14 Other Problems


This is an update to breaking and building a secure network. A key part of that advice is a combined solution to about 15 different serious problems with password-based authentication, including the Pass-The-Hash (PTH) attack. As a reminder, here are the problems:

  1. Admins leave passwords in shared drives & scripts, even if you tell them not to.
  2. Passwords and hashes can be dumped from memory - see Mimikatz, most famously. This creates a much longer exposure duration than Kerberos tickets. They can stay in memory as long as the system is online, and are good forever or for months until password reset.
  3. They can be keylogged. Also a very common attack with no real solution if you're using passwords.
  4. They can be guessed. Regardless of complexity requirements, users will find and use the simplest pattern allowed, and attackers routinely take advantage.
  5. Everybody reuses them. Even if you prevent it within your network, it will still happen. Once your users have had to memorize a complicated password, they'll use it when they need to sign up for RockYou or Facebook. Then when RockYou's passwords get hacked or the users' home computers gets hacked, the bad guys will have the passwords to your system. Pentesters will never find the "vulnerable user at home" since they're not allowed to hack home systems, but real bad guys use this all the time.
  6. They are hard to remember - causing a lot of the above problems as well as writing passwords down on sticky notes for random visitors to see or to end up on office photographs on instagram.
  7. Social engineering. Your admins will give their creds to a webpage that looks like their management portal or server login, but was really just from a spoofed email. Your users will give them up to somebody who sounds like tech support on the phone. This happens all the time. Unfortunately, users will frequently fail the security tasks given to them.
  8. Passing-the-hash - with a single hash from memory or local or domain database, you don't have to crack the password and can re-use it forever until password reset.
  9. Pots of gold - hash dumps. All credentials stored forever in great big pots of gold, like local and domain hash databases. Find the big one and be set for life. Huge exposure of all credentials, forever, by design.
  10. Easy lockouts - with no credentials but a list of usernames, an attacker can cause great pain and anguish by locking out everyone's account.
  11. Online brute force - Especially if lockouts are disabled, but even if not, attackers can brute force network logins "online" against all your users & admins, and probably find plenty of good passwords.
  12. NTLM relay - The Defcon 20 presentation titled "Owned in 60 Seconds: From Network Guest to Windows Domain Admin" probably wasn't exaggerating. This attack has been working for so long in so many different ways for so many different attackers, it would be funny if it wasn't so sad.
  13. NTLM authentication offline cracking - even if attackers couldn't do the above, they can obtain a challenge-response authentication by simply observing an NTLM authentication or by using a credential-harvesting server like Metasploit can stand up. With this information, the attackers can brute-force the challenge-response authentication offline at high speed (probably billions of tries per second) without triggering lockouts and crack many passwords.
    Edit: Or, they can use inherent weaknesses in the NTLM protocol pre-NTLMv2 to obtain the hash with 100% certainty, using a DES bruteforcing tool built to crack Windows challenge-response authentication, and then pass the hash to their heart's content.
  14. Cached credential offline cracking - if a highly privileged admin has logged into a box at some time in the past, even after reboots, even long ago, cached credentials (salted hash) can be left on the box. An attacker who compromises that system can brute-force the creds offline at high speed (probably billions of tries per second) without triggering lockouts or getting logged and crack the password.
  15. Painful post-attack cleanup - if an intruder got high privileges, when cleaning up after the intrusion, you must reset every password in addition to all your other tasks. This usually causes great anguish and lost work time, and takes so long, that many intruders get right back in again. Security does not just consist of how to prevent an attack, but also how to limit the damage of one and pain of recovering from one.

My proposed solution is to mandate smart card and Kerberos authentication for all accounts allowed to authenticate to remote network resources, especially with administrative privileges. Edit: Here's why it works. No other measures come close to solving all these problems, and for many of the problems, I am unaware of any other solution at all. One of the reasons I presented it is that sadly, both Microsoft and other security researchers did not consider this solution or discounted it as unrealistic, in both Microsoft's Mitigating Pass-the-Hash (PtH) Attacks and Other Credential Theft Techniques paper and by Skip Duckwall in the passing-the-hash blog, whose authors have pointed out many other issues with the Microsoft paper. Both understandably but regrettably stuck to the topic of PTH and did not mention the myriad other problems that would be solved by this proposal. To be clear, the paper and blog predate my presentation, but their objections do merit a response, so I will specifically address them here.

The objections generally either showed flaws with only implementating half a solution, like giving out smartcards but not requiring their use, or mandating smartcards but not Kerberos; or assuming legacy equipment or implementation difficulties will doom the project. This is due to a focus on what a large enterprise would be likely to implement with minimal effort right now. It discounts both my focus of recommendations that should be given when building a new network and the possibility of an enterprise becoming very serious about security, such as following a major breach. After all, every network that exists now was started at some point, so I think the "new network" issue is most significant in the long term. It reminds me of an immigration debate that focuses on people who are already here, paying less attention to future immigrants only to find that 30 years later, what happened to the future immigrants is all that mattered.

Specifically, here are the objections, and why they should not stop you from implementing this solution:

The Objections

It's expensive

But the benefits outweigh costs. I believe thousands of organizations have gotten smart cards. If you looked at all the wasted time and effort password policies and resets inflict upon all your users, as well as the huge cost of breaches, you would find they are worth it. MRI machines are more expensive, but if a doctor thinks you might have brain cancer, it would be medical malpractice not to prescribe one. I'd argue not prescribing the solution to these problems is security malpractice.

It's hard to implement

Smart cards been done many many times and instructions are available. Edit: I do not want to trivialize these problems, especially for those with large networks or many systems to support or convert over to smart cards and/or Kerberos. But there are many vendors who offer hardware and support, and now you've got instructions like this.

There's still hashes in AD that can be used...The NT password hash does not expire... you can brute-force passwords of smart card accounts...

Not if you force smart card logon and disable NTLM at least server side. All of these objections assume you only implemented half of the solution, which will definitely not solve the problem in question. Although it will help solve about 4 other problems, such as users creating passwords (shudder) or re-using them on websites like RockYou or for their Yahoo mail.

"File shares, Outlook, SQL databases, and several other applications typically cannot be configured to use multifactor auth"

But they can all use Kerberos and work nicely. Even in the realm of random third-party applications that require authentication, if you look, you will probably find that there is at least a plugin that enables Kerberos. I have been quite surprised in this regard. Worst case, you can get a competing product instead. See previous comments about setting up a network right the first time.

"Of course all service accounts on your network don't have elevated permissions, right?"

Service accounts can be set up to run scheduled tasks or services under a "S4U" login with even admin rights if necessary on any system and probably do whatever they need to. Yes, under this policy, you cannot allow any accounts with remote admin privileges to use a password or run unattended with reusable credentials. But your admins can schedule recurring automated maintenance scripts and other tasks on any and all systems without leaving powerful remote-admin-capable credentials lying around said systems. (those automatically scheduled tasks won't be able to turn around and remotely manage other boxes) That means keeping interactive, remote-admin-capable logins for only when they are sitting at an admin console with a smart card, but it can definitely be done.

"Disabling NTLM... Typically NAS/SAN devices typically aren't members of the domain, so they can't use Kerb. Same thing with network-aware devices like printers, digital senders, copy machines, etc..." "it cannot be easily implemented... non-Windows operating system dependencies..."

Two big problems with this statement. First, a quick Google reveals plenty of enterprise printers/scanners/copiers and other equipment that can work fine with Kerb. Even if old systems don't support it, once again as security professionals we should still be explaining how everyone else can secure their network and how to build a secure network. Samba has full Kerberos support last I checked, so that isn't an excuse either.
Second, this is a false dilemma. You could disable accepting NTLM on your Windows servers and workstations, which does not prevent the clients from sending NTLM to printers/other devices, or prevent the printers from accepting it. If you do this they'll work fine. Correct me if I'm missing something, but this would not undermine the security benefits of this proposal either; bad guys with hashes inside your network might authenticate to your printers, and be able to print unauthorized documents. Or maybe break your printer, but nothing too scary. Printer/device exploits are a separate issue.

"[U]p until fairly recently it was actually impossible to disable NTLM. The only way to do it is to use Windows Server 2008R2 running in the highest domain functional level and everybody's clients must be Windows 7."

First, once again see note on new networks. Second, that is only actually true for your domain controllers, not all your member servers and clients. If you configure the policy Network Security: Restrict NTLM: NTLM authentication in this domain then no NTLM authentication will work for domain accounts, even if the clients are Windows Vista or Windows XP. Windows 7 clients are only necessary if you want to use the Network Security: Restrict NTLM: Incoming NTLM Traffic approach. I admit this one can hurt for those with existing networks, but it has been four years, and it is relevant to those who have not upgraded simply because they did not see any compelling reasons to do so. Finally, XP/2003 systems really really should be put out of their misery or banished from the network for many reasons, and not just this one. (lack of ASLR (!), non-hardened heap, SDL, kernel signing enforcement, support ending soon, AppLocker, browser sandboxing...)

"does not mitigate theft and reuse of Kerberos tickets..."

Kerberos tickets pose far less of a credential exposure issue than hashes. Hashes stick around and are good basically forever (60 days or however long your password expiration is is a long long time to a hacker). Tickets expire after just a few hours. That means they are far less likely to be left lying around and found in the first place, and cannot be used for long periods of time if obtained. Not to mention the dozen or so other serious problems that will definitely be solved by these measures. It is true that this type of credential theft must be defended against, by doing things like limiting number of users with admin accounts, blocking those admin accounts from accessing the internet or email and using dedicated, firewalled admin systems, but that's not an excuse to underestimate the solution's impact or the problems it does solve.

This is just the beginning!

If you want to see some even bigger mythbusting than before, and more cool ways to make attackers' lives miserable, come to my Derbycon 2013 presentation.

, , , , , , ,

  1. #1 by Bill V on April 24, 2014 - 10:02 pm

    Saw your talk at Derby last year and really enjoyed it. I’m curious if you’ve ever heard of Authlite and your thoughts on it. It uses yubikeys for 2 factor.

    Any thoughts appreciated!

    • #2 by scriptjunkie on April 26, 2014 - 3:15 am

      I have not looked at it before. From a quick glance, they’re using AES and it doesn’t look like they have any hardware on the DC’s. which means the keys are stored, and thus stealable. So it sounds comparable to RSA tokens, still better than passwords, but doesn’t have all the benefits of smart cards.

(will not be published)