Why Encoding Does not Matter and How Metasploit Generates EXE’s


Payload executables generated by msfencode are commonly detected by antivirus engines, depending which antivirus engine is used. A common misconception is that the antivirus engines are actually detecting the shellcode, and therefore, the best way to avoid antivirus detection is to pick an encoder that the antivirus engine cannot handle, or encode many times. After all, those are both prominent options of msfencode.

Usage: /opt/framework-3.5.2/msf3/msfencode

OPTIONS:

-a The architecture to encode as
-b The list of characters to avoid: '\x00\xff'
-c The number of times to encode the data
-d Specify the directory in which to look for EXE templates
-e The encoder to use
...

If you just want to evade McAfee/Symantec, the basic msfencode command cat payload | msfencode -t exe > my.exe should work fine. But if you have ever tried to evade an antivirus like Avast! that tries to catch Metasploit payloads by changing encoder or number of times to encode, chances are, you were very frustrated and just wasted time. So how can they catch every encoder and more importantly, how can you evade them?

First, let’s see how Metasploit generates EXE’s. The relevant code is in lib/msf/util/exe.rb in the self.to_win32pe function. First the payload is placed within a “win32_rwx_exec” block:

# Copy the code to a new RWX segment to allow for self-modifying encoders
payload = win32_rwx_exec(code)

The function is defined later in the file:

# This wrapper is responsible for allocating RWX memory, copying the
# target code there, setting an exception handler that calls ExitProcess
# and finally executing the code.
def self.win32_rwx_exec(code)

This function writes a set of assembly instructions consisting mostly of the block_api code that forms the majority of the Metasploit win32 shellcode, while randomly inserting nop instructions and opcodes to jmp over randomly generated bytes. These assembly instructions will look up and call the VirtualAlloc function to allocate RWX memory, copy the target code there and execute the code.

Ignoring the inject block for now (the same principles apply; less random code and code is just in a new section rather than replacing old code) the function then finds the executable (.text) section. Then it lists the addresses that will be modified by the loader into an array called mines:

# We need to make sure our injected code doesn't conflict with the
# the data directories stored in .text (import, export, etc)
mines = []
pe.hdr.opt['DataDirectory'].each do |dir|
next if dir.v['Size'] == 0
next if not text.contains_rva?( dir.v['VirtualAddress'] )
mines << [ pe.rva_to_file_offset(dir.v['VirtualAddress']) - off_beg, dir.v['Size'] ]
end

It then finds, out of the remaining blocks, an executable block of sufficient size to store the payload with room to spare. Then it creates a sled of nops, and somewhat randomizes the entry point to land in them. At the end of the nops is a relative jump to the payload:

# Pad the entry point with random nops
entry = generate_nops(framework, [ARCH_X86], rand(200)+51)
...

# Relative jump from the end of the nops to the payload
entry += "\xe9" + [poff - (eidx + entry.length + 5)].pack('V')

Then it randomly changes 25% of the remaining executable bytes and the timestamp, and fixes the checksum.

So in summary, the exe's entry point is a random sequence of certain opcodes in the nop sled which does not look like a standard exe entry routine, followed by a jump to the win32_rwx_exec generated block of code that is identical to a metasploit shellcode block with some random nop and simple jmp instructions scattered inside. Then, if the antivirus engine can follow the jump to a dynamically, externally generated address, it will see the encoded payload, which will likely be much harder to distinguish than the previous block of code. In other words, if the AV is going to detect the exe as malicious, it is much easier to catch the code before the encoded payload than the encoded payload itself.

To demonstrate this, I used msfencode to generate an exe without any payload at all:
echo -n | msfencode -e generic/none -t exe > myn.exe
[*] generic/none succeeded with size 0 (iteration=1)

Despite the fact that there was no payload, upon uploading the VirusTotal, still 20 out of 41 antivirus engines, or 48.8% identified the file as malicious. See the report for yourself here.

To avoid this detection, just write your own exe or modify the source of an existing exe to get RWX memory, copy the payload into it, and execute it in a new way. Of course metasploit could do this, but then the antivirus companies, in their quest to enumerate badness, would identify whatever way metasploit used, and repeat ad nauseam. So have a little creativity yourself.

Oh, and by the way, msfencode is not useless, and neither are these encoders. They are both valuable for manually creating payloads to avoid certain bad characters, or evade an IDS in an exploit. These options just are not useful when creating an exe.

, , , , , ,

  1. #1 by Kevin on April 26, 2011 - 10:26 pm

    I actually just wrote a paper describing the different encoders and how they affect antivirus detection. However, I never knew the exact methods behind the creation of the binaries, so it was an interesting read. Thanks for sharing.

    By the way, if you’re interested in reading my article it’s here: http://technology-flow.com/articles/metasploit-encoding-antivirus-detection/ To summarize, encoding schemes that use non static variables, like information pulled from the host system, seemed to work the best.

    Regards,

    -Kevin

  2. #2 by scriptjunkie on April 27, 2011 - 12:42 am

    As another update, based on experiments Craig Freyman and I conducted, Craig was able to create an executable undetected by any of the antivirus engines. He first took the source of a very basic program, and just added a few function calls to get RWX memory and execute metasploit shellcode encoded just once with shikata_ga_nai.

    Opening a project, generating the shellcode, copying it into the source, and compiling the program is possibly the easiest way to get your undetectable exe.

  3. #3 by Kevin on April 27, 2011 - 5:03 pm

    Thats a really interesting result. So it appears that AV are detecting the Metasploit executable itself, rather than the shellcode residing inside. Thanks for that update.

  4. #4 by PWNZ on May 1, 2011 - 12:02 pm

    scriptjunkie :
    As another update, based on experiments Craig Freyman and I conducted, Craig was able to create an executable undetected by any of the antivirus engines. He first took the source of a very basic program, and just added a few function calls to get RWX memory and execute metasploit shellcode encoded just once with shikata_ga_nai.
    Opening a project, generating the shellcode, copying it into the source, and compiling the program is possibly the easiest way to get your undetectable exe.

    Could we have a sample article on that, please? :)

  5. #5 by Tom on February 7, 2013 - 10:38 pm

    As someone who evaded the security arm of one of the most litigious and pro-active gaming companies in the world, I can say from experience that scriptjunkie is right with #2, and I can even boil it down further:

    “writing your own code is possibly the easiest way to get your undetectable exe.”

    I was always amazed at how many of my “colleagues” got caught when I didn’t, simply because they were always *using someone else’s tools*. I guess the fact that I never used my code for criminal endeavors helped me maintain a low profile, but still, it seems that just the simple fact of being able to write and compile and link my own C/C++ code some how made me a magician compared to the other so-called “hackers.”

(will not be published)