x64 Calling convention

The x64 calling convention is a set of rules that dictate how functions receive parameters and return values, how the stack is managed, and how registers are used. This ensures that code generated by different compilers can interoperate.

Key Points of the x64 Calling Convention

Registers:
- General-purpose registers: RAX, RBX, RCX, RDX, RSI, RDI, RBP, RSP, R8-R15.
- Stack pointer: RSP.
- Base pointer: RBP.
Parameter Passing:
- The first six integer or pointer arguments are passed in registers RDI, RSI, RDX, RCX, R8, and R9.
- Additional arguments are passed on the stack.
Return Values:
- The primary return value is placed in RAX.
- If a function returns a structure or a union, the address of the return value is passed as a hidden first parameter.
Stack Management:
- The stack must be 16-byte aligned at the point of a function call.
- The caller is responsible for allocating space for the return address and any arguments that do not fit in registers.
Callee-saved Registers:
- Registers RBX, RBP, R12-R15 must be preserved by the callee.
- The callee must save and restore these registers if it uses them.

Example

Here is an example of a simple function in x64 assembly that adds two integers:

global add
add:
    add rdi, rsi  ; RDI = RDI + RSI
    mov rax, rdi  ; Move result to RAX
    ret           ; Return, result is in RAX

Explanation (1)

The function add takes two integer arguments in registers RDI and RSI.
It adds the values in RDI and RSI, storing the result in RDI.
It moves the result to RAX.
The ret instruction returns to the caller, with the result in RAX.

Example with Stack Usage

Here is an example of a function that uses the stack to store local variables:

global sum_array
sum_array:
    push rbp              ; Save base pointer
    mov rbp, rsp          ; Set base pointer
    sub rsp, 16           ; Allocate space for local variables

    mov qword [rbp-8], 0  ; Initialize sum to 0
    mov qword [rbp-16], 0 ; Initialize index to 0

loop:
    mov rax, rdi          ; Load array pointer
    mov rcx, [rbp-16]     ; Load index
    mov rdx, rsi          ; Load array length
    cmp rcx, rdx          ; Compare index with array length
    jge end_loop          ; If index >= length, exit loop

    mov rax, [rax+rcx*8]  ; Load array element
    add [rbp-8], rax      ; Add to sum
    inc qword [rbp-16]    ; Increment index
    jmp loop              ; Repeat loop

end_loop:
    mov rax, [rbp-8]      ; Move sum to return register
    mov rsp, rbp          ; Deallocate local variables
    pop rbp               ; Restore base pointer
    ret                   ; Return

Explanation (2)

The function sum_array takes two arguments: a pointer to an array in RDI and the array length in RSI.
It saves the base pointer (RBP) and sets up the stack frame.
It allocates space for local variables.
It initializes the sum and index to 0.
It enters a loop to iterate over the array, loading each element, adding it to the sum, and incrementing the index.
After the loop, it moves the sum to the return register (RAX), deallocates the local variables, and restores the base pointer.
It returns with the sum in RAX.

These examples illustrate the basic principles of the x64 calling convention, including register usage, parameter passing, and stack management.

Last modified: 14 December 2024