# Making System Calls in Assembly

## March 30, 2019

I was about to go to bed last night, and my brain started to behave like an overclocked Core i9 CPU (relax, it’s normal for my brain to do that at night time). I recalled that system calls are made by allocating specific values to a set of registers and then invoking the syscall instruction. I’ve coded an x86 bootloader in NASM before and thought it would be easy to make syscalls in assembly, so I decided to give it a try, “not tonight, I’ll do it tomorrow”, I said to myself and went to sleep. I woke up, turned on my Mac, booted my Ubuntu VM and looked up the Linux system call table for the x86-64 instruction set. I decided to make a mkdir system call in assembly because you can see the effect of the syscall in real time, and also because it’s fun to just mov some values and create a directory.

# How It Works

Before we start coding, it’s helpful to look up the man page for the mkdir system call.

Man page for the mkdir system call.

The man page says the argument mode specifies the mode for the new directory. “What is mode?”, you may ask, it’s the base permission of the directory that we will be creating. The effective permisson of the new directory will be determined by the mode number and the process’s umask number. The default umask number is 22 (octal) and the default mode of a directory with an effective permission drwxr-xr-x (default permission of a directory created when mkdir is called) will be 777 (octal). The other obvious argument that we’re interested in is the pathname.

# What We Need to Do

To create a directory using the mkdir syscall we need to:

• Set the umask number to 22 (octal).
• Specify the pathname for our directory.
• Set the directory mode number to 777 (octal).
• Exit.

I’ve already mentioned we need to mov some values to a specific set of registers before we can make a syscall. I’ve made a small table that you can refer to and code the assembly instructions required to make a syscall in assembly.

%rax System Call %rdi %rsi
60 sys_exit int error_code
95 sys_umask int mask
83 sys_mkdir const char *pathname int mode

To set the umask number, we need to mov the value 95 to the rax register and mov the mask number 22 to the rdi register before invoking the syscall instruction.

Now that we have set the umask number to 22 (octal), let’s make our mkdir syscall. We need to mov the value 83 to the rax register and mov the address of the pathname to the rdi register. Finally, we set the mode number by moving 777 (octal) to the rsi register.

Now, let’s write some instructions to exit the process and define our pathname in the data section.

The complete program should look this:

# Testing

Now, let’s see if our program works as intended. Let’s compile the program and run it.

Now, when we issue the ls -alh command we should see a directory named testdir with the file permission: drwxr-xr-x.

It works! If you want to write more assembly code to make some interesting syscalls, lookup the table found on this webpage. Have fun!