How to pass parameters to Linux system call?

You need to tell the build system that your system call requires 2 arguments and that they are of type int. This is so that the scripts that are part of the build system will generate appropriate wrappers for casting the arguments into the type you require. Instead of defining the actual handler like you did, you should use -

SYSCALL_DEFINE2(my_syscall_2, int, a, int, b) // Yes, there is a comma between the types and the argument names
{
    printk("my_syscall_2 : %d, %d\n", a, b);
    return b;
}

SYSCALL_DEFINEx is defined in linux/include/linux/syscalls.h.

You can look at an example in linux/fs/read_write.c


I found the solution. As @Ajay Brahmakshatriya answered, I should use SYSCALL_DEFINEx macro. And also, I should modify arch/x86/entry/syscalls/syscall_64.tbl as well.

Here's final summary.

How to add new system calls

First, modify arch/x86/entry/syscalls/syscall_64.tbl : add those lines below.

335     common     my_syscall_0     __x64_sys_my_syscall_0
336     common     my_syscall_1     __x64_sys_my_syscall_1
337     common     my_syscall_2     __x64_sys_my_syscall_2

Second, modify include/linux/syscalls.h : add those lines below.

asmlinkage long sys_my_syscall_0(void);
asmlinkage long sys_my_syscall_1(int);
asmlinkage long sys_my_syscall_2(int, int);

Third, create a new file for implementation. For my case, kernel/my_syscall.c.

#include <linux/syscalls.h>
#include <linux/kernel.h>

SYSCALL_DEFINE0(my_syscall_0)
{
    printk("my_syscall_0\n");
    return 0;
}

SYSCALL_DEFINE1(my_syscall_1, int, a)
{
    printk("my_syscall_1 : %d\n", a);
    return 0;
}

SYSCALL_DEFINE2(my_syscall_2, int, a, int, b)
{
    printk("my_syscall_2 : %d, %d\n", a, b);
    return b;
}

Fourth, add created file to Makefile in its directory. For my case, kernel/Makefile.

...
obj-y = fork.o exec_domain.o panic.o \
        cpu.o exit.o softirq.o resource.o \
        sysctl.o sysctl_binary.o capability.o ptrace.o user.o \
        signal.o sys.o umh.o workqueue.o pid.o task_work.o \
        extable.o params.o \
        kthread.o sys_ni.o nsproxy.o \
        notifier.o ksysfs.o cred.o reboot.o \
        async.o range.o smpboot.o ucount.o \
        my_syscall.o
...

Finally, compile and install the kernel. Now, you would be able to see new system calls working well.

#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>

int main()
{
    printf("1 : %d\n", syscall(335));
    printf("2 : %d\n", syscall(336, 1));
    printf("3 : %d\n", syscall(337, 2, 3));
    return 0;
}

dmesg command shows me that system calls are working well.

# dmesg
my_syscall_0
my_syscall_1 : 1
my_syscall_2 : 2, 3