Does Linux keeps a cache of groups members if on LDAP ? (Difference between groups vs getent group))

pam_ldap and nsswitch have no caching mechanisms, but nscd or sssd may be present on your system that implement cache.

To invalidate / flush nscd groups cache use:

sudo nscd --invalidate=group

To invalidate / flush sssd groups cache use:

sudo sss_cache -G

jopasserat's comment leads to another possible answer.

The groups command doesn't report group memberships of a user. It reports the group ID privileges of the current process. It only uses NSS to convert numeric group IDs to names.

When a user logs in, the group memberships are obtained from NSS, and the setgid and setgroups system calls are used to give the correct privileges to the user's initial process. All processes descended from there inherit the same privileges (except when a set-id program is executed).

If the configured privileges change, while the user is logged in, the existing processes are not affected. You have to log out and in again to gain privileges, and if you're trying to revoke privileges, you have to kill all of the user's processes to finish the job.

nscd and such can add extra layers of cache to worry about, but pre-existing user processes are effectively a privilege cache that is present in all configurations.