How to list all configuration files for an already installed package?

There's no need to use anything other than cat, if I understood your intention correctly:

cat /var/lib/dpkg/info/<package>.conffiles

should give you what you're after. For instance for package zsh:

% cat /var/lib/dpkg/info/zsh.conffiles
/etc/zsh/zlogin
/etc/zsh/zlogout
/etc/zsh/zprofile
/etc/zsh/zshenv
/etc/zsh/zshrc
/etc/zsh/newuser.zshrc.recommended

Regarding a case where there's no such file for a given package - it's up to the package maintainer to designate certain files as configuration. If this hasn't been done properly, you should file a bug where appropriate.

In such cases you have a couple of options.

1. List files belonging to the package that are in /etc/:

   dpkg -L package | grep '/etc'
   

2. Fetch and inspect the source package to find out how it was compiled (which should also show you where it expects its configuration files to be located).

   apt-get source package
   less package-x.y.z/debian/rules
   

3. Look up the upstream project page to find documentation.