Apple - How to have new email notification while Apple Mail.app is closed?

You cannot.

You need the Mail app running. Unlike Calendar events, checking for emails is not something that can run locally. A client application need to connect with your mail server and retrieve your e-mails.

Sure there could be some background process that does that without the need of the Mail app, but it would not make much sense; in the end you would still have a process doing the checking.

---

Something that may make the Mail app less intrusive to you desktops' organization etc is having it been launched automatically and hidden at startup.