How to I remove the Powershell start text?

Pass the -nologo option.

-NoLogo Starts the PowerShell console without displaying the copyright banner.

pwsh.exe -nologo ...other arguments...

Now you can add -nologo in Windows Terminal settings:

If you are using the new Windows Terminal then:

1. Go to Settings:

   

2. Add the argument -nologo to the PowerShell command line:

   

From @sandu's answer, it could be improved as below.

"terminal.integrated.profiles.windows": {
   "PowerShell": {
       "source": "PowerShell",
       "args": ["-noLogo"]
   }
},
"terminal.integrated.defaultProfile.windows": "PowerShell",