Yoast SEO plugin creates XML Sitemap with http. Is this correct for a https site?

You need to disable and reenable the sitemap, going to SEO > Sitemap XML. It probably will solve your problem (Yoast bug).

Yoast SEO uses the default site address set by your WordPress installation which is entered the first time you install WordPress. When enforcing a HTTPS on a WordPress set you must setup the current URL on both the WordPress Address and Site Address URL options found within the Dashboard in General Settings.

  1. Login to your WordPress Dashboard
  2. Click Settings
  3. Click General

Then your options should look something like this:

enter image description here

If you have these settings already in place then its likely your hosting is caching the current file which needs purging.