Wordpress - How can I load a page template from a plugin?

You can use the theme_page_templates filter to add templates to the dropdown list of page templates like this:

function wpse255804_add_page_template ($templates) {
    $templates['my-custom-template.php'] = 'My Template';
    return $templates;
    }
add_filter ('theme_page_templates', 'wpse255804_add_page_template');

Now WP will be searching for my-custom-template.php in the theme directory, so you will have to redirect that to your plugin directory by using the page_template filter like this:

function wpse255804_redirect_page_template ($template) {
    if ('my-custom-template.php' == basename ($template))
        $template = WP_PLUGIN_DIR . '/mypluginname/my-custom-template.php';
    return $template;
    }
add_filter ('page_template', 'wpse255804_redirect_page_template');

Read more about this here: Add custom template page programmatically

This is a combination of the above answer and the above comments which ended up working for me.

The function to add the plugin to the list of available templates:

function wpse255804_add_page_template ($templates) {
    $templates['my-custom-template.php'] = 'My Template';
    return $templates;
    }
add_filter ('theme_page_templates', 'wpse255804_add_page_template');

The function to point the template to the appropriate directory within the plugin:

function wpse255804_redirect_page_template ($template) {
    $post = get_post();
    $page_template = get_post_meta( $post->ID, '_wp_page_template', true );
    if ('my-custom-template.php' == basename ($page_template))
        $template = WP_PLUGIN_DIR . '/mypluginname/my-custom-template.php';
    return $template;
    }
add_filter ('page_template', 'wpse255804_redirect_page_template');