Wordpress - Orderby meta_value only returns posts that have existing meta_key

As stated in @ambroseya's answer, its supposed to work like that. Once you declare a meta query, even if you aren't looking for a specific value, it will only query posts with that meta key declared. If you want to include all posts sort them by the meta key, use the following code:

$args = array(
    'post_type' => 'news',
    'orderby' => 'meta_value',
    'order' => 'ASC',
    'meta_query' => array(
        'relation' => 'OR',
        array( 
            'key'=>'custom_author_name',
            'compare' => 'EXISTS'           
        ),
        array( 
            'key'=>'custom_author_name',
            'compare' => 'NOT EXISTS'           
        )
    ),
    'posts_per_page'=>-1
);

$query = new WP_Query($args);

echo $query->found_posts;

What this does is use an advanced meta query that looks for posts that do and don't have that meta key declared. Since the one with EXISTS is first, when you sort by meta_value, it'll use the first query.

I tried applying @Manny Fleurmond's answer and like @Jake I couldn't get it to work even after correcting the typo that 'orderby' => 'meta_key' should be 'orderby' => 'meta_value'. (And for completeness it should be 'posts_per_page' not 'post_per_page' but that doesn't affect the issue being looked at.)

If you look at the SQL query actually generated by @Manny Fleurmond's answer (having corrected the typos) this is what you get:

SELECT   wp_{prefix}_posts.* FROM wp_{prefix}_posts
LEFT JOIN wp_{prefix}_postmeta ON (wp_{prefix}_posts.ID = wp_{prefix}_postmeta.post_id AND wp_{prefix}_postmeta.meta_key = 'custom_author_name' )
LEFT JOIN wp_{prefix}_postmeta AS mt1 ON ( wp_{prefix}_posts.ID = mt1.post_id )
WHERE 1=1  AND ( 
    wp_{prefix}_postmeta.post_id IS NULL 
    OR 
    mt1.meta_key = 'custom_author_name'
) AND wp_{prefix}_posts.post_type = 'news' AND
(wp_{prefix}_posts.post_status = 'publish' OR wp_{prefix}_posts.post_author = 1 AND wp_{prefix}_posts.post_status = 'private')
GROUP BY wp_{prefix}_posts.ID ORDER BY wp_{prefix}_postmeta.meta_value ASC

This illustrates the way WP is parsing the query vars: it's creating a table for each meta_query clause, then figuring out how to join them and what to order by. The ordering would work fine if you were only using a single clause with 'compare' => 'EXISTS', but joining the second 'compare' => 'NOT EXISTS' clause with OR (as we must) messes up the ordering. The result is that LEFT JOIN is used to join both the first clause / table and the second clause / table - and the way WP puts everything together means that the table created using 'compare' => 'EXISTS' is actually being populated with meta_values from ANY custom field, not just the 'custom_author_name' field we are interested in. So I think ordering by that clause / table will only give the desired results if the particular post_type of 'news' only has a single custom field.

The solution that worked for my situation was to order by the other clause / table - the NOT EXISTS one. Seemingly counter-intuitive I know, but because of the way WP parses the query vars it is this table where meta_value is populated only by the custom field we are after.

(The only way I figured this out was by running the equivalent of this query for my case:

SELECT   wp_{prefix}_posts.ID, wp_{prefix}_postmeta.meta_value, mt1.meta_value FROM wp_{prefix}_posts
LEFT JOIN wp_{prefix}_postmeta ON (wp_{prefix}_posts.ID = wp_{prefix}_postmeta.post_id AND wp_{prefix}_postmeta.meta_key = 'custom_author_name' )
LEFT JOIN wp_{prefix}_postmeta AS mt1 ON ( wp_{prefix}_posts.ID = mt1.post_id )
WHERE 1=1  AND ( 
    wp_{prefix}_postmeta.post_id IS NULL 
    OR 
    mt1.meta_key = 'custom_author_name'
) AND wp_{prefix}_posts.post_type = 'news' AND
(wp_{prefix}_posts.post_status = 'publish' OR wp_{prefix}_posts.post_author = 1 AND wp_{prefix}_posts.post_status = 'private')
ORDER BY wp_{prefix}_postmeta.meta_value ASC

All I've done is changed the columns being shown and removed the GROUP BY clause. This then showed me what was going on - that the postmeta.meta_value column was pulling in values from all meta_keys, while the mt1.meta_value column was pulling in only meta_values from the news custom field.)

The Solution

As @Manny Fleurmond says, it is the first clause that is used for the orderby, so the answer is just to swap the clauses round, giving this:

$args = array(
    'post_type' => 'news',
    'orderby' => 'meta_value',
    'order' => 'ASC',
    'meta_query' => array(
        'relation' => 'OR',
        array( 
            'key' => 'custom_author_name',
            'compare' => 'NOT EXISTS'           
        ),
        array( 
            'key' => 'custom_author_name',
            'compare' => 'EXISTS'           
        )
    ),
    'posts_per_page' => -1
);

$query = new WP_Query($args);

Alternatively you can make the clauses associative arrays and order by the corresponding key, like so:

$args = array(
    'post_type' => 'news',
    'orderby' => 'not_exists_clause',
    'order' => 'ASC',
    'meta_query' => array(
        'relation' => 'OR',
        'exists_clause' => array( 
            'key' => 'custom_author_name',
            'compare' => 'EXISTS'           
        ),
        'not_exists_clause' => array( 
            'key' => 'custom_author_name',
            'compare' => 'NOT EXISTS'           
        )
    ),
    'posts_per_page' => -1
);

$query = new WP_Query($args);