Using a single SQL correlated sub-query to get two columns

I think you can use this approach:

SELECT b.row_id
     , b.x
     , b.y
     , b.something
     , a.x
     , a.y
  FROM my_table b
  left join my_table a on a.row_id = (b.row_id - 1)
                      and a.something != 42

@DavidEG posted the best syntax for the query.

However, your problem is definitely not just with the query technique. A JOIN instead of two subqueries can speed up things by a factor of two at best. Most likely less. That doesn't explain "hours". Even with millions of rows, a decently set up Postgres should finish the simple query in seconds, not hours.

• First thing that stands out is the syntax error in your query:

  ... WHERE a.row_id = (b.row_id - 1), a.something != 42
  

  AND or OR is needed here, not a comma.

• Next thing to check are indexes. If row_id is not the primary key, you may not have an index on it. For optimum performance of this particular query create a multi-column index on (row_id, something) like this:

  CREATE INDEX my_table_row_id_something_idx ON my_table (row_id, something)
  

• If the filter excludes the same value every time in something != 42you can also use a partial index instead for additional speed up:

  CREATE INDEX my_table_row_id_something_idx ON my_table (row_id)
  WHERE something != 42
  

  This will only make a substantial difference if 42 is a common value or something is a bigger column than just an integer. (An index with two integer columns normally occupies the the same size on disk as an index with just one, due to data alignment. See:

  • Calculating and saving space in PostgreSQL
  • Is a composite index also good for queries on the first field?

• When performance is an issue, it is always a good idea to check your settings. Standard settings in Postgres use minimal resources in many distributions and are not up to handling "millions of rows".

• Depending on your actual version of Postgres, an upgrade to a current version (9.1 at the time of writing) may help a lot.

• Ultimately, hardware is always a factor, too. Tuning and optimizing can only get you so far.