Remove specified non-numeric rows

JavaScript (ES6), 48 46 bytes

(a,l)=>a.filter(r=>l.some(c=>r[a=0,c]<1/0)||a)

Explanation

Expects an array of rows as arrays, and an array of 0-indexed numbers for the columns to check. Returns an array of arrays.

Straight-forward filter and some. Checks for NaN by using n < Infinity (true for finite numbers, false for NaNs).

var solution =

(a,l)=>
  a.filter(r=>     // for each row r
    l.some(c=>     // for each column to check c
      r[a=0,       // set a to false so we know the some was executed
        c]<1/0     // if any are not NaN, do not remove the row
    )
    ||a            // default to a because if l is of length 0, some returns false but
  )                //     we must return true

<textarea id="matrix" rows="5" cols="40">16 NaN 3 13
5 11 NaN 8
NaN 7 NaN 12
4 14 -15 1</textarea><br />
<input type="text" id="columns" value="0 2" />
<button onclick="result.textContent=solution(matrix.value.split('\n').map(l=>l.split(' ').map(n=>+n)),(columns.value.match(/\d+/g)||[]).map(n=>+n)).join('\n')">Go</button>
<pre id="result"></pre>

Pyth, 16 19 10 9 7 10 Bytes

Column indices start at zero. Input is a list of lists. Uses an empty string as non-numeric value. Takes list of column indices on the first line and the Matrix with the values on the second line.

?Qf-QxkTEE

Try it online!

Explanation

?Qf-QxkTEE       # Implicit: Q=column indices, E=Matrix

?Q       E       # If column list is empty no rows get removed
  f     E        # filter the given matrix by each row T
     xkT         # Get the indices of all occurences of an emtpy string (k) 
   -Q            # If the indices match with the given column indices, remove the row
    
     

Update: My first solution handled an empty list of column indices wrong. Fixed it (pretty ugly) at the cost of 3 Bytes. Gonna try to do it better after work...

Update 2: Golfed it down to 10 9 7 bytes, with some help from @FryAmTheEggman an by improving the algorithm significantly.

Update3: Fixed a bug @ThomasKwa discovered. His proposed 7-byte solution did not handle empty column indices right, so I just catch that case with a ternary here. I don't see how I can shorten this atm.

CJam, 18 bytes

{{1$\f=_!\se|},\;}

An unnamed block (function) expecting the matrix and the zero-based column indices on the stack (the matrix on top), which leaves the filtered matrix on the stack. I'm using the empty array "" as the non-numeric value.

Test it here.

Explanation

{     e# Filter the matrix rows based on the result of this block...
  1$  e#   Copy the column indices.
  \f= e#   Map them to the corresponding cell in the current row.
  _!  e#   Duplicate, logical NOT. Gives 1 for empty column list, 0 otherwise.
  \s  e#   Convert other copy to string. If the array contained only empty arrays, this 
      e#   will be an empty string which is falsy. Otherwise it will contain the numbers 
      e#   that were left after filtering, so it's non-empty and truthy.
  e|  e#   Logical OR.
},
\;    e# Discard the column indices.