Is there an R function to extract only numbers from a comma-separated string with many NA values to create a column with only the numbers?

An option is to loop through the row with apply, remove the NA elements and paste it together

before$new <- apply(before, 1, function(x) toString(x[!is.na(x)]))
before$new
#[1] "1, 3, 4" "4"       "1"       "2"       "3, 4"    "1, 3"   

Another possibility could be:

before$rowid <- 1:nrow(before)
aggregate(values ~ rowid, 
          paste0, collapse = ",",
          data.frame(before[5], stack(before[-5])))

  rowid values
1     1  1,3,4
2     2      4
3     3      1
4     4      2
5     5    3,4
6     6    1,3

foo = function(..., sep = ","){
    paste(..., sep = sep)
}

gsub(",?NA|NA,?", "", do.call(foo, before))
#[1] "1,3,4" "4"     "1"     "2"     "3,4"   "1,3"