Dropping multiple columns from Spark dataframe by Iterating through the columns from a Scala List of Column names

If you just want to do nothing more complex than dropping several named columns, as opposed to selecting them by a particular condition, you can simply do the following:

df.drop("colA", "colB", "colC")

Answer:

val colsToRemove = Seq("colA", "colB", "colC", etc) 

val filteredDF = df.select(df.columns .filter(colName => !colsToRemove.contains(colName)) .map(colName => new Column(colName)): _*) 

This should work fine :

val dropList : List[String]  |
val df : DataFrame  |
val test_df = df.drop(dropList : _*)