Menu
Newbedev LogoNEWBEDEV Python Javascript Linux Cheat sheet
Newbedev LogoNEWBEDEV
  • Python 1
  • Javascript
  • Linux
  • Cheat sheet
  • Contact

How to drop multiple column names given in a list from Spark DataFrame?

You can use the * operator to pass the contents of your list as arguments to drop():

df.drop(*drop_lst)

You can give column name as comma separated list e.g.

df.drop("col1","col11","col21")

This is how drop specified number of consecutive columns in scala:

val ll = dfwide.schema.names.slice(1,5)
dfwide.drop(ll:_*).show

slice take two parameters star index and end index.

Tags:

Dataframe

Apache Spark

Pyspark

Apache Spark Sql

Pyspark Sql

Related

How do I check if IEnumerable has a single element? Should I erase build number when uploading a new version? Is there an XML schema extension for Visual Studio Code? 2d-Array with more than 65535^2 elements --> Array dimensions exceeded supported range Babel Plugin/Preset files are not allowed to export objects, only functions How to mock a csv file masking image with CSS Xunit 2.3.0 Unable to pass dates as inline params REALM - System.TypeInitializationException: The type initializer for 'Realms.Realm' threw an exception. ---> System.DllNotFoundException How to read form values in controller? How to exclude embeded Tomcat in Spring Boot application How to update/insert an other document in cloud firestore on receiving a create event for a collection using functions

Recent Posts

Pandas how to find column contains a certain value Recommended way to install multiple Python versions on Ubuntu 20.04 Build super fast web scraper with Python x100 than BeautifulSoup How to convert a SQL query result to a Pandas DataFrame in Python How to write a Pandas DataFrame to a .csv file in Python
© 2021 newbedevPrivacy Policy