/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

1 vote169 views1 answer
Time it in jupyter notebook
Python
0 vote173 views1 answer
Data don't retrieve from database view in django
Python Django
60 votes154 views4 answers
Datetime year python
Python
0 vote183 views1 answer
Django model plural
Python Django
1 vote151 views1 answer
Python install tabulate
Python
5 votes160 views1 answer
Python how move file to directory
Python
1 vote156 views3 answers
Python extract zip file without directory structure
Python
4 votes160 views6 answers
Python subtract days from date
Python
31 votes176 views6 answers
How to remove whitespace from string in python
Python
0 vote170 views2 answers
Fibonacci series python program
Python