/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

1 vote156 views2 answers
Python long
Python
1 vote137 views1 answer
Matplotlib remove x and y labels
Python
0 vote219 views1 answer
How to create a spark schema using a string
Python
1 vote179 views3 answers
Uninstall python ubuntu 18.04
Python
1 vote211 views1 answer
Get a list of open applications python
Python
0 vote175 views1 answer
Stackoverflow: install old version of networkx
Python
1 vote181 views1 answer
Pyspark when otherwise multiple conditions
Python
0 vote161 views1 answer
Renaming Column Name Dataframe
Python
0 vote204 views3 answers
Django forms I cannot save picture file
Python Django
2 votes169 views2 answers
What is role of ALLOWED_HOSTs in Django
Python