/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

9 votes220 views2 answers
Django app
Python
1 vote224 views1 answer
Add directory to pythonpath(in ~/.bashrc
Python
1 vote265 views1 answer
Python towers of hanoi recursive
Python
0 vote239 views1 answer
Inverse mask python
Python
3 votes276 views1 answer
Django import model from another app
Python Django
0 vote252 views2 answers
Python book
Python
11 votes230 views5 answers
Python sorted lambda
Python
0 vote236 views2 answers
.size pandas
Python
-2 votes263 views1 answer
How to get data from django session
Python Django
3 votes221 views2 answers
Python 3.9 ModuleNotFoundError: No module named 'distutils.sysconfig'
Python