/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

0 vote340 views1 answer
Np.arange and np.linspace difference
Python
26 votes353 views3 answers
Discord py bot status
Python
13 votes383 views4 answers
Rotate xticks matplotlib
Python
16 votes333 views3 answers
Python split range equally
Python
1 vote334 views2 answers
How to convert datetime to integer in python
Python
15 votes319 views3 answers
Max int python
Python
-1 vote325 views2 answers
Scaling
Python
1 vote325 views1 answer
Breaking big csv into chunks pandas
Python
0 vote364 views1 answer
How to check how many key value pairs are in a dict python
Python
13 votes342 views4 answers
Even in python
Python