/Python/Groups the DataFrame using the specified columns

Groups the DataFrame using the specified columns

# Groups the DataFrame using the specified columns

df.groupBy().avg().collect()
# [Row(avg(age)=3.5)]
sorted(df.groupBy('name').agg({'age': 'mean'}).collect())
# [Row(name='Alice', avg(age)=2.0), Row(name='Bob', avg(age)=5.0)]
sorted(df.groupBy(df.name).avg().collect())
# [Row(name='Alice', avg(age)=2.0), Row(name='Bob', avg(age)=5.0)]
sorted(df.groupBy(['name', df.age]).count().collect())
# [Row(name='Alice', age=2, count=1), Row(name='Bob', age=5, count=1)]

Source: spark.apache.org

Similar codes

17 votes169 views4 answers
Inverse matrix numpy
Python
0 vote167 views1 answer
Pytorch freeze layers
Python
7 votes192 views1 answer
Float to percentage python
Python
2 votes187 views1 answer
Tensorflow reduce_sum
Python
0 vote173 views1 answer
Python define random variables name
Python
12 votes158 views2 answers
How to shutdown a computer with python
Python
1 vote177 views1 answer
Get adjacent cells in grid
Python
2 votes186 views2 answers
Python format 001
Python
4 votes175 views3 answers
How to create one list from 2d list python
Python
2 votes183 views1 answer
Python apply a function to a list inplace
Python