Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

1 vote161 views1 answer
Python if boolean
Python
1 vote217 views2 answers
How to swap two variables without using third variable python
Python
0 vote201 views1 answer
Datetime column only extract date pandas
Python
7 votes201 views2 answers
Python how to use logarithm
Python
0 vote193 views1 answer
Get height of image in pygame
Python
1 vote205 views1 answer
Python how to create attribute of class while iterating a list
Python
9 votes197 views3 answers
How to make a random variable in python
Python
6 votes195 views2 answers
Max int value in python
Python
0 vote194 views1 answer
Code suggestion html not working in django-html
Python
0 vote200 views1 answer
Tkinter hide widget
Python