Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote239 views1 answer
What is a rare earth
Python
0 vote269 views1 answer
Use of numpy matrix in tkinter python 3
Python
26 votes246 views6 answers
List files python
Python
0 vote269 views1 answer
Sikuli guide text
Python
0 vote278 views4 answers
Python char to hex
Python
3 votes207 views1 answer
Python list insert at front
Python
1 vote274 views2 answers
Python not null
Python
0 vote242 views1 answer
How do you amke function in python
Python
7 votes252 views1 answer
Filter dataframe multiple conditions
Python
2 votes267 views2 answers
No module named 'storages'
Python