Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote285 views1 answer
Stellargraph python
Python
22 votes310 views6 answers
Python numpy array to list
Python
0 vote308 views1 answer
Reload is not defined python 3
Python
1 vote267 views1 answer
No module named "pyttsx3"
Python
1 vote296 views1 answer
How to hide axis in matplotlib
Python
2 votes268 views4 answers
Update link python is python 3
Python
0 vote336 views1 answer
Broadcast in python
Python
0 vote293 views1 answer
Dont show certain legend labels
Python
18 votes324 views4 answers
Reset index python
Python
1 vote265 views1 answer
Add seconds to datetime python
Python