Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote215 views1 answer
Tessa thompson
Python
0 vote197 views1 answer
Cieling function pandas
Python
3 votes208 views1 answer
Mouse bottom in pygame
Python
1 vote216 views1 answer
Time it in jupyter notebook
Python
0 vote209 views1 answer
How to play mp3 audio in python
Python
204 votes269 views6 answers
Random int python
Python
37 votes210 views5 answers
Get requests python
Python
0 vote249 views1 answer
Threshold meaning in pandas dropna
Python
0 vote287 views1 answer
Discord pycord add a URL button in a subclassed view
Python
0 vote223 views1 answer
Create_user() missing 1 required positional argument: 'username'
Python