Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

2 votes260 views5 answers
Check boolean python
Python
0 vote274 views1 answer
Python get image's text from web request
Python
0 vote274 views1 answer
Legend of colorbar python
Python
124 votes291 views6 answers
Random range python
Python
0 vote291 views1 answer
Mechanize python #2
Python
8 votes256 views3 answers
How to playsound in python
Python
0 vote282 views1 answer
How to make booking website using django step by step
Python
12 votes303 views6 answers
Curl in python
Python
1 vote268 views1 answer
Python head function show all columns
Python
-1 vote267 views4 answers
Python creare decoratori
Python