Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

7 votes256 views1 answer
Learn python the hard way pdf
Python
10 votes304 views3 answers
Image to text python
Python
0 vote254 views1 answer
Django extend user model
Python
0 vote255 views1 answer
Fatal Python error: init_interp_main: can't initialize time
Python
3 votes262 views4 answers
If or Python
Python
1 vote292 views2 answers
Fast fourier transform python
Python
2 votes277 views2 answers
Check string similarity python
Python
0 vote282 views1 answer
Python http handler iis
Python
1 vote227 views1 answer
Flask app route in different files
Python
0 vote260 views1 answer
Tkinter mouse loading cursor
Python