Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

2 votes120 views3 answers
Python download for ubuntu 20.04
Python
0 vote106 views1 answer
Decreasing for loop python
Python
0 vote109 views1 answer
Python take input without displaying it
Python
2 votes114 views3 answers
Python datetime no milliseconds
Python
1 vote108 views2 answers
Convert timestamp to period pandas
Python
4 votes133 views2 answers
Change django admin title
Python Django
22 votes111 views6 answers
Add python to path
Python
4 votes98 views1 answer
.text python
Python
2 votes113 views3 answers
Cosine interpolation
Python
0 vote113 views1 answer
Yml anaconda
Python