Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

2 votes298 views3 answers
Sum array python
Python
2 votes384 views2 answers
Numpy print full array to srdout
Python
-2 votes315 views1 answer
Python telegram bot get user name
Python
3 votes321 views1 answer
Tick labels vertical matplotlib
Python
0 vote383 views1 answer
Get name of a file in python
Python
5 votes375 views2 answers
Df.shape 0
Python
0 vote378 views1 answer
Python def text
Python
2 votes363 views1 answer
Convert float in datetime python
Python
0 vote402 views1 answer
Django auto complete light styling
Python Django
38 votes358 views2 answers
Python create new folder if not exist
Python