Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote266 views1 answer
Convert price to float python
Python
0 vote252 views1 answer
Finns = False
Python
2 votes266 views1 answer
ValueError: Found array with dim 3. Estimator expected <= 2.
Python
0 vote270 views1 answer
Day 14 scope hackerrank solution in python
Python
2 votes208 views2 answers
Python install docx
Python
0 vote276 views1 answer
Celery timezone setting django
Python Django
0 vote254 views2 answers
Django edit object foreign key id
Python
2 votes275 views1 answer
Check corently installed epython version
Python
9 votes261 views2 answers
Tar unpack to folder
Python
1 vote261 views1 answer
TypeError: strptime() argument 1 must be str, not Series
Python