Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

2 votes239 views1 answer
Replace space with _ in pandas
Python
5 votes236 views1 answer
Append item to array python
Python
2 votes242 views1 answer
Python sort dataframe by list
Python
1 vote239 views2 answers
If condition in print statement python
Python
14 votes311 views5 answers
Remove extra spaces python
Python
0 vote247 views1 answer
Use mongo replica set python
Python
0 vote242 views1 answer
Discord.py get client avatar
Python
0 vote237 views1 answer
Model.fit(X_train, Y_train, batch_size=80, epochs=2, validation_split=0.1)
Python
0 vote281 views1 answer
Flask read form data
Python Flask
0 vote234 views1 answer
Read://https_www.tumblr.com/?url=https://www.tumblr.com/login?redirect_to=%2Fneue_web%2Fiframe%2Freblog%2F629907744681590784%2FjQw7OUs8&3739a18c-0c68-43cc-a4cb-b8b99e9bfd72=a52e06db-92b6-4b86-b3c5-fa2ab267405c
Python