/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote293 views1 answer
Get mode dataframe
Python
-1 vote314 views1 answer
Import "reportlab.pdfgen.canvas" could not be resolved
Python
-1 vote325 views1 answer
Check runtime python
Python
0 vote326 views1 answer
Remove tab space from string in python
Python
1 vote352 views2 answers
Django sum get 0 if none
Python Django
0 vote325 views1 answer
# Import KNeighborsClassifier from sklearn.neighbors
Python
22 votes337 views6 answers
How to merge two dataframes
Python
2 votes325 views1 answer
Plot histogram in seaborn
Python
20 votes342 views1 answer
How to iterate through images in a folder python
Python
0 vote313 views1 answer
Find highest correlation pairs pandas
Python