/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote109 views1 answer
Python import environment variables
Python
0 vote110 views1 answer
Pil
Python
0 vote177 views1 answer
Python generator for reading and writing file
Python
-1 vote139 views1 answer
Get git sha
Python
1 vote122 views1 answer
Django get form id from request
Python
14 votes68 views2 answers
Tkinter clear canvas
Python
5 votes116 views2 answers
Making variable if it is none python
Python
1 vote110 views2 answers
Json.stringify equivalent in python
Python
0 vote108 views1 answer
Dump ML Model
Python
0 vote108 views1 answer
Python construct a string
Python