/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote286 views4 answers
Process.join()
Python
0 vote303 views1 answer
%d%m%y python
Python
1 vote289 views1 answer
No such table: django_session admin
Python
-3 votes308 views4 answers
Python datetime format yyyy-mm-dd
Python
-1 vote282 views1 answer
Print multiplication table python
Python
6 votes308 views5 answers
Bar plot matplotlib
Python
0 vote275 views1 answer
Pyqt5 qpushbutton disable
Python
0 vote289 views1 answer
Streaming upload requests python
Python
5 votes291 views2 answers
Pandas conditional collumn
Python
0 vote284 views1 answer
Name decorator in python
Python