/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

8 votes285 views3 answers
How to maximize pandas output python
Python
0 vote263 views1 answer
Find highest value in array python
Python
0 vote272 views1 answer
Get next multiple of a number
Python
1 vote311 views2 answers
Runserver coomand in django
Python Django
0 vote292 views1 answer
Python how to acquire the html code for a website
Python
1 vote285 views1 answer
How to urllib3
Python
0 vote350 views1 answer
Arabic text recognition from pdf using python
Python
-2 votes262 views1 answer
Python IOError
Python
8 votes263 views2 answers
Determinant of matrix in python
Python
0 vote276 views1 answer
Pandas to tensor torch
Python