/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

17 votes163 views4 answers
Python delete value from dictionary
Python
27 votes162 views2 answers
Auto-py-to-exe with python3
Python
0 vote182 views1 answer
How to run loops 3 times in python
Python
10 votes167 views4 answers
Panda python
Python
0 vote192 views1 answer
Flask print request headers
Python
0 vote212 views1 answer
Python legend outside
Python
0 vote186 views1 answer
How to fill a shape with colour in python turtle
Python
0 vote164 views1 answer
Python access each group
Python
0 vote235 views1 answer
How to create my own exception in python
Python
3 votes239 views3 answers
Django.db.utils.OperationalError: no such table:
Python Django