/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote258 views1 answer
Matlab index last element
Python
0 vote273 views1 answer
Python running mean pandas
Python
0 vote283 views1 answer
Scree plot sklearn
Python
15 votes294 views2 answers
Delete the duplicates in python
Python
34 votes263 views2 answers
Python create nested directory
Python
3 votes280 views1 answer
Django flush
Python
1 vote286 views1 answer
Select rows which entries equals one of the values pandas
Python
3 votes285 views1 answer
Dns request scapy
Python
149 votes304 views6 answers
How to make a random number generator in python
Python
0 vote260 views2 answers
Json to base64 python
Python