/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote346 views1 answer
Pandas rename columns whitespace with underscore
Python
0 vote347 views1 answer
Run file as administrator python
Python
3 votes320 views5 answers
Calc
Python
0 vote372 views2 answers
Python def example
Python
0 vote362 views1 answer
Don't Get Volunteered!
Python
0 vote368 views1 answer
Matplotlib bold
Python
6 votes343 views3 answers
Df index drop duplicates
Python
1 vote367 views1 answer
Pyspark take random sample
Python
1 vote344 views1 answer
How to do processing on html file using python
Python
-1 vote397 views1 answer
Python scatterplot figsize
Python