/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

2 votes165 views2 answers
Python redis
Python
0 vote174 views1 answer
Json python no whitespace
Python
1 vote157 views1 answer
How to download excel file from s3 using python
Python
0 vote205 views1 answer
Substring in a django template?
Python Django
0 vote169 views2 answers
How to round to 3 significant figures in python
Python
0 vote177 views1 answer
Python get column from grouped dataframe
Python
0 vote150 views1 answer
Hashing in python using quadratic probing
Python
2 votes154 views1 answer
Open image in numpy
Python
8 votes183 views3 answers
Django order by
Python
0 vote166 views10 answers
Logistic Regression with a Neural Network mindset python example
Python