/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote163 views1 answer
Get label text tkinter
Python
0 vote225 views1 answer
Conda install nltk
Python
0 vote210 views1 answer
Cant import flask mail
Python
0 vote266 views1 answer
Recall calculate confusion matrix .ravel()
Python
-2 votes222 views1 answer
Python for-while
Python
0 vote207 views1 answer
Custom auth django channels
Python
4 votes204 views1 answer
Matplotlib plot dashed
Python
5 votes203 views1 answer
Check django object exists
Python
1 vote216 views1 answer
Pandas percentage change across multiple periods
Python
0 vote220 views1 answer
How to use fastapi ApiClient with pytest
Python