/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

11 votes255 views6 answers
Python merge dict
Python
20 votes268 views2 answers
Fastapi
Python
0 vote673 views1 answer
Pandas calculate same day 3 months ago dateoffset
Python
13 votes251 views3 answers
Np.hstack
Python
0 vote243 views1 answer
Python import multiple csv
Python
18 votes253 views2 answers
Get ascii value of char python online
Python
-1 vote285 views1 answer
Create list of unique values from dictionary
Python
2 votes248 views5 answers
Torch.stack
Python
0 vote256 views1 answer
Huggingface default cache dir
Python
0 vote303 views1 answer
Join a list of items with different types as string in Python
Python