/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote260 views1 answer
Convert float with missing values to integer
Python
0 vote219 views1 answer
Python anonymous function
Python
9 votes221 views6 answers
Python sort list
Python
0 vote216 views1 answer
Python pyqt5 checkbox trigger
Python
0 vote238 views1 answer
Print lowercase in python
Python
0 vote218 views1 answer
Recover dict from 0-d numpy array
Python
1 vote217 views1 answer
Complete the function digits(n) that returns how many digits the number has.
Python
0 vote243 views1 answer
What is * in argument list in python
Python
4 votes263 views2 answers
Make a nested list flat python
Python
8 votes223 views4 answers
Venv activate windows 10
Python