/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote247 views1 answer
Python regular expression escape vertical bar |
Python
1 vote241 views1 answer
Assert len(lex) < self.bucket_specs[-1][1]
Python
0 vote260 views1 answer
Watershed Segmentation
Python
2 votes265 views1 answer
ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True site:stackoverflow.com
Python
0 vote241 views1 answer
Program to add first and last digit of a number in python
Python
0 vote254 views1 answer
IBM Maximo automation python
Python
0 vote220 views1 answer
Hebrew range
Python
0 vote241 views1 answer
CMake Error at pybind11/tools/FindPythonLibsNew.cmake:131 (message): Python config failure:
Python
3 votes193 views1 answer
Cannot import name 'flask' from partially initialized module 'flask' (most likely due to a circular import)
Python
0 vote237 views3 answers
Pandas replace nulls with zeros
Python