/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote313 views1 answer
Django 2.2 disable cache settings.STATIC_URL
Python Django
0 vote255 views1 answer
Dataframe memory usage
Python
2 votes256 views1 answer
Python method to check for uppercase character
Python
4 votes272 views3 answers
How to append variable in Python
Python
2 votes268 views1 answer
Sacar la posicion en una lista python
Python
-1 vote283 views1 answer
Python tkinter focus on entry
Python
0 vote346 views2 answers
Plt.xlabel not working
Python
0 vote285 views1 answer
Name 'StringType' is not defined
Python
0 vote253 views1 answer
Latex maths to python parser
Python
3 votes275 views1 answer
Numpy arrauy to df
Python