/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

1 vote403 views1 answer
Insert column at specific position in pandas dataframe
Python
1 vote338 views1 answer
Hoe maak je machten in python
Python
40 votes347 views3 answers
Train dev test split sklearn
Python
1 vote330 views2 answers
Python cmd colors
Python
0 vote340 views1 answer
Insertar en una lista anidada python
Python
0 vote335 views1 answer
Assign elements of list to variables python
Python
0 vote345 views2 answers
Django run command from code
Python
0 vote389 views1 answer
Extra import on django
Python Django
0 vote336 views1 answer
Inverse box-cox transformation python
Python
0 vote340 views1 answer
How to find the indexes of a substring in a string in python
Python