Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote341 views1 answer
Np.loadtext
Python
1 vote304 views1 answer
Groupby as_index=false
Python
3 votes298 views1 answer
Nlargest
Python
1 vote356 views1 answer
Numpy series reset index
Python
1 vote309 views1 answer
Python clear memory
Python
0 vote290 views2 answers
Render template in django
Python
0 vote370 views1 answer
Ffff in decimal python
Python
5 votes334 views6 answers
Valid Palindrome
Python
0 vote309 views1 answer
How to define the range of values in seaborn heatmap
Python
0 vote317 views1 answer
Open link in tab new trong selenium python
Python