Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

1 vote188 views1 answer
How to add decorators with class in django
Python Django
1 vote301 views1 answer
Python Split a file path into root and extension
Python
19 votes157 views1 answer
10 minute timer
Python
0 vote160 views1 answer
How to sharpen image in python using cv2
Python
4 votes164 views2 answers
How to get runtime of a function in python
Python
1 vote214 views1 answer
Django manager
Python Django
0 vote167 views2 answers
How to download file in python
Python
0 vote186 views1 answer
Input function in django
Python Django
-2 votes181 views2 answers
Python find lcm
Python
0 vote170 views1 answer
Windows error message python
Python