Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

52 votes341 views2 answers
Index in list
Python
16 votes346 views6 answers
Joins in pandas
Python
0 vote362 views1 answer
Difference in django project view and app view
Python Django
0 vote355 views1 answer
Shape of passed values is indices imply
Python
0 vote354 views1 answer
Python - How to subtract values from dictionaries
Python
3 votes377 views3 answers
Import flask
Python
0 vote324 views1 answer
For loop practice problems python
Python
0 vote356 views3 answers
Boto3 python s3
Python
0 vote357 views1 answer
Python random phone number
Python
0 vote337 views1 answer
How to send variable to python using xlwings
Python