Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote237 views1 answer
Django form field add attrs
Python
5 votes247 views3 answers
Sys python
Python
2 votes233 views2 answers
Iterate through attributes of class python
Python
1 vote248 views2 answers
Tensor get value
Python
0 vote263 views2 answers
Requests get cookies from response
Python
0 vote228 views1 answer
Numpy fill empty array
Python
0 vote270 views1 answer
Python elasticsearch docker from within other container
Python
0 vote246 views1 answer
Python get cos sim
Python
0 vote237 views1 answer
Change a color on touch roblox
Python
0 vote252 views1 answer
Apply boolean to list
Python