Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

8 votes197 views3 answers
Np.arange in python
Python
0 vote183 views1 answer
Parcours de graph en largeur
Python
0 vote166 views1 answer
2d list input in python
Python
0 vote168 views1 answer
Arabert
Python
0 vote216 views1 answer
Calculate term frequency python
Python
1 vote228 views1 answer
Robot framework log from python
Python
4 votes191 views3 answers
Txt to list python
Python
22 votes187 views6 answers
Python strings
Python
-1 vote220 views1 answer
Pyhton mcq
Python
4 votes232 views2 answers
Convert array to dataframe python
Python