Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote206 views1 answer
Python Remove Character from String using replace()
Python
84 votes377 views2 answers
Python with statement file does not exist exception
Python
0 vote209 views1 answer
Length of list without len function
Python
0 vote212 views1 answer
Vscode how to extend output size in jupyter notebook
Python
1 vote221 views1 answer
Python edit item in list
Python
4 votes214 views2 answers
Add y axis label matplotlib
Python
4 votes221 views2 answers
ImportError: No module named django.core.wsgi
Python
3 votes219 views3 answers
Python check ram usage
Python
1 vote302 views2 answers
Django password hashing
Python Django
0 vote217 views2 answers
Turtle keep window open
Python