Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

1 vote199 views3 answers
Python pdf to excel
Python
0 vote283 views2 answers
Box detection
Python
0 vote275 views1 answer
Flask sqlalchemy query specific columns
Python Flask
0 vote250 views1 answer
How to chose right epoch
Python
5 votes270 views1 answer
Python test is nan
Python
0 vote260 views2 answers
Django run manage.py from python
Python Django
7 votes273 views3 answers
How to delete migrations in django
Python Django
0 vote293 views2 answers
Django template date format yyyy-mm-dd
Python Django
7 votes263 views1 answer
Replace cell pandas
Python
0 vote256 views1 answer
Wifite subsystem
Python