Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

28 votes551 views3 answers
Django abstractuser
Python Django
0 vote334 views1 answer
Django creat app return _bootstrap._gcd_import
Python Django
0 vote340 views1 answer
Brew install pip python3
Python
0 vote333 views1 answer
Install reportlab python
Python
6 votes375 views1 answer
Python soap
Python
0 vote336 views1 answer
How to capture multiple screens with ImageGrab
Python
0 vote353 views1 answer
Hungarian algorithm python
Python
0 vote352 views1 answer
Divide by zero error python exception handling
Python
0 vote353 views1 answer
How to let the user input desmials in python
Python
0 vote332 views1 answer
TypeError: 'frozenset' object is not callable
Python