Connect to spark cluster

from pyspark.conf import SparkConf
from pyspark.sql import SparkSession, Catalog

spark_conf = SparkConf()
spark_conf.setAll([
    ('spark.master', 'spark://<master-ip>:7077'),
    ('spark.app.name', 'myApp'),
    ('spark.submit.deployMode', 'client'),
    ('spark.ui.showConsoleProgress', 'true'),
    ('spark.eventLog.enabled', 'false'),
    ('spark.logConf', 'false'),
    ('spark.driver.bindAddress', '0.0.0.0'),
    ('spark.driver.host', '<master-ip>')
])

spark = SparkSession.builder.config(conf=spark_conf).getOrCreate()

Similar codes

1 vote307 views1 answer
Mean of torch tensor
Python
0 vote397 views2 answers
How to refresh page in flask
Python Flask
1 vote314 views3 answers
How to access variable of one function in another function in python
Python
0 vote351 views1 answer
Popup flutter
Python
5 votes350 views2 answers
A2+b2=c2 calculator
Python
13 votes329 views2 answers
Map function using lambda in python
Python
1 vote326 views1 answer
Python turtle shooting game
Python
0 vote348 views1 answer
Count total number of null, isna sum python
Python
0 vote317 views1 answer
Python django os error
Python
0 vote344 views1 answer
Read text file line by line using the readline() function
Python