When I start pyspark, a SparkSession is automatically generated and available as 'spark'. I would like to print/view the details of the spark session but am having a lot of difficulty accessing these parameters.
Pyspark auto creates a SparkSession. This can be created manually using the following code:
- Code: Select all
from pyspark.sql import SparkSession
spark = SparkSession\
I would like to view/print the appname and config options. The reason I would like to see these is as a result of another issue that I am experiencing which this may shed light on.
Thank for any insight, the help is very much appreciated, thanks Michael