Monitoring¶
General documentation on monitoring Spark jobs can be found at: https://spark.apache.org/docs/latest/monitoring.html
Command Line¶
A listing of all Spark applications can be found along with their Application ID by typing:
yarn application -list
To cancel a job, users can issue the following command with the Application ID obtained above.
yarn application -kill <Application ID>
Web UI¶
Note
Accessing the web UI requires you to be connected to the Booth VPN vpn.chicagobooth.edu
Spark history server¶
YARN¶
Apache Hadoop YARN is the agent responsible for managing resources on Vulcan. The following URL provides a listing of YARN applications where users can find their application logs.