> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/installation/installing-data-quality-observability-self-hosted-on-self-hosted-spark-standalone/spark-launch-scripts.md).

# Spark launch scripts

To launch a Spark Standalone cluster with the launch scripts, you should create a file called `conf/workers` in your Spark directory, which must contain the hostnames of all the machines where you intend to start Spark workers, one per line. If conf/workers does not exist, the launch scripts defaults to a single machine (localhost), which is useful for testing. Note, the master machine accesses each of the worker machines via ssh. By default, ssh is run in parallel and requires password-less (using a private key) access to be setup. If you do not have a password-less setup, you can set the environment variable SPARK\_SSH\_FOREGROUND and serially provide a password for each worker.

Once you’ve set up this file, you can launch or stop your cluster with the following shell scripts, based on Hadoop’s deploy scripts, and available in `SPARK_HOME/sbin`:

<table><thead><tr><th width="152">Shell Script</th><th>Description</th></tr></thead><tbody><tr><td><code>sbin/start-master.sh</code></td><td>Starts a master instance on the machine the script is executed on.</td></tr><tr><td><code>sbin/start-workers.sh</code></td><td>Starts a worker instance on each machine specified in the <code>conf/workers</code> file.</td></tr><tr><td><code>sbin/start-worker.sh</code></td><td>Starts a worker instance on the machine the script is executed on.</td></tr><tr><td><code>sbin/start-all.sh</code></td><td>Starts both a master and a number of workers as described above.</td></tr><tr><td><code>sbin/stop-master.sh</code></td><td>Stops the master that was started via the <code>sbin/start-master.sh</code> script.</td></tr><tr><td><code>sbin/stop-worker.sh</code></td><td>Stops all worker instances on the machine the script is executed on.</td></tr><tr><td><code>sbin/stop-workers.sh</code></td><td>Stops all worker instances on the machines specified in the <code>conf/workers</code> file.</td></tr><tr><td><code>sbin/stop-all.sh</code></td><td>Stops both the master and the workers as described above.</td></tr></tbody></table>

{% hint style="info" %}
These scripts must be executed on the machine you want to run the Spark master on, not your local machine.
{% endhint %}

## Starting and stopping Spark standalone

To start Spark standalone:

```
cd /home/owldq/owl/spark/sbin
./start-all.sh
```

To stop Spark standalone:

```
cd /home/owldq/owl/spark/sbin
./stop-all.sh
```

## Starting Spark standalone with separate workers

To start Spark Standalone with separate workers:

```
SPARK_WORKER_OPTS=" -Dspark.worker.cleanup.enabled=true -Dspark.worker.cleanup.interval=1799 -Dspark.worker.cleanup.appDataTtl=3600"
 
### 1 start master
/home/owldq/owl/spark/sbin/start-master.sh
 
### 2 start workers 
SPARK_WORKER_INSTANCES=3;/home/owldq/owl/spark/sbin/start-slave.sh spark://$(hostname):7077 -c 5 -m 20g
```