> For the complete documentation index, see [llms.txt](https://docs.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.collibra.com/data-quality-self-hosted/connections/supported-connections/supported-remote-files/connecting-to-hadoop-distributed-file-system-hdfs.md).

# Connecting to Hadoop Distributed File System (HDFS)

This section contains an overview of Hadoop Distributed File System (HDFS).

## General information

<table><thead><tr><th width="152">Field</th><th>Description</th></tr></thead><tbody><tr><td>Data source</td><td>Hadoop Distributed File System (HDFS)</td></tr><tr><td>Supported versions</td><td>N/A</td></tr><tr><td>Connection string</td><td><code>hdfs://</code></td></tr><tr><td>Packaged?</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr><tr><td>Certified?</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr><tr><td colspan="2">Supported features</td></tr><tr><td>Analyze data</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr><tr><td>Archive breaking records</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="line"> No</td></tr><tr><td>Estimate job</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr><tr><td>Pushdown</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="line"> No</td></tr><tr><td colspan="2">Processing capabilities</td></tr><tr><td>Spark agent</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr><tr><td>Yarn agent</td><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td></tr></tbody></table>

## Minimum user permissions

In order for Collibra DQ to access your HDFS bucket, you need the following permissions.

* Read access to the path in your HDFS connection.

## Recommended and required connection properties

<table><thead><tr><th>Required</th><th>Connection Property</th><th>Type</th><th>Value</th></tr></thead><tbody><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td><td>Name</td><td>Text</td><td>The unique name of your connection. Do not use spaces in your connection name and only use valid characters.</td></tr><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td><td>Connection URL</td><td>String</td><td><p>The connection string path of your HDFS connection. The path must start with <code>hdfs://</code> and point to the root bucket, not a sub-folder.</p><div data-gb-custom-block data-tag="tabs"><div data-gb-custom-block data-tag="tab" data-title="Example" data-icon="glasses-round"><p><code>hdfs://&#x3C;name node>:8020/</code></p></div></div></td></tr><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td><td>Target Agent</td><td>Option</td><td>The <a href="/data-quality-self-hosted/installation/agent.md">Agent</a> used to submit your DQ Jobs.</td></tr><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td><td>Auth Type</td><td>Option</td><td><p>The method to authenticate your connection.</p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The configuration requirements are different depending on the Auth Type you select. See <a href="#authentication">Authentication</a> for more details on available authentication types.</p></div></td></tr><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-d06dd40622fc7d030e21f7c0303a90eb28d229fc%2Fok-large%402x.png?alt=media" alt="" data-size="line"> Yes</td><td>Save Credentials</td><td>Option</td><td>Select this option after you enter your connection details.</td></tr><tr><td><img src="https://4121921244-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FWCrP7Albs8DfJjIQxUxw%2Fuploads%2Fgit-blob-de45f34193ae06a2b093c2dfb8fb80cfda945166%2Fclose-large%402x.png?alt=media" alt="" data-size="line"> No</td><td>Properties</td><td>String</td><td><p>The configurable driver properties for your connection. Multiple properties must be semicolon-delimited, and each property=value pair, including the last one, must end with a semicolon (;). <code>abc=123;test=true;</code></p><p>To ensure that your remote procedures are secure within Collibra DQ, we recommend defining the following driver property:</p><pre><code>hadoop.rpc.protection=privacy
</code></pre></td></tr></tbody></table>

## Authentication

Select an authentication type from the dropdown menu. The options available in the dropdown menu are the currently supported authentication types for this data source.

{% tabs %}
{% tab title="Keytab" %}

<table><thead><tr><th width="152">Field</th><th>Description</th></tr></thead><tbody><tr><td>Principal</td><td>The service principal used to let Collibra DQ access your connection.</td></tr><tr><td>Key</td><td>The keytab used to authorize your connection.</td></tr></tbody></table>
{% endtab %}

{% tab title="TGT Cache" %}

<table><thead><tr><th width="152">Field</th><th>Description</th></tr></thead><tbody><tr><td>Principal</td><td>The service principal used to let Collibra DQ access your connection.</td></tr><tr><td><a href="/data-quality-self-hosted/connections/authentication/create-kerberos-authentication-files.md">TGT</a></td><td>The Ticket Granting Ticket used to authorize your connection.</td></tr></tbody></table>
{% endtab %}
{% endtabs %}
