> For the complete documentation index, see [llms.txt](https://docs.trustlogix.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.trustlogix.io/data-sources/databricks.md).
# Databricks
To register a Databricks account into TrustLogix follow the below steps
1\. Create a data source entry in TrustLogix
2\. Run the auto generated notebooks in Databricks environment
3\. Gather all required information about Databricks account
4\. Update the details in TrustLogix and Save
#### **Databricks User Permissions To Register An Account**
* **Account Admin Permission**: Required to register a new Databricks data source in TrustLogix. The user initiating the registration must have Account Admin privileges on the Databricks account.
* For Registering New Workspace to an Existing Data Source
* **Workspace Admin**: Required to execute the necessary notebooks for workspace registration.
* **Metastore Admin**: Required to execute the necessary notebooks for workspace registration.
**Note:** Both Workspace Admin and Metastore Admin permissions are required on the Databricks side to successfully register a new workspace to an existing Databricks data source.
#### Permissions Required by TrustLogix Service Principal(s)
* TrustLogix uses service principals to access your Databricks environment. Two separate service principals are used depending on the feature components. The permissions required vary based on your deployment model.
* Permissions required to access **TrustLogix and System Catalogs** for Control Plane and Data Plane service principals based on deployment model
* `USE CATALOG, USE SCHEMA, BROWSE, CREATE FUNCTION, CREATE TABLE, SELECT, MANAGE, MODIFY` on `TLX_POLICY_DB` catalog
* `USE CATALOG, USE SCHEMA, SELECT` on `system` catalog
* `CAN USE` on `TLX_WAREHOUSE` compute
* `User` permission on all registered workspaces within the TrustLogix Console
* Data Plane Service Principal Permissions for Trustlet or Hybrid tenants
* The data plane service principal requires the following permissions on securable catalogs:
* `USE CATALOG` - Allow the service principal to use the catalog
* `USE SCHEMA` - Allow the service principal to use schemas within the catalog
* `BROWSE` - Allow browsing of catalog objects
* `SELECT` - Allow read access to tables and views
* `MANAGE` - Allow management of catalog objects
* Control Plane Service Principal Permissions for Trustlet or Hybrid tenants
* The control plane service principal requires the following permissions on securable catalogs:
* `USE CATALOG` - Allow the service principal to use the catalog
* `USE SCHEMA` - Allow the service principal to use schemas within the catalog
* `BROWSE` - Allow browsing of catalog objects
* `APPLY TAG` - Allow applying tags to securable objects
* SaaS Tenant Deployment
* For SaaS tenant deployments, a single TrustLogix service principal is used with the following permissions on securable catalogs:
* `USE CATALOG` - Allow the service principal to use the catalog
* `USE SCHEMA` - Allow the service principal to use schemas within the catalog
* `BROWSE` - Allow browsing of catalog objects
* `SELECT` - Allow read access to tables and views
* `MANAGE` - Allow management of catalog objects
* `APPLY TAG` - Allow applying tags to securable objects
* **Important Notes on Service Principal Permissions**
* **No Admin Permissions Required**: TrustLogix service principals do **NOT** require Account Admin, Workspace Admin, or Metastore Admin permissions to function properly.
* **Principle of Least Privilege**: Service principals are granted only the minimum permissions necessary to perform their designated functions.
* **Group Membership Feature Limitation**: The group membership-related feature in Access Analyzer will be **disabled** by default for TrustLogix service principals.
* **To Enable Group Membership Feature**: If you need to enable group membership-related features in Access Analyzer, the TrustLogix service principal must have at least **Workspace Admin** or **Account Admin** permissions.
#### **Create a** Databricks **Data Source In TrustLogix**
* Login to TrustLogix console and register a new Databricks data source from Menu → Data sources → Register new data source
* Provide the data source name, & click on Register data source
* Form will be refreshed with additional fields and script Download option
**Run the notebooks in** Databricks **environment**
* Click on Download button to download the notebooks
* Extract the downloaded Zip file which will contain **multiple** Python notebook files
* Ensure that each of the notebook prerequisites are performed before that particular notebook is run by providing all the arguments like the Account ID, Service Principal Name, Access Token etc.
* After successful run of each of the notebooks, required objects are created in Databricks
* Capture the output of the notebook and also gather
* Account ID
* Workspace Deployment Name
* SQL Warehouse Identifier
* Service Principal Name
* Client ID
* Client Secret
#### How to register additional workspaces to an existing Databricks data source
The following are necessary to add additional workspaces to an existing Databricks data source
1. TrustLogix Service Principal Id
2. TrustLogix Client Id
3. TrustLogix DP Service Principal Id(Only for trustlet/hybrid tenant)
4. TrustLogix DP Client Id(Only for trustlet/hybrid tenant)
5. Workspace URL e.g adb-180424056746427.6.azuredatabricks.net
Here, 1, 2 belong to the TrustLogix Control Plane Service Principal and 3, 4 belong to the DataPlane Service Principal. You can get the above from the Results section of the initial tlx\_databricks\_grants notebook used while registering Databricks account in the TrustLogix console.
**Step-1: Run following notebooks**
1. tlx\_databricks\_grants\_register\_workspace.ipynb
2. tlx\_databricks\_grants\_to\_cp\_service\_principal\_scheduler.ipynb(Only for trustlet/hybrid tenant)
3. tlx\_databricks\_workspace\_admin\_grant.ipynb
\#1 and #3 can be run from any workspace. #2 need to be run from new workspace to be registered.
> To execute the above notebooks in Databricks, it is suggested that the user or service principal must have a minimum of the following permissions:
>
> * **Workspace Admin**
> * **Metastore Admin**
>
> Account Admin permission not required.
**Step-2: Add the workspaces and warehouses in the TrustLogix console**
* Navigate back to TrustLogix data source form page
* Update following for the new workspace
* Workspace Deployment Name(s)
* SQL Warehouse Identifier(s)
* Click on save, system will validate the connection details and saves them
* If all details are valid, Databricks datasource should be updated successfully
**Step-3: Update secrets on AWS SSM (or Azure Key Vault) corresponding to the new workspace**
```
tlx---jdbcurl
tlx---warehouse
tlx---oauthEndpoint
```
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.trustlogix.io/data-sources/databricks.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.