Databricks
To register a Databricks account into TrustLogix follow the below steps
1. Create a data source entry in TrustLogix
2. Run the auto generated notebooks in Databricks environment
3. Gather all required information about Databricks account
4. Update the details in TrustLogix and Save
Databricks User Permissions To Register An Account
Account Admin Permission: Required to register a new Databricks data source in TrustLogix. The user initiating the registration must have Account Admin privileges on the Databricks account.
For Registering New Workspace to an Existing Data Source
Workspace Admin: Required to execute the necessary notebooks for workspace registration.
Metastore Admin: Required to execute the necessary notebooks for workspace registration.
Note: Both Workspace Admin and Metastore Admin permissions are required on the Databricks side to successfully register a new workspace to an existing Databricks data source.
Permissions Required by TrustLogix Service Principal(s)
TrustLogix uses service principals to access your Databricks environment. Two separate service principals are used depending on the feature components. The permissions required vary based on your deployment model.
Data Plane Service Principal Permissions for Trustlet or Hybrid tenants
The data plane service principal requires the following permissions on securable catalogs:
USE CATALOG- Allow the service principal to use the catalogUSE SCHEMA- Allow the service principal to use schemas within the catalogBROWSE- Allow browsing of catalog objectsSELECT- Allow read access to tables and viewsMANAGE- Allow management of catalog objectsMODIFY- Allow modification of catalog objects
Control Plane Service Principal Permissions for Trustlet or Hybrid tenants
The control plane service principal requires the following permissions on securable catalogs:
USE CATALOG- Allow the service principal to use the catalogUSE SCHEMA- Allow the service principal to use schemas within the catalogBROWSE- Allow browsing of catalog objectsAPPLY TAG- Allow applying tags to securable objects
SaaS Tenant Deployment
For SaaS tenant deployments, a single TrustLogix service principal is used with the following permissions on securable catalogs:
USE CATALOG- Allow the service principal to use the catalogUSE SCHEMA- Allow the service principal to use schemas within the catalogBROWSE- Allow browsing of catalog objectsSELECT- Allow read access to tables and viewsMANAGE- Allow management of catalog objectsMODIFY- Allow modification of catalog objectsAPPLY TAG- Allow applying tags to securable objects
Important Notes on Service Principal Permissions
No Admin Permissions Required: TrustLogix service principals do NOT require Account Admin, Workspace Admin, or Metastore Admin permissions to function properly.
Principle of Least Privilege: Service principals are granted only the minimum permissions necessary to perform their designated functions.
Group Membership Feature Limitation: The group membership-related feature in Access Analyzer will be disabled by default for TrustLogix service principals.
To Enable Group Membership Feature: If you need to enable group membership-related features in Access Analyzer, the TrustLogix service principal must have at least Workspace Admin or Account Admin permissions.
Create a Databricks Data Source In TrustLogix
Login to TrustLogix console and register a new Databricks data source from Menu → Data sources → Register new data source
Provide the data source name, & click on Register data source
Form will be refreshed with additional fields and script Download option
Run the notebooks in Databricks environment
Click on Download button to download the notebooks
Extract the downloaded Zip file which will contain multiple Python notebook files
Ensure that each of the notebook prerequisites are performed before that particular notebook is run by providing all the arguments like the Account ID, Service Principal Name, Access Token etc.
After successful run of each of the notebooks, required objects are created in Databricks
Capture the output of the notebook and also gather
Account ID
Workspace Deployment Name
SQL Warehouse Identifier
Service Principal Name
Client ID
Client Secret
How to register additional workspaces to an existing Databricks data source
The following are necessary to add additional workspaces to an existing Databricks data source
TrustLogix Service Principal Id
TrustLogix Client Id
TrustLogix DP Service Principal Id(Only for trustlet/hybrid tenant)
TrustLogix DP Client Id(Only for trustlet/hybrid tenant)
Workspace URL e.g adb-180424056746427.6.azuredatabricks.net
Here, 1, 2 belong to the TrustLogix Control Plane Service Principal and 3, 4 belong to the DataPlane Service Principal. You can get the above from the Results section of the initial tlx_databricks_grants notebook used while registering Databricks account in the TrustLogix console.
Step-1: Run following notebooks
tlx_databricks_grants_register_workspace.ipynb
tlx_databricks_grants_to_cp_service_principal_scheduler.ipynb(Only for trustlet/hybrid tenant)
tlx_databricks_workspace_admin_grant.ipynb
#1 and #3 can be run from any workspace. #2 need to be run from new workspace to be registered.
To execute the above notebooks in Databricks, it is suggested that the user or service principal must have a minimum of the following permissions:
Workspace Admin
Metastore Admin
Account Admin permission not required.
Step-2: Add the workspaces and warehouses in the TrustLogix console
Navigate back to TrustLogix data source form page
Update following for the new workspace
Workspace Deployment Name(s)
SQL Warehouse Identifier(s)
Click on save, system will validate the connection details and saves them
If all details are valid, Databricks datasource should be updated successfully
Step-3: Update secrets on AWS SSM (or Azure Key Vault) corresponding to the new workspace
Last updated
Was this helpful?

