Looker Tips and Tricks - Hub and Spoke Implementation

by Vyshnavi. CR

Looker Tips and Tricks - Hub and Spoke Implementation

In a large organization when we have a huge number of users, the requirements are difficult to handle. Looker helps us provide a generic data model and allow the users for appending or eliminating the code. The Hub and Spoke implementation help us achieve this.

In Looker, we can use a feature called project_import to implement a hub and spoke architecture between different Looker projects. This enables a team to define a central and generic repository of code that gets broadcasted to the other development teams. Those teams in turn can extend the files created in the hub project, changing or appending the code for their own use. The spoke repository can have any customizations on top of the Hub repository using the refinements or extends feature that looker has provided. The refinements feature helps us to adapt the existing view or explore and add additional columns/joins and present it. With the help of refinements, we can have multiple versions of the same view /explore. We can have different roles /groups created in looker for maintaining the security for the spoke users. The Development users will have access to change the code in Hub as well as the spoke repository. We can create a superuser privilege for the users who can modify the Lookml code in spoke and business users will have access to the dashboards.

Here is an example for Hub and Spoke implementation

Hub Repository Setup

• Setup the Hub repository along with GIT Configuration. Once you complete the GIT configuration you get the below screen
• The hub project should impose the most rigid level of access control, as we only want a limited number of developers to change the code. We will need to enforce the restriction in GitHub as well for these sets of developers to open/merge a pull request.
• The hub project should be organized well using the folder structure (IDE Folder structure). This will help the inherited code customization feasible in spoke
• Maintain good coding standards for all the view files. Provide descriptions for any custom dimensions/ measures
• Implement the logic in LookML rather than using Table Calculations which will help one to reuse in Spoke
• Create different model files according to the connections and use a common data group
  
  

Spoke Repository Setup

• Setup the Spoke repository along with GIT Configuration. We will connect the Spoke repository to the Hub repository in the below process. Once Looker authenticates the spoke repository you will get the below screen
• The spoke project can inherit the code from Hub
• Create a project manifest file in order to define the dependency. We have two options to define the dependency between the Hub and Spoke
• Looker will create a Manifest lock file to track the version of the remote project(s). We don't have to make any changes to the manifest lock file

• Local Dependency - The local_dependency parameter provides the details of a local GIT repository containing a LookML project that has files that you want to use in this project. The imported_projects has all the files of the project test that is imported.

• Remote Dependency - The remote_dependency parameter provides the details of an external Git repository containing a LookML project that has files that you want to use in this project. You can include multiple remote_dependency statements in the manifest file to import multiple projects.

• Once you add the remote dependency parameters you might get the manifest file error. The below error is because of the spoke repository unable to connect to Hub repository

• Add the looker generated configuration key to connect to Hub. Go to Project Settings -> Import Credentials -> Edit. Copy the Ssh key and add it to the Hub repository

• Once you add the Ssh key and validate we are all set to go. The Spoke repository is configured with the Hub repository code imported under 'imported_project'
• We have to include the view files with the include parameter after inheriting the hub code to perform customizations on the spoke
• In the Manifest File, the subparameter 'ref' that specifies a Git branch, a Git release tag, or the SHA of a commit in the Git repository of the remote dependency parameter should be updated as and when there are changes on the hub code to reflect the same changes on the spoke
• We can reuse the code with extends and refinements
  
  

Advantages of Hub and Spoke

• The Spoke can have any customizations on top of hub repository code using refinements/extends

• Multiple teams can use the generic hub code and work on the hub code according to their requirements in spoke

• Spoke can divide LookML Developers to keep consistency

Vyshnavi CR is a Lead Consultant in KPI Partners. Her professional focus is on data warehousing and business intelligence. She has implemented many BI solutions on Oracle BI Applications and the Google cloud platform. Has worked on ETL tools like Informatica, ODI, and reporting tools like OBIEE and Looker.