1. Tutorial overview
Last Updated: 2024-01-17
Background
Starburst Galaxy provides seamless access to multiple data sources through a single point of access. This includes data held inGoogle Cloud Storage (GCS). For this type of connection, a GCS service account JSON key is required to authenticate the connection between your Starburst Galaxy account and your GCS bucket.
Scope of tutorial
This tutorial will show you how to create a Google IAM service account and generate the associated JSON key. Additionally, you will learn how to grant the service account access to a GCS bucket to facilitate this connection.
Learning objectives
Once you've completed this tutorial, you will be able to:
- Configure a Google Cloud Storage service account JSON key.
- Allow a GCS service account to access a GCS bucket.
Prerequisites
- You need a Starburst Galaxy account to complete this tutorial. Please see Starburst Galaxy: Getting started for instructions on setting up a free account.
- This tutorial comes with a bring your own storage requirement. Before proceeding with this lesson, you must already have a Google Cloud Storage (GCS) bucket.
About Starburst tutorials
Starburst tutorials are designed to get you up and running quickly by providing bite-sized, hands-on educational resources. Each tutorial explores a single feature or topic through a series of guided, step-by-step instructions.
As you navigate through the tutorial you should follow along using your own Starburst Galaxy account. This will help consolidate the learning process by mixing theory and practice.
2. Create a Google IAM service account
Background
Google Cloud Identity and Access Management (IAM) services allow applications to gain secure access to Google Cloud Platform (GCP) resources. This typically happens when applications and virtual machines (VMs) running on GCP need to interact with other GCP services or external systems securely.
IAM Authentication
IAM service accounts are associated with a unique email address and set of credentials. The combination is used to authenticate the service account when making API calls or accessing resources. Service accounts are granted role-based permissions like regular Google accounts, allowing them to access specific resources or perform certain actions within a GCP project.
Starburst Galaxy authentication
The service account you create in this tutorial will authenticate access between Starburst Galaxy and your GCS bucket.
Step 1: Access IAM Service Accounts settings
You'll begin your work in the IAM Service Accounts section of the Google Cloud console.
- Sign in to your Google Cloud account.
- Using the Google Cloud search bar, enter IAM.
- In the search results, select IAM.
- Using the IAM & Admin navigation menu on the left, select Service Accounts.
Step 2: Select project and create new service account
In the context of GCP, a Project is used to group resources together for billing, access control, and management. Projects serve as containers for different GCP resources, including virtual machines, storage buckets, and databases and are one of the fundamental organizational units used in the GCP platform.
- Using the Project menu, select the Google Project that you would like to use for this tutorial.
- Click the + CREATE SERVICE ACCOUNT button.
Step 3: Configure service account
Now it's time to add some important details to your new service account, including a name and description.
- In the Service account name field, enter a meaningful name.
- In the Service account ID field, leave the default ID unchanged.
- In the Service account description field, enter a description.
- Click the DONE button.
Step 4: Create Service account key
In the context of GCP, a service account key is a file containing authentication credentials. It is used to authenticate the GCP service accounts when making API calls or accessing GCP resources.
The key is typically a JSON file containing access information, including the service account ID, private key, and other metadata. .
- Using the filter field, enter your Service account name.
- In the Actions column, use the ellipsis icon to expand the Actions menu.
- Select Manage keys.
- Expand the ADD KEY drop-down menu.
- Select Create new key.
- Ensure the radio button for JSON is selected.
- Click the CREATE button to download the JSON file to your workstation.
Step 5: Secure Service account key
Now that you've created your Service account key, it's important to keep it safe. We recommend copying it into a secure secrets manager or password vault.
Later, you will use this key when you create a GCS catalog in Starburst Galaxy.
- Locate your Service account key.
- Open the file using a text editor of your choice.
- Copy the contents of the JSON key file into a secure password manager.
Step 6: Store Service Account email
To grant access from your service account to your GCS bucket, you will need to supply the service account email.
You will copy your Service Account email and save it in a safe place for future reference.
- Select the Details tab for your service account.
- Copy the Email and paste it in a text editor for use later in this tutorial.
3. Grant access to GCS bucket for service account
Background
You're almost finished! For the final step, your new service account needs to be granted access to your GCS bucket. This is required for authentication between Starburst Galaxy and your GCS bucket.
Step 1: Locate GCS bucket
GCS buckets are located in the Cloud Storage section of the Google Cloud console. You're going to start by locating the GCS bucket that you want to connect.
- Using the search bar at the top of the console, search for Cloud Storage.
- Select Buckets from the list of results.
- In the filter field, enter the name of your GCS bucket.
- In the results below, select the Name of your bucket.
Step 2: Edit permissions
Now it's time to grant access from your service account to your GCS bucket. In particular, you need to grant the service account Storage Admin permissions. This will allow your service account to read and write to your GCS bucket.
- Select the PERMISSIONS tab.
- Click the GRANT ACCESS button.
- In the New principals field, enter the Service Account email that you stored earlier in this tutorial.
- Expand the Select a role drop-down menu.
- Hover over Cloud Storage and select Storage Admin.
- Click the SAVE button.
4. Tutorial wrap-up
Tutorial complete
Congratulations! You have reached the end of this tutorial, and the end of this stage of your journey.
That's all for this tutorial. You are now ready to securely connect your Starburst Galaxy account to Google Cloud Storage.
Continuous learning
At Starburst, we believe in continuous learning. This tutorial provides the foundation for further training available on this platform, and you can return to it as many times as you like. Future tutorials will make use of the concepts used here.
Next steps
Starburst has lots of other tutorials to help you get up and running quickly. Each one breaks down an individual problem and guides you to a solution using a step-by-step approach to learning.
Tutorials available
Visit the Tutorials section to view the full list of tutorials and keep moving forward on your journey!