Continuing from my last post, I am going to describe how a pipeline can access secrets from Azure Key Vault. I am going to explain the process through a demo. We would be covering the following points in this post –
- Create an ASP.NET Core web application and display some configuration values
- Create a Key Vault and add some secrets
- Create Key Vault access policy for pipeline service principal
- Update pipeline to use Key Vault task
- Run the pipeline and view the updates
I wont spend time explaining creating the web application. Although, I will provide screenshot of what I did without going into a lot of details. Additionally, I wont get into the details of creating the CI/CD pipelines. My web application will be hosted as an Azure app service, I wont get into that detail as well. The focus is to demonstrate how to access Key Vault from pipelines and we will stick to that. I am using visual studio 2019 and am working with .NET Core 3.1. So, lets get started.
Note: The source code is available in my Github repository in this path.
1. Create an ASP.NET Core web application
Step 1: To start with, I created an ASP.NET web application. Following are the steps –
Step 2: This step is important and interesting. What I am trying to do here allowing the appsettings.json file to store some configuration values like connectionString and testToken. Later in the flow, we will see how those values are fetched from the Key Vault and gets replaced. Therefore, go ahead and make these changes.
Step 3: Now, try running the application. As a result, you can see that the 2 values, connectionString and testToken, are displayed on the screen. We will see how to override these values in the DevOps pipeline with values retrieved from Key Vault.
2. Create a Key Vault and add secrets
Step 4: Next, let’s go into the Azure portal and create the Key Vault. I’ve navigated to the resource group that contains the App Service that Azure DevOps is using to deploy my project to. Add a new resource, find Azure Key Vault and click Create. Enter a name for the Key Vault and choose a region. Select the default pricing tier, which uses software-backed keys. But you can also choose the premium tier to store your keys in hardware security modules. I won’t add an access policy yet. I’ll allow access from all virtual cnetworks. Skip adding tags for now. After that, Click ‘Create’ to create the Key Vault.
Step 5: The next thing I will do is create some secrets. These will be the values we use to override the ones that developers are storing in the appsettings.json code in the Azure Repo. I’m acting as an administrator in operations who has permissions to create keys. Go to the Secrets tab and click the Generate/Import button. I am going to manually create this secret. Therefore, I’ll give it the same name as what’s in the app settings file, but this really doesn’t matter as long as I map it within the DevOps pipeline. It’s just a secret here. Note the values. I have replaced the ‘local’ keyword with ‘AzureKeyVault’ so that we can test the override.
Keep everything else intact. After that, click on Create.
Step 6: Similarly, create the other secret for testToken.
3. Create Key Vault access policy for pipeline service principal
Step 7: Now we need to give Azure DevOps permission to access these secrets. For that, we need the service principal that you would have created while creating a service connection in Azure DevOps. So, let me go back into the DevOps portal. I’m inside my project and I’ll go down to Project settings and go to Service connections. Here you can see the service connection that the deployment task uses to push the complied web app to Azure App Service. So let’s get the name of that service principal by clicking on Manage Service Principal. The Azure portal opens up. After that, copy the display name of the service principal.
Step 8: Let’s go back into Key Vault. Go down to Access policies in the left menu, and add a new access policy. I want to give specific permissions to that service principal. Click on Select Principal and paste the service principal name that you copied earlier. The particular value will show up as search result. Choose it and click Select.
Step 9: After that, go to Secret Permissions and select Get and List as that’s what we need. Once you add these, go back to the access policy screen. Click on Save. Don’t forget to save these changes, otherwise the new access policy won’t get added.
4. Update pipeline to use Key Vault task
We need to update 2 things in the pipeline. Firstly, add the Azure Key Vault task. Secondly, update the App Service Deploy task to override the values selected from the Key Vault. Now, it really depends on how you have configured your pipeline. In other words, whether you have added the App Service Deploy task in your CI pipeline or CD pipeline. Wherever it is, the process is the same. The only difference being, if modifying the CI pipeline, you would update your YAMP file. If modifying the CD pipeline, you will be adding the task and configuring it in the classic UI way.
In my case, the App Service Deploy task is present in my Release pipeline. Therefore, I will update my release pipeline. But, don’t worry, once you see what to do, you can easily do it in your build pipeline.
4.1 Add Azure Key Vault task
Step 10: In the release pipeline, click on x job, x task. In my case, I just have a Deploy Azure App Service task. We need to add a task before the Deploy App Service task. Search for ‘Key Vault’ and the result will display Azure key Vault task. Add it.
We will be connecting to Azure using the service connection we created earlier – that will show up the Azure Subscription dropdown. Once we choose that, the Key Vault list updates to include the ones we have access to. The reason we can see this is not because of the permissions we just added. It’s actually because the Key Vault was created in the resource group that the service connection is scoped to. When you created the service connection, you had to choose a resource group.
4.2 Update Azure Deploy App Service task
Step 11: At this point, all of the Key Vault secrets are available as environment variables within the pipeline. Next, let’s update the App Service deploy task so that we can override the values that are stored in the appsettings.json file.
Click on the Deploy Azure App Service task. Find the setting, Application and Configuration Settings. In the App setting textbox, add the code, -connectionString “$(connectionString)” -testToken “$(testToken)”
-connectionString denotes the parameter to be overwritten. $(connectionString) is the value from key vault.
Step 12: At the end, click on Save and provide a comment.
5. Run the pipeline and view the updates
Step 13: I want to do the whole deal. Which means, I will do a code change, push it to DevOps and run both my CI and CD pipeline. Therefore, I changed my welcome version to 2.0 in my Index.cshtml file and pushed it to Azure repos. This triggered both the build and release pipeline.
Step 14: After the successful completion of both the pipelines, I went to my Azure App Service in Azure Portal and clicked on the URL to launch the webpage. Voila!! The values are updated from the Azure Key Vault.
One last thing I want you all to notice. In the Azure Portal, go to your web app and click on Configuration under Settings. Here those two variables appear because we pushed them as part of the deployment task. And these values were used to override the ones in the appsettings.json file.
I hope this was helpful. Do let me know in case of any questions. Until next time, goodbye and stay safe.