Automatically Detect and Prevent Secrets Leaked into Code within Azure DevOps
Table of Contents
Objective
Security vulnerabilities introduced by hardcoded secrets, passwords, or tokens in your source code can significantly compromise the safety of your application and/or infrastructure. A single API key or database connection string committed to a repository can be a gateway for attackers. But how do you ensure sensitive information never enters your codebase? The answer lies in automation.
In this blog post, you’ll learn how to automate secret detection in Azure DevOps using Gitleaks. We’ll set up a pipeline that listens for pull requests (PRs), fetches the latest code changes, and scans them for leaks — failing the PR if any secrets are found.
Why Do We Need Secret Detection
According to GitGuardian’s 2024 report, “State of Secrets Sprawl”, nearly 13 million new secrets were discovered in public GitHub commits, marking a 28% increase. Out of the 1.1 billion commits scanned, more than 1 in 10 commit authors were found to have leaked some form of a secret, with approximately 7 out of every 1,000 commits exposing at least one.
Similarly, Sophos reported that in 2023, compromised credentials became the leading root cause of cyberattacks for the first time. During the first half of the year, compromised credentials accounted for 50% of root causes, compared to 23% from vulnerability exploitation.
What is Gitleaks
Gitleaks is an open-source tool that scans your repositories for secrets like passwords, API keys, and tokens. It provides:
- Pattern-based detection to identify sensitive data formats like API keys, passwords, private SSH keys, and more.
- Customizable configurations to define your organization-specific rules for detecting secrets.
- Fast scanning of both your source code and Git history, enabling you to catch leaks in real-time.
When used in Azure DevOps pipelines, Gitleaks can help enforce strict security policies by running automated scans during pull requests and code merges.
How to Set Up Gitleaks for PR Scanning in Azure DevOps
Set Up Pipeline
1. Go to your Azure DevOps organization and go to Pipelines > New Pipeline
2. In the “Where is your code” pane, select Azure Repos Git
- Select your Repository; I selected my DevOps repository in my example.
4. Next, in the “Configure” screen, click Starter Pipeline
5. Finally, replace the YAML starter code with the YAML code below and save it.
Important: If you have a branch policy not allowing commits directly to
mainyou may have to do a pull request from a feature branch. You can just create theyamlfile, perform a pull-request, and then create a new pipeline but select Existing Azure Pipelines YAML file
pr:
autoCancel: true
branches:
include:
- main
trigger: none
pool:
vmImage: 'ubuntu-latest'
steps:
- bash: |
mkdir -p $(Build.ArtifactStagingDirectory)/gitleaks-report
displayName: 'Create Report Directory'
- bash: |
wget https://github.com/gitleaks/gitleaks/releases/download/v8.18.1/gitleaks_8.18.1_linux_x64.tar.gz
tar -xzf gitleaks_8.18.1_linux_x64.tar.gz
chmod +x gitleaks
sudo mv gitleaks /usr/local/bin/
displayName: 'Install Gitleaks'
- bash: |
if ! command -v gitleaks &> /dev/null; then
echo "##vso[task.logissue type=error]Gitleaks is not installed or not in PATH"
exit 1
fi
echo "##[section]Gitleaks installation verified"
gitleaks version
displayName: 'Verify Gitleaks Installation'
- bash: |
# Check for gitleaks.toml and run appropriate command
if [ -f "gitleaks.toml" ]; then
echo "##[section]Found custom gitleaks.toml in repository. Using custom configuration."
config_message="Using custom configuration from repository's gitleaks.toml"
gitleaks_output=$(gitleaks detect -v --config gitleaks.toml 2>&1) || exit_code=$?
else
echo "##[warning]No gitleaks.toml found in repository. Using default Gitleaks configuration."
config_message="Using default Gitleaks configuration (no gitleaks.toml found)"
gitleaks_output=$(gitleaks detect -v 2>&1) || exit_code=$?
fi
# Save the output to a file
echo "$gitleaks_output" > $(Build.ArtifactStagingDirectory)/gitleaks-report/scan-output.txt
# Create a readable summary
echo "Gitleaks Scan Summary" > $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "===================" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "Scan completed at: $(date)" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "Configuration: $config_message" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
if [ "$exit_code" -eq "1" ]; then
findings_count=$(echo "$gitleaks_output" | grep -c "Finding:")
echo "🚨 SECURITY ALERT: Found $findings_count potential secret(s) in your code! 🚨" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "Critical Security Issues Found:" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "- Potential secrets or credentials were detected in your changes" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "- These could pose a significant security risk if exposed" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "Required Actions:" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "1. Review the scan-output.txt file in pipeline artifacts" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "2. Remove or revoke any exposed secrets immediately" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "3. Replace secrets with secure environment variables" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "4. Consider using Azure Key Vault for sensitive information" >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "##vso[task.logissue type=error]🚨 SECURITY ALERT: $findings_count potential secret(s) detected in your code! ($config_message)"
echo "##vso[task.logissue type=error]Review the 'Gitleaks Security Report' artifact for details and take immediate action."
echo "##vso[task.complete result=Failed;]Security scan failed - secrets detected"
exit 1
else
echo "✅ SUCCESS: No secrets were detected in the scan." >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "Code review passed security checks." >> $(Build.ArtifactStagingDirectory)/gitleaks-report/summary.txt
echo "##[section]✅ No secrets detected - scan passed successfully ($config_message)"
echo "##vso[task.complete result=Succeeded;]Security scan passed - no secrets detected"
fi
displayName: 'Run Gitleaks Scan'
- task: PublishBuildArtifacts@1
inputs:
pathToPublish: '$(Build.ArtifactStagingDirectory)/gitleaks-report'
artifactName: 'Gitleaks Security Report'
condition: succeededOrFailed()
displayName: 'Publish Scan Results'
Require the Pipeline to Pass to Approve the Pull Request
Next, we want to ensure that if the pipeline fails (i.e., secrets are detected within the pull request), it cannot be approved until resolved.
1. In Azure DevOps, go to Repos.
2. In the repository drop-down in the top breadcrumb bar, click Manage Repositories.
3. In the All Repositories pane, click the repository that you set the pipeline up in.
4. In the Policies pane, scroll to the bottom and click on the main branch
Important: If your main branch is named something other than
main, you must ensure theyamlpipeline is looking at the correct branch. By default, its looking at themainbranch.
5. Click the + button under Build Validation
6. Under Build pipeline, select the pipeline we created earlier. Give it a good name, and then click save.
Custom Configuration for Gitleaks (Optional)
By default, Gitleaks uses the configuration file located here. However, your organization might not care about most of the secrets listed in the default configuration, by placing a file named gitleaks.toml at the root of your repository, Gitleaks will automatically load that configuration file. The pipeline already contains logic to search for this file when it clones the repo, and if its present, use the configuration file. If it’s not there, it will load the default configuration.
Test the Pull Request Secret Scanning
Failed Run
Now that we’ve set up the Azure DevOps pipeline and configured the branch policy to require the pipelines to run successfully, let’s test it out and see it in action.
1. Back in the Repository, in a branch called dev I created a new PowerShell file called connect-mggraph.ps1 and hard-coded two secrets.
2. Next, I will create a new Pull Request to add this file from my dev branch to my main branch as shown below.
3. The pipeline is automatically triggered Once I create the Pull Request.
4. After a few seconds, we can see that the pipeline failed because it found two secrets. It also lets me know that it used the default configuration because no configuration file is present in my repository.
5. If I click on the pipeline name, DevOps - Secret Scanning I can view the pipeline details. Under related I can see that one artifact was attached to this pipeline. If I click it I can view further details on what the scan found.
6. Clicking on the artifact will show me the names of the two files.
7. The scan-output.txt file shows me the following:
○
│╲
│ ○
○ ░
░ gitleaks
Finding: $MSGraph = "[1;3;mgfZ8Q~zH8uZNI~D4z0dsfdsfZUvkZbGauc4Z[0m"
Secret: [1;3;mgfZ8Q~zH8uZNI~D4z0dsfdsfZUvkZbGauc4Z[0m
RuleID: azure-ad-client-secret
Entropy: 4.462815
File: connect-mggraph.ps1
Line: 1
Commit: 815c8079b4c372fe4674357c3d6c2e9882082fd3
Author: Brad Wyatt
Email: [email protected]
Date: 2024-12-08T01:36:48Z
Fingerprint: 815c8079b4c372fe4674357c3d6c2e9882082fd3:connect-mggraph.ps1:azure-ad-client-secret:1
Finding: $OpenAI_APIKey = "[1;3;msk-1234567890abcdef1234567890abcdef[0m"
Secret: [1;3;msk-1234567890abcdef1234567890abcdef[0m
RuleID: generic-api-key
Entropy: 4.214997
File: connect-mggraph.ps1
Line: 5
Commit: 815c8079b4c372fe4674357c3d6c2e9882082fd3
Author: Brad Wyatt
Email: [email protected]
Date: 2024-12-08T01:36:48Z
Fingerprint: 815c8079b4c372fe4674357c3d6c2e9882082fd3:connect-mggraph.ps1:generic-api-key:5
[90m1:37AM[0m [32mINF[0m 1 commits scanned.
[90m1:37AM[0m [32mINF[0m scan completed in 8.54ms
[90m1:37AM[0m [31mWRN[0m leaks found: 2
and the summary.txt file contains the following:
Gitleaks Scan Summary
===================
Scan completed at: Sun Dec 8 01:37:01 UTC 2024
Configuration: Using default Gitleaks configuration (no gitleaks.toml found)
🚨 SECURITY ALERT: Found 2 potential secret(s) in your code! 🚨
Critical Security Issues Found:
- Potential secrets or credentials were detected in your changes
- These could pose a significant security risk if exposed
Required Actions:
1. Review the scan-output.txt file in pipeline artifacts
2. Remove or revoke any exposed secrets immediately
3. Replace secrets with secure environment variables
4. Consider using Azure Key Vault for sensitive information7. Lastly, viewing the details of the pipeline, we can see that since secrets were detected from the scan, the job is in a failed status, which means the entire pipeline is in a failed status.
Successful Run
1. Now, I will go back and edit the file in my dev branch that contains the hard-coded secrets and remove them.
2. Next, I will go back to Pull Request and see that it re-scanned and found no issues and it allows us to approve and complete the pull request.
3. Reviewing the artifacts for the pipeline shows new messages. The summary.txt now shows the following message:
Gitleaks Scan Summary
===================
Scan completed at: Sun Dec 8 01:49:25 UTC 2024
Configuration: Using default Gitleaks configuration (no gitleaks.toml found)
✅ SUCCESS: No secrets were detected in the scan.
Code review passed security checks.
and scan-output.txt shows the following:
○
│╲
│ ○
○ ░
░ gitleaks
[90m1:49AM[0m [32mINF[0m 1 commits scanned.
[90m1:49AM[0m [32mINF[0m scan completed in 14.2ms
[90m1:49AM[0m [32mINF[0m no leaks found
4. Finally, if we wanted to review the details of the pipeline for the successful run we would see the following:
My name is Bradley Wyatt; I am a 5x Microsoft Most Valuable Professional (MVP) in Microsoft Azure and Microsoft 365. I have given talks at many different conferences, user groups, and companies throughout the United States, ranging from PowerShell to DevOps Security best practices, and I am the 2022 North American Outstanding Contribution to the Microsoft Community winner.