ARMLinker 1.0.0 released
ARM What?
I've been having fun with ARM Templates the last couple of months. It's a wonderful way to keep your Azure Resource definitions in source control. Not to mention being able to parameterize deployment to different environments, and not least keeping your secrets neatly tucked away in a vault.
However, compiling a set of resources from multiple files currently requires you to put your templates online. I want to keep most of our customer products' templates private, and to do that one have to jump through hoops to copy the files over to a storage account and link to the given URLs. It kind of defeats the whole purpose for me.
So I went and created a small tool to be able to link templates locally.
How to use it
There's an installable project type for Visual Studio called "Azure Resource Group". When you create one you get a few files:
- Deploy-AzureResourceGroup.ps1
- azuredeploy.json
- azuredeploy.parameters.json
You can stuff all of the resources you require in the azuredeploy.json file, and finally deploy them using a wizard, or run the PowerShell script in a CD pipeline.
By installing ARMLinker you can start running the tool to link other JSON files into the main azuredeploy.json file.
install-module ARMLinker
Let's say we have a Logic App (what I've been doing).
To deploy it and its connections and other needed resources, we often want
a bounch of secret keys for different APIs and such.
Here's a trimmed down sample of a Logic App that runs a SQL command:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"Tags": {
"type": "object",
"defaultValue": {
"Customer": "My customer",
"Product": "Their Logic App",
"Environment": "Beta"
}
},
"SQL-Server": {
"defaultValue": "some.database.windows.net",
"type": "string"
},
"SQL-User": {
"defaultValue": "appuser",
"type": "string"
},
"SQL-Password": {
"defaultValue": "",
"type": "securestring"
},
"SQL-Database-Name": {
"defaultValue": "beta-database",
"type": "string"
}
},
"variables": {
"ConnectionName": "[replace(concat(parameters('Tags').Customer, '-', parameters('Tags').Product, '-SQLConnection-', parameters('Tags').Environment), ' ', '')]",
"LogicAppName": "[replace(concat(parameters('Tags').Customer, '-', parameters('Tags').Product, '-', parameters('Tags').Environment), ' ', '')]"
},
"resources": [
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"location": "westeurope",
"name": "[variables('ConnectionName')]",
"properties": {
"api": {
"id": "[concat(subscription().id,'/providers/Microsoft.Web/locations/westeurope/managedApis/sql')]"
},
"displayName": "sql_connection",
"parameterValues": {
"server": "[parameters('SQL-Server')]",
"database": "[parameters('SQL-Database-Name')]",
"authType": "windows",
"username": "[parameters('SQL-User')]",
"password": "[parameters('SQL-Password')]"
}
}
},
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[variables('LogicAppName')]",
"dependsOn": [ "[resourceId('Microsoft.Web/connections', variables('ConnectionName'))]" ],
"location": "westeurope",
"properties": {
"state": "Enabled",
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
},
"SQL-Server": {
"defaultValue": "",
"type": "string"
},
"SQL-Database-Name": {
"defaultValue": "",
"type": "string"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Day",
"interval": 1
},
"type": "Recurrence"
}
},
"actions": {
"Execute_a_SQL_query_(V2)": {
"runAfter": {},
"type": "ApiConnection",
"inputs": {
"body": {
"query": "select 'do something really useful' as task"
},
"host": {
"connection": {
"name": "@parameters('$connections')['sql']['connectionId']"
}
},
"method": "post",
"path": "/v2/datasets/@{encodeURIComponent(encodeURIComponent(parameters('SQL-Server')))},@{encodeURIComponent(encodeURIComponent(parameters('SQL-Database-Name')))}/query/sql"
}
}
},
"outputs": {}
},
"parameters": {
"$connections": {
"value": {
"sql": {
"connectionId": "[resourceId('Microsoft.Web/connections', variables('ConnectionName'))]",
"connectionName": "variables('ConnectionName')",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Web/locations/westeurope/managedApis/sql"
}
}
},
"SQL-Server": {
"value": "[parameters('SQL-Server')]"
},
"SQL-Database-Name": {
"value": "[parameters('SQL-Database-Name')]"
}
}
}
}
]
}
The parameters here are ARM template parameters. The most interesting one is the secret password for the database server. It's secret, so it's not supposed to live in our parameter file or source control. We've also got the ID of the connection, which will be the real ID in the actual deployed Logic App.
There's a fancy way to go about keeping the password in a key vault on Azure, and the Visual Studio Wizard is really helpful with putting it into a vault.
When we're done and ready for production, a parameter file may look like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"Tags": {
"value": {
"Customer": "My customer",
"Product": "Their Logic App",
"Environment": "Production"
}
},
"SQL-Database-Name": {
"value": "production-database"
},
"SQL-Password": {
"reference": {
"keyVault": {
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/Vault-Group/providers/Microsoft.KeyVault/vaults/OurKeyVault"
},
"secretName": "CustomerSQLPassword"
}
}
}
}
One of the beauties of using Logic Apps is that it have this nice GUI to work with in the portal. There's also an extension for Visual Studio to be able to edit them within Visual Studio.
However, the definition will look like this when viewed with the code editor. (I removed the bulk of it, but notice the parameters)
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Execute_a_SQL_query_(V2)": {
"inputs": {
"body": {
"query": "select 'do something really useful' as task"
},
"host": {
"..."
},
"..."
},
"runAfter": {},
"type": "ApiConnection"
}
},
"...",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
},
"..."
},
"triggers": {
"..."
}
},
"parameters": {
"$connections": {
"value": {
"sql": {
"connectionId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/CustomerResourceGroup/providers/Microsoft.Web/connections/MyCustomer-TheirProduct-SQLConnection-Prod",
"connectionName": "MyCustomer-TheirProduct-SQLConnection-Prod",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Web/locations/westeurope/managedApis/sql"
}
}
},
"SQL-Database-Name": {
"value": "production-database"
},
"SQL-Server": {
"value": "some.database.windows.net"
}
}
}
Notice that the parameters are all filled out. We can't copy this into our ARM template since it's all real Resource ID references.
There's another way to get only the definition. We can use the Az.LogicApp powershell module:
(get-azlogicapp -resourcegroupname CustomerResourceGroup -name mycustomer-theirproduct-prod).definition.ToString()
It will give us only the definition part of the template.
Both gives us a means to put only the definition of the logic app into a file in our local project.
Now we can go back to the ARM template and replace the definition with a simple link to the file.
Say we Set-Content the result of the statement above into a file called "logicapp.json". We can modify the ARM template as such:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"..."
},
"variables": {
"..."
},
"resources": [
{
"type": "Microsoft.Web/connections",
"..."
},
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[variables('LogicAppName')]",
"dependsOn": [ "[resourceId('Microsoft.Web/connections', variables('ConnectionName'))]" ],
"location": "westeurope",
"properties": {
"state": "Enabled",
"definition": {
"templateLink" {
"uri": ".\logicapp.json"
}
},
"parameters": {
"$connections": {
"value": {
"sql": {
"connectionId": "[resourceId('Microsoft.Web/connections', variables('ConnectionName'))]",
"connectionName": "variables('ConnectionName')",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Web/locations/westeurope/managedApis/sql"
}
}
},
"..."
}
}
}
]
}
By running ARMLinker we will have the same generated file as we started with,
but we can use the GUI for the logic app and easily fetch the new JSON for it.
Convert-TemplateLinks azuredeploy.json azuredeploy.linked.json
For now, I've actually turned those around and put the "linked" template in a file called azuredeploy.linked.json in order to generate the "conventional" azuredeploy.json file.
More options
When using the "copy content from the editor" method mentioned above, we have to make sure to copy only the definition object. Otherwise we'll bring the concrete parameters into the local file.
Do not despair!
There's another option that doesn't match the official schema for "templateLink". By adding a property called "jsonPath" we can point to an object deeper in the linked file. Say we copy the content from the online editor.
We can modify the linked template as such:
"definition": {
"templateLink" {
"uri": ".\logicapp.json",
"jsonPath": "definition"
}
},
It will now only merge the definition part from the logicapp.json file.
I only implemented dot separated paths for now, so exotic paths to arrays or paths with special characters won't work.
IE. resources[0]['very fancy'].thing won't work, but things.with.dots will work.
Plans and dreams
This is pretty much only a workaround while waiting for Microsoft to realise this is totally useful and obvious.
I originally intended it to be a Custom Tool for Visual Studio, but I could not for the life of me figure out how to enable Custom Tools in projects not of the C# or Visual Basic archetypes.
If anyone picks up on it, I'll happily discuss new features and even happierly receive meaningful pull requests.
Other than that, I believe it does the job properly. It can be used in CD pipelines. It should even work for any JSON, not necessarily ARM templates.
I would really appreciate your feedback, and hope you like it!
Now go commit and deploy something automagically while fetching coffee! 🤘😁🦄