Building Duplicacy from Source using Microsoft Azure Devops Pipelines

How-to
#1

I recently wanted to build duplicacy from the latest commit on the master repository. I am a .NET guy so I did not have any go tools installed on my PC, so I figured I would experiment with using Azure Devops pipelines to build it for me. I thought I would write this up to share my experience.

DISLOSURE - I have zero experience with go, and have only a cursory knowledge of the go commands run as part of this pipeline. This was only an experiment and I wouldn’t recommend using this for a “production” system unless more experienced devs can chime in. All that said, this build process resulted in an exe that ran just fine on my PC and I was able to run test backups and restores with no issue.

Azure Devops
Just some background, Azure devops is similar to github in that they host free git repositories and have CI/CD tools. It’s completely free, at least for this scope. Assuming I’m not doing something terrible with building the exe I don’t know about, this could be a really easy and quick way to build from source without needing to install all the go tools locally. I highly recommend checking it out, github is better for pure source control but azure devops has some awesome CI/CD features. The basic process goes like this:

  1. Clone duplicacy github repo into an azure devops repo
  2. Create a build pipeline specifying a series of go get, go mod, and go build commands.
  3. Run pipeline, download artifact which contains the exe

Steps to Create Exe

  1. Open azure devops by going here
  2. You’ll need to create a microsoft account if you don’t already have one, and will be prompted to supply some details if you’re never used the service before.
  3. Eventually you’ll get taken to a screen prompting you to create a new project. A project can have multiple repos and pipelines. Choose between public or private and enter a project name.
  4. The project will get created and from then on you’ll be able to access it via https://dev.azure.com/name-from-signup/project-name
  5. On the left hand side you’ll see some icons, click on the red icon labeled repos
  6. You’ll get taken to a screen that looks like the screenshot below. Click on the Import button under the Import a repository section.
    image
    image916×737 32.9 KB
  7. After clicking on import choose Git as the repository type and https://github.com/gilbertchen/duplicacy.git as the Clone URL. This will spin for a while as it grabs the code.
  8. Now that you have the code in your repo, you can create a pipeline to build the exe. On the left had side, click the blue rocketship icon labeled Pipelines.
  9. There will be a screen prompting you to create your first pipeline, hit the create pipeline button.
  10. On the Where is your code? screen, choose Azure Repos Git (YAML). Note you can choose to build directly from github here, but I think you need to be the github repository owner for that, I’m not sure. But in theory you could skip right to this step without ever cloning the repo into Azure.
  11. On the next screen select the repository you just cloned into azure devops.
  12. On the Configure your pipeline screen, choose either “Go” or “Starter pipeline”. It doesn’t really matter, because you’ll be replacing the contents of any template you choose. NOTE: I played around for quite a while with the template they give you out of the box. Only after googling and experimenting did I figure this out.
  13. Once you choose a pipeline you’ll get prompted to edit a YAML file. Replace the entire contents with this:
variables:
  GOOS: windows
  GOARCH: amd64

trigger: 
 - master

pool:
   vmImage: 'ubuntu-latest'

steps: 
- task: GoTool@0
  inputs:
    version: '1.16.0'
- task: Go@0
  inputs:
    command: 'mod'
    workingDirectory: '$(System.DefaultWorkingDirectory)'
    arguments: 'init duplicacy'
- task: Go@0
  inputs:
    command: 'get'
    workingDirectory: '$(System.DefaultWorkingDirectory)'
    arguments: '-d -u ./...'
- task: Go@0
  inputs:
    command: 'build'
    workingDirectory: '$(System.DefaultWorkingDirectory)'
    arguments: '-o duplicacy.exe duplicacy/duplicacy_main.go'
- task: CopyFiles@2
  inputs:
    TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts@1
  inputs:
     artifactName: drop
  1. This uses version 1.16.0 of go to run the build. It first runs go mod init duplicacy on the root folder of the repository. It then runs get -d -u ./…, and finally runs build -o duplicacy.exe duplicacy/duplicacy_main.go.
  2. The variables at the top, windows and amd64, define the platform and architecture. These will control the platform for which the exe is built. The list of values you can use for this can be found in the golang.org docs here
  3. Click the save and run button. This will commit the .yml file into the root of your repo and will be used for subsequent builds.
  4. The pipeline will get created and you’ll see a jobs icon at the bottom of your screen, shown in the screenshot below. You can click on the job to get more details, but they make it really hard to find the actual build artifact, so I suggest waiting here and refreshing your page until it’s complete.
    image
    image1610×452 17.8 KB
  5. After a few minutes the job will complete and the job icon will change to a green checkmark. Click on it. On the next screen you’ll see a black console log on the right hand side. At the bottom of that log there is a link that says “1 artifact produced” as seen in the screenshot below
    image
  6. You’ll be taken to an artifacts screen with a drop directory. The duplicacy.exe executable will be in the root of that folder. The drop directory also includes the source used during the build process.

All done! This pipeline will trigger any time a new commit occurs on the master branch in your azure devops repo. If there’s any branch or commit that hasn’t been integrated into the official duplicacy master branch yet, you could easily fetch that branch into your azure devops repo, commit, and boom you have another build ready to go.

If any of the go commands run during this build process are really terrible and shouldn’t be run please let me know! Again this was more just me experimenting and wanting to share in case people were interested. Enjoy!

2 Likes