AppVeyor is a great continuous integration tool for Visual Studio projects. You can use it for free to build and deploy your open-source code. AppVeyor has a great integration with GitHub which allows you to create seamless CI scenarios.
Today we will configure AppVeyor to build an open-source project which uses CUDA Toolkit as well as CuDNN framework.
Let's make a quick recap of how AppVeyor build environments work. When you start a new build, a new pre-configured virtual machine is deployed to the cloud. This machine has a lot of compilers, runtimes and toolkits installed. You can run some pre-build scripts and install custom tools that you need to build your project. Then an ordinary MSBuild is called to build the solution that you specify in project settings. After the solution is built and tests are run, virtual machine gets destroyed. As you can see, there is no persistent build environment that we can configure and run all our builds at. Instead, we need to setup the environment for each new build.
To build our project we need two things to be present in the environment: CUDA Toolkit and CuDNN framework. The easiest way to install CUDA Toolkit would be to download the whole distribution from NVidia's website and install the components that we need to build the project. The problem is that the whole distribution occupies 1.2G of disk space and it's highly inefficient to download it every time we want to build a project.
Customized CUDA Toolkit distribution
The best way to circumvent the size problem is to create a custom CUDA Toolkit distribution that contains only the components we need. Here's how we are going to do this:
- Download current version of CUDA Toolkit from https://developer.nvidia.com/cuda-downloads. Be sure to choose the
exe (local)version to get the full distro.
- Extract the distribution using 7Zip or another archive tool.
- Create a new folder to hold your custom distro and copy the following files and folders from the original distro:
compiler\ cublas\ cublas_dev\ CUDADevelopment\ cudart\ CUDARuntimes\ CUDAVisualStudioIntegration\ curand\ curand_dev\ NVI2\ cuda.zip EULA.txt license.txt ListDevices.txt NVMUP.cfg Setup.cfg setup.exe
- Remove NSight installation files
NVIDIA NVTX Installer.x86_64.*.msifrom
- Remove all xml nodes that have
NVTXin them from
- Pack the contents of your custom distribution folder with your favorite archiving tool and upload it to some location from which you will be able to download the file. For my project I uploaded the distro to a Dropbox folder, created a sharing link and modified the link so that "dl" parameter at the end is "1":
https://www.dropbox.com/s/[some stuff]/cuda.zip?dl=1. Now you can use this link for direct download.
The second component that you may require is CuDNN framework. Unfortunately, you can't download it with a direct link because NVidia wants you to register a free developer account. Since the download is actually free, you can upload the archive to your private Dropbox account providing you register at NVidia and download the archive yourself. Let's assume you've uploaded CuDNN somewhere and now have a private download link.
Configuring AppVeyor project
Now we will write a simple batch file to download and install CUDA Toolkit and CuDNN to AppVeyor environment. Go to your AppVeyor project settings, open the
Environment pane and choose
cmd in the
Install script option. Paste the following script:
@echo off echo Downloading CUDA toolkit 9 appveyor DownloadFile [link to custom CUDA distribution] 7z x cuda9.zip -ocuda cd cuda echo Installing CUDA toolkit 9 setup.exe -s compiler_9.0 ^ cublas_9.0 ^ cublas_dev_9.0 ^ cudart_9.0 ^ curand_9.0 ^ curand_dev_9.0 ^ visual_studio_integration_9.0 if NOT EXIST "%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin\cudart64_90.dll" ( echo "Failed to install CUDA" exit /B 1 ) cd .. echo Downloading cuDNN appveyor DownloadFile [link to cuDNN zip file] 7z x cudnn7.zip -ocudnn copy cudnn\cuda\bin\*.* "%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin" copy cudnn\cuda\lib\x64\*.* "%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0\lib\x64" copy cudnn\cuda\include\*.* "%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0\include" set PATH=%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin;%PATH% set CUDA_PATH=%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0 set CUDA_PATH_V9_0=%ProgramFiles%\NVIDIA GPU Computing Toolkit\CUDA\v9.0 nvcc -V cd "%APPVEYOR_BUILD_FOLDER%"
Substitute the URLs for your distributions and click
Save. Now our AppVeyor environment is ready to build any project that uses CUDA Toolkit and CuDNN.