Serverless CI/CD for .NET Core for AWS
Deploying resources to the cloud can be a challenging problem to solve. There are so many ways that one can try to do this, and then variations depending on the resource or even language the developer is using. I am going to show you how we use a combination of the AWS CDK, nuke.build, and github to create a simple AWS CodePipeline that is the start of a CI/CD process you can use for all of your .net AWS projects or even projects outside of AWS.
First, we need to get the AWS CDK installed if we have not already. This can be done by running npm install -g aws-cdk from the command line. This will install the CDK globally on your machine, so it is usable for future endeavors. Next, we need to add a CDK .net project to the Visual Studio solution that we want to deploy to the cloud via CI/CD.
(The following is how to add a CDK project to an existing solution, you can follow AWS instructions if starting from scratch, I prefer to do it this way.)
Navigate to the root directory for the solution you are working with and create a new folder called whatever you want your CDK project to be named (I will use NukeCdkCICDExampleCdk) navigate to ‘NukeCdkCICDExampleCdk’ and run cdk init app — language csharp. This will generate some files plus another directory called src. Copy the files to the root of your solution, navigate inside src and copy the ‘NukeCdkCICDExampleCdk’ project to wherever you keep your other projects for your solution. Open the cdk.json file and modify the attribute app so that the path matches the location of the CDK project (for me app becomes dotnet run -p src/NukeCdkCICDExampleCdk/NukeCdkCICDExampleCdk.csproj). The rest of the ‘NukeCdkCICDExampleCdk’ folder you created can now be deleted.
Now inside your IDE you will need to add the project to the solution (in Rider this can be done by right clicking the solution and clicking add existing project).
Go to the Program.cs file and uncomment the following lines:
Env = new Amazon.CDK.Environment
{
Account = System.Environment.GetEnvironmentVariable(“CDK_DEFAULT_ACCOUNT”),
Region = System.Environment.GetEnvironmentVariable(“CDK_DEFAULT_REGION”),
}
Make sure you have created a profile in your AWS credentials file for wherever you want this to deploy.
Navigate via command line to the root of your solution you wish to deploy, run the following commands (assumes linux terminal, windows will differ):
export CDK_DEFAULT_ACCOUNT=[ACCOUNT_NUMBER]
export CDK_DEFAULT_REGION=[REGION]
export AWS_PROFILE=[PROFILENAME]
Now we need to bootstrap the AWS account we want to deploy to by running cdk bootstrap.
After that we can test our work so far by running cdk deploy. This should give a green checkmark and return the arn of the stack you just created if successful:
Next step is to create a class to represent our CodePipeline/CodeBuild that will do the actual building of our solution (ie NukeCdkCICDExampleCodePipeline). NukeCdkCICDExampleCodePipeline needs to implement Amazon.Cdk.Construct.
Inside of the constructor we will place the following:
An s3 bucket for the CodeBuild to use (you will need to install nuget package for cdk s3)
Bucket codeBuildBucket = new Bucket(this, “NukeCdkCICDExample Code Pipeline Bucket”);
A role for the CodeBuild to use with the following AWS managed policies:
var codeBuildRole = new Role(this, “NukeCdkCICDExample Codebuild Role”, new RoleProps{AssumedBy = new ServicePrincipal(“codebuild.amazonaws.com”)});
codeBuildRole.AddManagedPolicy(ManagedPolicy.FromManagedPolicyArn(this,”S3 full access managed policy codebuild”,”arn:aws:iam::aws:policy/AmazonS3FullAccess”));
codeBuildRole.AddManagedPolicy(ManagedPolicy.FromManagedPolicyArn(this,”SecretsManager full access managed policy codebuild”,”arn:aws:iam::aws:policy/SecretsManagerReadWrite”));
codeBuildRole.AddManagedPolicy(ManagedPolicy.FromManagedPolicyArn(this,”CodePipeline full access managed policy codebuild”,”arn:aws:iam::aws:policy/AWSCodePipeline_FullAccess”));
codeBuildRole.AddManagedPolicy(ManagedPolicy.FromManagedPolicyArn(this,”CodeBuild full access managed policy codebuild”,”arn:aws:iam::aws:policy/AWSCodeBuildAdminAccess”));
codeBuildRole.AddManagedPolicy(ManagedPolicy.FromManagedPolicyArn(this, “IAM full access managed policy codebuild”, “arn:aws:iam::aws:policy/IAMFullAccess”));
o S3 full access (to manage the bucket you created above)
o SecretsManager full access to get secrets stored in AWS to allow secure storage of github tokens
o CodePipeline full access needed to modify the pipeline you are creating
o CodeBuild full access needed to modify build you are creating
o IAM full access to manage this role and others you may create via CDK
o *NOTE BEST PRACTICE TO CREATE ROLE WITH MINIMAL ACCESS NEEDED THIS IS JUST FOR DEMO PURPOSES!
A PipelineProject aka CodeBuild (will need nuget for CodeBuild)
PipelineProject project = new PipelineProject(this, “NukeCdkCICDExample Pipeline Project”, new PipelineProjectProps
{
ProjectName = “NukeCdkCICDExample-CodeBuild”,
Role = codeBuildRole,
BuildSpec = BuildSpec.FromObject(new Dictionary<string, object>
{
[“version”] = “0.2”,
[“phases”] = new Dictionary<string, object>
{
[“install”] = new Dictionary<string, object>
{
[“runtime-versions”] = new Dictionary<string, object>()
{
{“dotnet”, “5.0”}
},
[“commands”] = new []
{
“export PATH=\”$PATH:/root/.dotnet/tools\””,
“dotnet tool install — global Nuke.GlobalTool”,
“npm install -g aws-cdk”
}
},
[“build”] = new Dictionary<string, object>
{
[“commands”] = $”nuke DeployNukeCdkCICDExampleCdkStack “ +
$” — awsregion {createCodePipelineRequest.AwsRegion} “ +
$” — apigitbranchname {createCodePipelineRequest.GitBranchName} “ +
$” — awsaccount {createCodePipelineRequest.AwsAccount} “
}
}
}),
Environment = new BuildEnvironment
{
EnvironmentVariables = new Dictionary<string, IBuildEnvironmentVariable>{
{“DOTNET_ROOT”, new BuildEnvironmentVariable
{
Value = “/root/.dotnet”
}},
{“CDK_DEFAULT_ACCOUNT”, new BuildEnvironmentVariable
{
Value = createCodePipelineRequest.AwsAccount
}},
{“CDK_DEFAULT_REGION”, new BuildEnvironmentVariable
{
Value = createCodePipelineRequest.AwsRegion
}}
},
BuildImage = LinuxBuildImage.STANDARD_5_0
}
});
o We pass the role created above as the role to this project
o BuildSpec is created programmatically here and you can all the things needed to configure the CodeBuild, note here we install the CDK and nuke, specify our .net version, specify our build command that calls nuke and deploys the CDK, and finally we set environment vars for the build. You can add any other configuration just as you would to any buildspec.
A source output (you will need nuget for CodePipeline)
var sourceOutput = new Artifact_();
An OAuth token for github (I store it in secrets manager to be secure also needs to be stored as plain text for example code to work)
var githubToken = SecretValue.SecretsManager(“Github_Token”);
And finally we need to add the CodePipeline itself:
Pipeline pipeline = new Pipeline(this, “NukeCdkCICDExample Pipeline”, new PipelineProps
{
ArtifactBucket = codeBuildBucket,
Stages = new IStageProps[]{new StageOptions
{
StageName = “Source”,
Actions = new IAction[]{new GitHubSourceAction(new GitHubSourceActionProps
{
ActionName = “Github_Source”,
OauthToken = githubToken,
Owner = “liberty-lake-cloud”,
Repo = “Nuke_Cdk_CICD_Example”,
Branch = createCodePipelineRequest.GitBranchName,
Output = sourceOutput,
Trigger = GitHubTrigger.WEBHOOK
}) }
},
new StageOptions
{
StageName = “Build”,
Actions = new IAction[]{ new CodeBuildAction(new CodeBuildActionProps
{
ActionName = “NukeCdkCICDExample_Build”,
Project = project,
Input = sourceOutput,
})}
}
}
});
o We use the bucket created above as the ArtifactBucket
o The source stage is configured as a GitHubSourceAction (nuget for CDK CodePipeline Actions is needed). The output will be the sourceOutput configured above, and the OauthToken comes from above as well. Replace the rest of the GitHubSourceActionProps as needed for your repository.
o The build stage will use the PipelineProject from above as the project and the input will be the sourceOutput from above.
Find the main stack file itself, it will end in Stack.cs. Add CfnParameters for AwsRegion, AwsAccount, and GitBranchName these will all be required parameters for the stack we are creating as well as for our CodePipeline. New up an instance of the pipeline you created above and pass in the necessary CfnParameters, I like to create a request object to keep my code cleaner.
public class NukeCdkCICDExampleCdkStack : Stack
{
internal NukeCdkCICDExampleCdkStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
{
var awsRegion = new CfnParameter(this, “awsRegion”,
new CfnParameterProps
{
Type = “String”,
Description = “Aws region everything is running in”
});
var gitBranchName = new CfnParameter(this, “gitBranchName”,
new CfnParameterProps
{
Type = “String”,
Description = “Git Branch Name”
});
var awsAccount = new CfnParameter(this, “awsAccount”,
new CfnParameterProps
{
Type = “String”,
Description = “Aws Account”
});
var createCodePipelineRequest = new CreateCodePipelineRequest()
{
AwsRegion = awsRegion.ValueAsString,
GitBranchName = gitBranchName.ValueAsString,
AwsAccount = awsAccount.ValueAsString,
};
var nukeCdkCICDExampleCdkCodePipeline = new NukeCdkCICDExampleCodePipeline(this,
“NukeCdkCICDExampleCdk CodePipeline”, createCodePipelineRequest);
}
}
The final component to our automated build project will be to add the nuke.build project.
From the command line you will need to run dotnet tool install Nuke.GlobalTool –global. Afterwards navigate to the root of your solution and run nuke :setup:. You can accept the defaults, make note if your projects are in a different directory then it specifies you will want to fix this. In Build.cs remove GitRepository and GitVersion as these are not passed by CodePipeline (it may be possible to use the v2 implementation of Github/CodePipeline integration to fix this, but I have not tried as of yet). Add parameters AwsRegion, GitBranchName, and AwsAccount to allow for deployment to any account and from any branch name:
[Parameter]
string AwsRegion { get; }
[Parameter]
string GitBranchName { get; }
[Parameter]
string AwsAccount { get; }
Add a target to build your CDK stack:
Target DeployNukeCdkCICDExampleStack => _ => _
.DependsOn(Compile)
.Executes(() =>
{
var cdkPath = ToolPathResolver.GetPathExecutable(“cdk”);
var buildCommand = $”deploy — require-approval never “ +
$” — parameters awsRegion={AwsRegion} “ +
$” — parameters gitBranchName={GitBranchName} “ +
$” — parameters awsAccount={AwsAccount}”;
ProcessTasks.StartProcess(toolPath: cdkPath, arguments: buildCommand, workingDirectory: RootDirectory)
.AssertWaitForExit(); ;
});
From the command line make sure you have the following environment vars set (this is linux windows will differ):
export CDK_DEFAULT_ACCOUNT=[ACCOUNT_NUMBER]
export CDK_DEFAULT_REGION=[REGION]
export AWS_PROFILE=[PROFILENAME]
Run the following command:
nuke DeployNukeCdkCICDExampleCdkStack \
— awsregion us-west-2 \
— gitbranchname main \
— awsaccount [ACCOUNT_NUMBER]
You should see NUKE printed on the screen followed by several different steps, the last step will be DeployNukeCdkCICDExampleCdkStack which is actually publishing your stack to AWS. You will see a printout of the stack publishing steps followed lastly by a printout of the build status (succeeded or not).
Congratulations! You now have a working serverless CI/CD build.
Changes you make to the branch you specified when you published this, will now automatically be pushed to AWS every time you merge. Add an API gateway? It will build and deploy upon merge. Need a lambda or two? Just define them in the CDK and merge to almost instantly add functionality. This build is highly customizable allowing for adding more stages for testing, deployment, or even entirely different builds (a build for a UI project for instance). Notifications for things such as build status can also be added as well as many other useful features.
Incorporating nuke lets developers easily interact with the .net codebase and allows for almost unlimited build options that can be implemented via code instead of script. Nuke further allows for the testing of your build locally at ant step you want, and also makes your build highly portable. Combined with CodePipeline and CodeBuild this portability makes this solution great for those who have a need to deploy to multiple AWS accounts for compliance or other reasons. Another great use for this solution is for temporary cloud solutions. With one command you can deploy a solution to the cloud, and another removes it and any ongoing cost that may be incurred. Need a dev, test and prod environment? No problem just deploy this as many times with the appropriate configuration and you can have as many environments as necessary, and then revoke the stack when the environment is no longer needed.
I hope this article has been helpful and informative. Look for more articles coming that will build upon this small project!
Github repo containing working example:
https://github.com/liberty-lake-cloud/Nuke_Cdk_CICD_Example
Resources:
