Chapter 2: If you build it, they will come
As a new contributor to this project you really know nothing about how it organizes its code and what it needs to be in a a "valid" state. Luckily, however, you know about the MATLAB build tool! In fact, once you take a look around you notice that there is a file called buildfile.m in the root of the project, and just as if you saw a Makefile, a Rakefile, a pom.xml, or any of these standard markers for build tools in other languages, you already know how to interact with it. Super pumped.
One thing you can do to simply start interacting with this project is see what build tasks you can perform on this project by typing
buildtool -tasks
clean - Delete task outputs and traces
ctf - Create a deployable archive for MATLAB Production Server
ctfIntegTest - Run integration tests against CTF archive.
deploy - Produce and test a ctf archive to deploy to a MATLAB Production Server
doc - Generate doc pages from scripts and examples
lint - Identify code issues
mex - Compile all mex files
mex_convec - Build MEX file
mex_yprime - Build MEX file
moveSrc - Move static source to shipping toolbox folder
pcode - Create P-code files
release - Produce a fully qualified toolbox for release
setupCompiler - Setup MEX compiler
tbxIntegTest - Run integration tests against packaged toolbox
test - Run the unit tests.
toolbox - Create an mltbx toolbox package
workshop - Generate html from the workshop mlx files
This shows a variety of tasks. Some of these may look interesting, but still you don't really know what the basic "vanilla" development workflow is. A good rule of thumb here, just like one might invoke the default make target when working with Makefiles, is to run the default build target for the MATLAB build. Let's do that and see what the build does. So go ahead and just invoke buildtool with no inputs to go through the "default" build. Before you do this, make a mental note to observe the current folder browser to see what is happening.
>> buildtool
** Starting lint
Analysis Summary:
Total Files: 24
Errors: 0 (Threshold: 0)
Warnings: 2 (Threshold: Inf)
Results:
SARIF: results/code-issues.sarif
** Finished lint
** Starting setupCompiler
MEX configured to use 'Xcode with Clang' for C language compilation.
To choose a different language, select one from the following:
mex -setup C++
mex -setup FORTRAN
Compiler is detected and properly setup.
** Finished setupCompiler
** Starting mex_convec
Building with 'Xcode with Clang'.
MEX completed successfully.
** Finished mex_convec
** Starting mex_yprime
Building with 'Xcode Clang++'.
MEX completed successfully.
** Finished mex_yprime
** Starting mex
** Finished mex
** Starting test
......
Generating test report. Please wait.
Preparing content for the test report.
Adding content to the test report.
Writing test report to file.
Test report has been saved to:
/Users/acampbel/scratch/msd4/results/test-results.html
MATLAB code coverage report has been saved to:
/Users/acampbel/scratch/msd4/results/coverage/index.html
Test Summary:
Total Tests: 6
Passed: 6
Failed: 0
Incomplete: 0
Duration: 1.2027 seconds testing time.
Test Results:
HTML: results/test-results.html
Code Coverage:
HTML: results/coverage/index.html
** Finished test
** Starting doc
Exporting code/examples/BuildPlanDetails.mlx to html
Exporting code/gettingStarted.mlx to html
** Finished doc
** Starting pcode
Extracting help text for: pcode/springMassDamperDesign.m
Produced help file in: toolbox/springMassDamperDesign.m
** Finished pcode
** Starting moveSrc
Copying "code/examples/BuildPlanDetails.mlx" to "toolbox/examples/BuildPlanDetails.mlx".
Copying "code/gettingStarted.mlx" to "toolbox/gettingStarted.mlx".
Copying "code/simulateSystem.m" to "toolbox/simulateSystem.m".
Copying "code/viewPipeline.m" to "toolbox/viewPipeline.m".
** Finished moveSrc
** Starting toolbox
Packaging toolbox: release/Mass-Spring-Damper.mltbx
** Finished toolbox
** Starting tbxIntegTest
...
Test Summary:
Total Tests: 3
Passed: 3
Failed: 0
Incomplete: 0
Duration: 1.2816 seconds testing time.
** Finished tbxIntegTest
** Starting release
** Finished release
>>
Kicking off this command begins executing the build. What may be less obvious though is that it is orchestrating everything that needs to happen in order to complete the default task. The default task for this build is release, which produces a releasable *.mltbx file. However, not only does it produce the toolbox package, it also ran whatever tasks were required to do so and in the right order. It ran important tasks to keep your quality high such as tests and code analysis. It also produced doc HTML pages from *.mlx examples, it p-coded files with sensitive content, and even compiled the needed mex files automatically. Oh wait mex needed to run the setupCompiler task first? No problem. Taken care of. What a gem!
If you take a look at the current folder browser you can see several items that were created as a result of this build. For example:
- a results folder that includes test results and coverage artifacts.
- a toolbox folder that contains built mex files, the HTML doc to ship, p-coded files and copies of all the m-file and mlx sources required to ship in your toolbox package.
- a release folder that includes an installable Mass-Spring-Damper.mltbx file
You should quickly see that most tasks don't rerun because they support the incremental build features of the build tool, and since we have already built it, and we didn't make any changes, there is no need to rebuild. This is a huge efficiency saver when it comes to day to day development work.
For example, clean one of the tasks, say the pcode task to delete its output artifacts:
Now run the default build again and you can see only the pcode tasks needed to get rebuilt. Not even the toolbox task needs to rerun, because we didn't change the pcode source, so the files inside the toolbox we've already created are still valid.
Let's take a peek inside the buildfile to see how this has been done.
First you can see the buildfile is just a MATLAB function. In this function a build plan is created and this build plan has dictionary-like behavior as you create and assign tasks. Tasks can be assigned two ways:
- Through pre-made tasks that can be created and added to the plan. Examples of this are the CleanTask, MexTask, TestTask, CodeIssuesTask, and the PcodeTask.
- Through local functions defined in the buildfile function that end in "Task". These are added to the plan automatically when you pass the localfunctions call directly to the buildplan when you create it. This is valuable when there is no pre-made task available in your task library that meets the need of your build step.
Much of the buildfile consists of setting these tasks on the plan and setting their Dependencies and their Inputs and Outputs. It is often the case that Inputs and Outputs for pre-made tasks do not need to be set, as they are defined automatically but the pre-made task. For example, the MexTask automatically registers the *.c & *.cpp source files as Inputs to the task, and the produced MEX files as the Outputs of the task. This enables the build tool to know when it needs to rebuild.
OK, you have opened the project and because it is setup with the build tool you now know how to be productive. Let's move on to the task at hand, which is adding the CTF archive. To do this, you simply need to leverage MATLAB Compiler SDK APIs to produce a Production Server Archive and add that to your build file. Open the build file and uncomment lines 164-184.
showCode buildfile.m 166:186
166 %% The "ctf" task action
167 function ctfTask(context)
168 % Create a deployable archive for MATLAB Production Server
169
170 ctfArchive = context.Task.Outputs(1).paths;
171 ctfBuildResults = context.Task.Outputs(2).paths;
172
173 % Create the archive options for the build.
174 [ctfFolder, ctfFile] = fileparts(ctfArchive);
175 opts = compiler.build.ProductionServerArchiveOptions(...
176 ["code/simulateSystem.m", "pcode/springMassDamperDesign.m"], ...
177 FunctionSignatures="buildutils/simulateSystemFunctionSignatures.json", ...
178 OutputDir=ctfFolder, ...
179 ArchiveName=ctfFile, ...
180 ObfuscateArchive="on");
181
182 % Build the archive
183 buildResults = compiler.build.productionServerArchive(opts);
184
185 save(ctfBuildResults,"buildResults");
186 end
You can see this is straightforward. The steps are:
- Define a local function that ends in "Task".
- Grab the output information needed from the build context (we will cover where these outputs come from).
- Create the ProductionServerArchiveOptions. This involves determining the preferred option values for the archive and other activities such as writing the FunctionSignatures.json file. We have already written a FunctionSignatures file for you in this project.
- Use these options to create the archive.
- Save the build results meta-data to a .mat file for good measure. You will see why a bit later.
Now that the action is defined, define the dependencies and I/O on the build plan by uncommenting lines 78-85
showCode buildfile.m 80:87
80 %% Build a MATLAB Production Server archive
81 % Ad hoc task, task action defined in toolboxTask local function
82 plan("ctf").Dependencies = ["lint","test"];
83 plan("ctf").Inputs = ["code", "pcode", "buildutils/simulateSystemFunctionSignatures.json"];
84 plan("ctf").Outputs = [...
85 resultsFolder + "/ctf-archive/MassSpringDamperService.ctf", ...
86 resultsFolder + "/ctf-build-results.mat", ...
87 resultsFolder + "/ctf-archive"];
These lines configure the task as follows:
- Makes the task dependent on passing the code analysis rules and the unit tests. We should not package the archive if the code doesn't yet pass our quality requirements.
- Defines the inputs to the archive, namely the code folder, the pcode folder, and the FunctionSignatures.json file. If anything changes in these files and folders, the task knows to rebuild.
- Defines the outputs to produce. These outputs are processed in the actual local function that defines the task action. In this case it points to the ctf output file, a mat file to save the build results data structure, and the overall folder where the archive is produced, so that if anything changes in that whole folder, a rebuild will be triggered. Also, the ctf file and the mat file outputs are used inside of the task local function in order to know where to actually create these artifacts.
Now go ahead and invoke your new task:
Look in your results folder and you now have the ctf archive, it's auxiliary files, and a new ctf-build-results.mat file. Nice work.