Chapter 3: I only deploy on Fridays

OK, you now have:

Successfully created a fresh fork/clone of the code
Quickly and easily built a distributable toolbox
Added your enhancement to create a production server archive from the algorithm
Ensured this was a high quality submission.....(wait a minute)

Actually, you haven't done anything to assess or improve the quality. Does this CTF work? We actually have no idea. We have no data to suggest it does other than the fact that something was created by our ctf build step. Let's get a little test focused for a minute here.

You may be familiar with the unit test framework built in MATLAB. The astute observer noticed that tests were run as part of the build that produced the toolbox. Unit tests were written against the code, and integration tests ran against the toolbox package to ensure everything was packaged up correctly.

You haven't changed any code of the toolbox, so there shouldn't be a need to write any unit tests. However you have produced a new environment in which the code will run, namely on a production server. You need to have evidence that this deployment will work in production and produce correct results.

With the MATLAB Test product, you can now write equivalence tests to confirm that transformations that occur for Coder and Compiler products produces equivalent results as the tried and tested algorithm in MATLAB. For this workshop, we have written this test for you. Navigate to the integTests/equivalence folder and open MPSEquivalenceTest.m

openFile integTests/equivalence/MPSEquivalenceTest.m

Observe this test does the following:

showCode integTests/equivalence/MPSEquivalenceTest.m 1
1     classdef MPSEquivalenceTest < matlabtest.compiler.TestCase

It inherits from matlabtest.compiler.TestCase, which contains several methods to enable equivalence test authoring. Note there also is a very similar matlabtest.coder.TestCase for Coder workflows.

showCode integTests/equivalence/MPSEquivalenceTest.m 27:30
              % Load the results built in a prior build step
              disp("Load the build results from the ""ctf"" task")
              loadedData = load(resultsFile);
              buildResults = loadedData.buildResults;

It loads the build meta-data that we saved to a .mat file when we added the ctf build task. This data is all that is needed in order for the test to execute in the production server environment.

showCode integTests/equivalence/MPSEquivalenceTest.m 32:36
              % Execute the runtime inputs (damping) on the server
              disp("Executing design on a local Production Server")
              design.k = 5e5;
              design.c = damping;
              executionResults = testCase.execute(buildResults,{design},"simulateSystem");

It executes the built artifact in its deployed environment, in this case a locally spun up production server.

showCode integTests/equivalence/MPSEquivalenceTest.m 38:40
              % Verify server execution is equivalent to the local results
              disp("Verifying results match MATLAB results")
              testCase.verifyExecutionMatchesMATLAB(executionResults);

It then verifies that the execution on the server matches the results given from MATLAB. This method is not only easy and convenient, it also provides valuable diagnostic information when it fails.

Note, while in this case the executable was built as part of the overall build process in the build file, the equivalence TestCase also has a convenient method to build the executable directly in the test if desired (not shown in this test).

Alright, now let's add this to our build and run it! Open your buildfile and uncomment lines 88 to 96:

openFile buildfile.m 90
showCode buildfile.m 90:98
  %% Integration tests - back-to-back equivalence tests for the production server archive
  plan("ctfIntegTest") = TestTask("integTests/equivalence",SourceFiles=["code","pcode"], ...
      Description="Run integration tests against CTF archive.", ...
      Dependencies="ctf");
  
  
  %% Create the deploy task - does nothing but depends on other tasks
  plan("deploy") = matlab.buildtool.Task(Dependencies="ctfIntegTest", ...
      Description="Produce and test a ctf archive to deploy to a MATLAB Production Server");

Here we add a new ctfIntegTests task to the build, and define the outputs of the ctf task we created last time to the Inputs of our new test task. Also we add a new deploy task that simply depends on our new test task. This means that to deploy we not only need to create the ctf, we also need to test against it. Let's now run the build, including our new equivalence tests:

buildtool deploy
** Starting lint
Analysis Summary:
    Total Files: 21
         Errors: 0 (Threshold: 0)
       Warnings: 3 (Threshold: Inf)

Results:
    SARIF: results\win64\code-issues.sarif
** Finished lint

** Starting setupCompiler
MEX configured to use 'Microsoft Visual C++ 2022 (C)' for C language compilation.

To choose a different language, execute one from the following:
 mex -setup C++ 
 mex -setup FORTRAN
Compiler is detected and properly setup.
** Finished setupCompiler

** Starting mex_convec
Building with 'Microsoft Visual C++ 2022 (C)'.
MEX completed successfully.
** Finished mex_convec

** Starting mex_yprime
Building with 'Microsoft Visual C++ 2022'.
MEX completed successfully.
** Finished mex_yprime

** Starting mex
** Finished mex

** Starting test
.....
.
.
Generating test report. Please wait.
    Preparing content for the test report.
    Adding content to the test report.
    Writing test report to file.
Test report has been saved to:
 D:\a\Mass-Spring-Damper-PFT-2024\Mass-Spring-Damper-PFT-2024\results\win64\test-results.html
MATLAB code coverage report has been saved to:
 D:\a\Mass-Spring-Damper-PFT-2024\Mass-Spring-Damper-PFT-2024\results\win64\coverage\index.html

Test Summary:
    Total Tests: 7
         Passed: 7
         Failed: 0
     Incomplete: 0
       Duration: 2.6631 seconds testing time.
                 

Test Results:
    HTML: results\win64\test-results.html

Code Coverage:
    HTML: results\win64\coverage\index.html
** Finished test

** Starting ctf
** Finished ctf

** Starting ctfIntegTest
Load the build results from the "ctf" task
Executing design on a local Production Server
Verifying results match MATLAB results
.Load the build results from the "ctf" task
Executing design on a local Production Server
Verifying results match MATLAB results
.Load the build results from the "ctf" task
Executing design on a local Production Server
Verifying results match MATLAB results
.

Test Summary:
    Total Tests: 3
         Passed: 3
         Failed: 0
     Incomplete: 0
       Duration: 25.7766 seconds testing time.
                 
** Finished ctfIntegTest

** Starting deploy
** Finished deploy

open Chapter_4

Chapter 3: I only deploy on Fridays

Preface & TOC | Chapter 4