Unit Testing Framework for SHA256

Comfort, peace of mind, relieve. Unit testing can bring this kind of emotion. I forgot to mention, it brings dividends too. This article shows that creating a unit testing infrastructure can be very simple. This unit testing implementation was built to help testing the SHA256, a by The Books Implementation

Unit Testing: Why

A few reasons. It self documents the tests run on a piece of code. It shows what part of the code is being tested. It is part of the code so it speaks the developers language. And I think one of the most important, every time the code is changed, we can run exact the same tests over and over and make sure the change did not break any test case. Comfort, peace of mind, relieve.

To me, unit testing is at the same level of version control, it brings comfort, peace of mind, relieve.

Unit Testing: Limitations

Unit tested code is not necessarily correct. Even though the parts are correct, it does not mean the whole will behave as expected. Unit testing does not work very well with software that handles user input or other environment events, these inputs need to be emulated, which is never the same as the actual world.

Unit Test Framework

There are a lot of unit test frameworks to choose from. Some simple, some complicated. It is better having a simple unit testing framework implemented than having a fancy, full of unneeded features and not implemented. For instance, I looked at the Google Test, I am sure it is good and nice, but it requires pthread, which is not currently available in Webassembly. But again, why over complicate? I chose the simpler minimum unit test framework .

MinUnit, the two Macros Unit Test

MinUnit is more than enough. It does not have any dependency. It can run even on environments with no Operating System.

I have changed it a little bit, lets look into it.

There are just two macros, mu_assert() and mu_run_test(), they are defined with comments below. Use mu_assert() to verify the result of a test, the mu_assert test input is the test condition, and message is the message to be displayed in case the test fails. Use mu_run_test() to execute a function that contains one or more tests.

Content of my modified minunit.h


/* file: minunit.h */

// Minimum Unit Testing, assert macro.
// If the test condition is 0(FALSE), then print message and return -1
// If the test condition is not 0(TRUE), increment test number and
// go to the next test
//
#define mu_assert(message, test) \
    do \
    { \
        DBG_LOG(DEBUG_WASM_DEBUG, "Running test:%d\n", tests_run); \
        if(!(test)) \
        { \
            DBG_LOG(DEBUG_WASM_DEBUG, "%s-FAILED!\n", message); \
            return -1; \
        } \
        tests_run++; \
    } while (0)

// Minimum Unit Testing, run test macro.
// Execute test_func(), if test_func() is successfull go to the next test,
// return -1 otherwise
//
// test_func() should return 0 if succssfull
//
#define mu_run_test(test_func) do{ if( test_func() )  return -1; } while (0)

extern int tests_run;
 

Confused about the while(0)? Read While(0), Why Macros Use It?.

What To Test?

We are going to test the SHA256, a by The Books Implementation. At some point we need to figure out what needs to be tested. Here are some guidelines:

  • Feed zero length. This is easy.
  • Look for corner cases. It is amazing how many errors occur around +-1 counts. For instance, imagine you need to construct a 10 meters fence and use one pole every 1 meter, how many poles do you need? Not 10. It takes 11 poles. This kind of situations generates a lot of bugs! One simple mental model to use is to reduce the situation to a minimum and examine it, for instance, it would take 2 poles to build a 1 meter fence, so it is safe to say it takes the (number of meters +1) poles to build this kind of fence.Examining our SHA256 implementation, I can see there is an important corner case when the input message has 448 bits:
    
      // Pad the message so that its length is congruent to 448 modulo 512
      if( l%512 < 448)
         k = 448 - l%512;
      else
         k = 512 + 448 - l%512;

    This is clearly a test case that needs to be exercised

  • Corner case +-1. Just an addiction to the previous test case, sometimes the corner case manifests itself at +-1 counts.
  • Large data. Most important to test for memory leaks. But it also can help find bugs where the implementation locks up or overrun buffers.

SHA256 Unit Test Module

Please look at the sha256-unit-test.c below and its comments. We basically create the sha_test_vector_entry_t structure that holds all needed data in order to exercise a test case, then we create the sha_test_vector table with all test cases. Most of the SHA256 test data I got from SHA256 useful test vectors.


#include <string.h>
#include <emscripten.h>

#include "sha256.h"
#include "debug-wasm.h"
#include "minunit.h"

// This variable identify what test is running
int tests_run;

static int SHA256_UT_TableBasedTests(void)
{
    typedef struct _SHA_TEST_VECTOR_ENTRY_t_
    {
        char        *desc;              // Test description, shows when test fails
        uint8_t     *expected_result;   // Expected result
        uint8_t     *input;             // input to be fed to the tested function
        size_t      input_size;         // input size
        size_t      input_times;        // how many times data is fed to the tested function
    } sha_test_vector_entry_t;
    
    unsigned char shaResult[SHA256_DIGEST_SIZE];    // SHA result goes here
    
    // Test vectors
    sha_test_vector_entry_t sha_test_vector[]=
    {
        {
            "No data",
            (uint8_t []){0xe3,0xb0,0xc4,0x42,0x98,0xfc,0x1c,0x14,0x9a,0xfb,0xf4,0xc8,0x99,0x6f,0xb9,0x24,0x27,0xae,0x41,0xe4,0x64,0x9b,0x93,0x4c,0xa4,0x95,0x99,0x1b,0x78,0x52,0xb8,0x55},
            0,
            0,
            1
        },
        {
            "When known abc",
            (uint8_t []){0xba,0x78,0x16,0xbf ,0x8f,0x01,0xcf,0xea ,0x41,0x41,0x40,0xde ,0x5d,0xae,0x22,0x23 ,0xb0,0x03,0x61,0xa3 ,0x96,0x17,0x7a,0x9c,0xb4,0x10,0xff,0x61,0xf2,0x00,0x15,0xad},
            (uint8_t []){'a','b','c'},
            3,
            1
        },
        {
            "440bits = corner case-1 byte",
            (uint8_t []){0xaa,0x35,0x3e,0x00,0x9e,0xdb,0xae,0xbf,0xc6,0xe4,0x94,0xc8,0xd8,0x47,0x69,0x68,0x96,0xcb,0x8b,0x39,0x8e,0x01,0x73,0xa4,0xb5,0xc1,0xb6,0x36,0x29,0x2d,0x87,0xc7},
            (uint8_t []){'a','b','c','d','b','c','d','e','c','d','e','f','d','e','f','g','e','f','g','h','f','g','h','i','g','h','i','j','h','i','j','k','i','j','k','l','j','k','l','m','k','l','m','n','l','m','n','o','m','n','o','p','n','o','p'},
            55,
            1
        },
        {
            "448bits = corner case",
            (uint8_t []){0x24,0x8d,0x6a,0x61,0xd2,0x06,0x38,0xb8,0xe5,0xc0,0x26,0x93,0x0c,0x3e,0x60,0x39,0xa3,0x3c,0xe4,0x59,0x64,0xff,0x21,0x67,0xf6,0xec,0xed,0xd4,0x19,0xdb,0x06,0xc1},
            (uint8_t []){'a','b','c','d','b','c','d','e','c','d','e','f','d','e','f','g','e','f','g','h','f','g','h','i','g','h','i','j','h','i','j','k','i','j','k','l','j','k','l','m','k','l','m','n','l','m','n','o','m','n','o','p','n','o','p','q'},
            56,
            1
        },
        {
            "456bits = corner case+1 byte",
            (uint8_t []){0x4e,0x0e,0xa7,0x75,0xaa,0x77,0x66,0xcf,0x27,0x36,0xd5,0xa4,0x0e,0x4c,0x6f,0x76,0xe1,0xc4,0xc6,0x9e,0xda,0x0f,0x3c,0x78,0x2c,0x5a,0x2b,0xd7,0x7b,0x3f,0x96,0x95},
            (uint8_t []){'a','b','c','d','b','c','d','e','c','d','e','f','d','e','f','g','e','f','g','h','f','g','h','i','g','h','i','j','h','i','j','k','i','j','k','l','j','k','l','m','k','l','m','n','l','m','n','o','m','n','o','p','n','o','p','q','r'},
            57,
            1
        },
        {
            "896bits = 2*corner case",
            (uint8_t []){0xcf,0x5b,0x16,0xa7,0x78,0xaf,0x83,0x80,0x03,0x6c,0xe5,0x9e,0x7b,0x04,0x92,0x37,0x0b,0x24,0x9b,0x11,0xe8,0xf0,0x7a,0x51,0xaf,0xac,0x45,0x03,0x7a,0xfe,0xe9,0xd1},
            (uint8_t []){'a','b','c','d','e','f','g','h','b','c','d','e','f','g','h','i','c','d','e','f','g','h','i','j','d','e','f','g','h','i','j','k','e','f','g','h','i','j','k','l','f','g','h','i','j','k','l','m','g','h','i','j','k','l','m','n','h','i','j','k','l','m','n','o','i','j','k','l','m','n','o','p','j','k','l','m','n','o','p','q','k','l','m','n','o','p','q','r','l','m','n','o','p','q','r','s','m','n','o','p','q','r','s','t','n','o','p','q','r','s','t','u'},
            112,
            1
        }
    };

    // Loop all test cases
    for(size_t test=0 ; test<6 ; test++ )
    {
        // Feed data into tested function this many times
        for( size_t input_times=0 ; input_times<sha_test_vector[test].input_times ; input_times++ )
        {
            // Feed data
            Sha256_Compute(sha_test_vector[test].input, sha_test_vector[test].input_size, shaResult);
        }
        
        // Did it work?
        mu_assert(sha_test_vector[test].desc, (memcmp( shaResult, sha_test_vector[test].expected_result, SHA256_DIGEST_SIZE )==0));
    }
        
    return 0;
}

// Run SHA256 unit test
// returns 0 if all tests passed, non 0 if at least a test failed
//
EMSCRIPTEN_KEEPALIVE int SHA256_Unit_Test(void)
{
    mu_run_test(SHA256_UT_TableBasedTests);
    return 0;
}
 

Where is The Large Data Test Case?

One million a characters caused my Firefox browser to become unresponsive, I have decided to address this condition on the next article or else I would mix unit test subject with how not hang the browser with your script.

Running the Unit Test

Everything passed, this is how it should look like:

sha256-all-pass

Need to Test the Unit Test!

It happened to me before, I have created the unit tests, all passed, but the test wasn’t test anything! Short term happiness. Do not forget to test your unit test! Here I just changed one byte of the expected results array for 448 bits so the result won’t match, you should see an error:

Final Details

This code builds on top of the SHA256, a by The Books Implementation article, there you can find the source code for sha256.c, sha256.h, debug-wasm.c and debug-wasm.h. Following is the content of the sha256.html file, which invokes the Webassembly SHA256 unit test function:


<html>
<head>
  <meta charset="UTF-8">
  <script type="text/javascript" src="sha256.js"></script>
  <script>
    Module()
    .then(function(instance){
        var exports = instance['asm']; // the .js file puts the exports in the 'asm' field
        // Add button listener, calls Webassembly's sha256Test() when clicked
        var button = document.getElementById('sha256Button');
        button.addEventListener('click', function() {
            exports._SHA256_Unit_Test();
        }, false);
      }
  );
  </script>
</head>
<body>
  <input type="button" id="sha256Button" value="click here to run SHA256 UNIT TESTs"/>
</body>
</html>
 

Compiling it:


emcc sha256.c sha256-unit-test.c debug-wasm.c -O1 -s MODULARIZE=1 -s WASM=1 -o sha256.js 
 

Start EMSCRIPTEN server:


emrun --no_browser --port 8080 .
 

Point your browser to: http://localhost:8080/sha256.html


Leave a message below. Webassembly is evolving rapidly, please let me know if this post got outdated.

Enjoyed this post?

Don't miss new posts: Share it with your friends:

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *