Testing as an act of fun
Testing as an act of fun
In which Rik demonstrates how he tests his canvas library
Welcome! First off, a note about my Substack posting rate.
There's a reason why I'm not a professional writer (apart from the poetry, of course) or even a paid-by-the-word journalist. It's deadlines. I don't like them; they irritate me.
There is nothing more dispiriting than having to sit down in the morning knowing that at some point in the day I'm going to have to generate 800 creative, curated words on Topic Whatever which I can then dispatch into a cacophonic world for the delight of whoever stumbles across them.
Substack tutorials insist I send out regular posts to "build a readership". They also advise me to "monetize" my writing. The thing is, I'm not here to build a readership: people can subscribe and unsubscribe as they see fit and, honestly, it won't bother me. I'm writing mainly for myself here.
As for charging people to access and read my witterings - that's never going to happen! As I said in a previous post, "... one of the key purposes of this venue is to give me the space to memorialise myself. A place to note down snippets of my life and then, maybe, explore them ..." And as entertaining as such a car wreck of an endeavour may be, I see no reason to charge people for the privilege of witnessing it.
Now that's out of the way, let's move on to the main event of this post.
There's a number of reasons why I haven't posted for a few weeks. First, I got distracted by the United Kingdom's recent General Election. Which was, for the most part, very tedious. How many times can a governing party drive its election campaign into a ditch, or a wall, at full speed? We now know the answer to that question - though I have a feeling that there's a couple of parties in North America determined to show us Brits how to do it properly.
(Aside: thank the Deity-of-choice for Ed Davey, bless his still-damp cotton socks!)
My other major reason for not building my Substack readership is because I've been working on Scrawl-canvas, my open-source Javascript library which aims to make coding up responsive, interactive and more accessible HTML5 canvas elements both easier, and more fun.
... Which is the subject of today's post. So: onwards and upwards!
The joys of maintaining a free-to-use Javascript library
I have a Day Job where I get paid to write front-end code. Most of the code I write ends up in internal tools, though sometimes I'll do a small bit of work on one of my employer's websites.
My Day Job code relies heavily on Javascript frameworks and libraries to Get Stuff Done. It makes good sense: if the user needs to type their name into a form, why write the code to handle all the associated functionality when someone's already written a React component to manage that exact scenario? Plumb it in, write the tests; push and go. Next ticket, please!
For most of my Day Job colleagues, that's enough coding to satisfy their requirements. Outside of our daily office routines they head off and have proper, interesting lives with friends and families and adventures and stuff.
Me? I have an open source Javascript library to develop and maintain. Evenings and weekends are for wimps!
Now there's a whole shedload of things I could write about the work I do on my library. It is, after all, a 2D canvas library which helps developers build animations, infographics - even entire interfaces if they want to head down that rabbit hole of frustrations and pain! But for this post I've decided to concentrate on one of the last steps in the process of releasing a new library update: testing the code across browsers.
tl;dr? - I do the final testing manually. Some of the tests are quite pretty. On a good day I can get through all 170 tests in an hour. And yes, I've screen captured me performing the tests so I can share them with you, dear reader, today!
Testing, Testing ...
Before I talk about the recordings I've made, I need to explain why I have 170 tests which I need to run manually across browsers every time I want to push a new version of the library into production.
First, the library is written in Javascript. Yes, I know - feel free to spit in my general direction! If I was to start writing the library from scratch today, I'd probably try to write it in Typescript. But the result would be a very different library, with a very different API.
To compound the stupidity, the library doesn't include unit tests. I do make use of a linting library which, nowadays, is good for catching all sorts of bad code choices. Again, if I was starting from scratch there would definitely be unit tests but, after ten years, the library's functionality space is too big! I just don't have the time, or the will, to do the necessary work.
Still, code has to be tested. Which is why the library comes with a suite of 170 (and counting) demos.
When I write new functionality to add to the library, I start by writing a demo: a hand-coded HTML web page with an associated (hand-coded) Javascript file. The web page will often include a set of UI controls - select boxes, range inputs, etc - which allow me to adjust relevant values, the results of which I can then see in real-time in the canvas element. Development occurs. Once everything is performing the way I expect it to, and all the other demos continue to work as expected, the work is done.
For the final pre-release tests, I review every demo in Chrome, Firefox and Safari. I check the browsers side-by-side; navigation in one browser triggers navigation in the other two using a piece of magic called BrowserSync. As magic goes, I can't recommend this server enough - before I added it to the library's dev-dependencies final testing could easily take me half-a-day, sometimes longer!
Testing results - part 1 (4mins 50secs)
Note that I won't be commenting on every test, just a few of the interesting bits.
00:07 - Create, clone, kill, resurrect. The first few tests all involve Block and Wheel entitys. One of the main thing I'm looking for here is to make sure cloning functionality works, alongside deleting and restoring the entitys. A key issue the library needs to guard against is memory leaks: when a Block or Wheel gets killed, references to it need to be deleted everywhere in the code base, not just the canvas display.
00:19 - Position by reference. Scrawl-canvas (SC) makes heavy use of positioning things relative to other things, rather than relying on (nested) groups/layers and x/y coordinates.
00:27 - Gradients. SC goes above-and-beyond the gradient functionality supplied by the native Canvas API. These are the tests where we properly meet user controls - it's easy to forget I'm testing and instead lose myself in play.
01:15 - Stress testing tween functionality. Sadly Safari can't keep up with the other browsers.
01:40 - Filters. These are always fun. If recent code changes haven't touch filter functionality I can test them here then skip a bunch of filter-specific tests further down the line.
02:01 - The first test featuring the Picture entity. Again, a very capable time waster if I forget I'm supposed to be testing, not playing. The video test shortly after includes a device camera feed!
02:54 - Path-based entitys. Easy to whizz through if the path functionality has not been touched.
03:47 - Colors. Just check-and-go. The days of variability in the displayed colors between browsers have, thankfully, passed.
04:10 - The gradient stress test. Another place where we can test the CPU to destruction!
04:45 - Sprite sheets. I've only coded up one test for sprite sheets, which barely touches their functionality. Something I'll have to fix one day!
Testing results - part 2 (7mins 25secs)
00:04 - Grid entity. Shameful confession time. This is functionality in need of a use case - the concept seemed cool when I coded it up, but grids are on the list of things that I will probably, one day, deprecate.
00:44 - Loom entity. Not many canvas libraries can do this sort of magic. Enjoy!
01:13 - Canvas settings demo. A useful test if I ever find myself playing in canvas/cell code. More useful as a learning opportunity - canvases used to be difficult to set up correctly in SC, though I think I've managed to get rid of most of that DevEx pain.
01:37 - The interactive video test. It's important to test this with the keyboard as well as the mouse!
02:52 - Kaleidoscope clock. One of the few tests where Chrome is significantly slower than the other browsers. A mystery I'm not planning to solve anytime soon.
03:11 - Accessibility testing. This is one of the key accessibility tests - I don't perform all the checks on this run (bad Rik!).
04:58 - Clipping tests. I don't like the clipping functionality in the Canvas API. There's better ways to achieve the same results.
05:34 - Some more kill functionality tests. I can't stress enough how important it is to test that the library gets rid of all references to killed SC artefacts to avoid memory leaks.
06:07 - Filtering a media stream. Tests have a right to be fun if they want to be fun!
06:38 - Frame rates. A good example of a test that covers different types of functionality, in this case delta animations and, separately, maximum frame rates. It turns out that some displays can handle faster frame rates, which also depends on the browser. Part of the testing regime not shown in these videos is where I drag the browser tab running the test between display screens with differing capabilities just to make sure things don't fall apart when the drag completes.
07:00 - Noise assets. This test should come with a warning because it can seriously distract from the testing task at hand. The next test (Reaction-Diffusion assets) is a similar time threat.
Testing results - part 3 (19mins 21secs)
I have to apologise for the length of this video. I got so involved in the testing, I forgot I was recording it!
00:37 - Raw asset manipulation with added video recording functionality. Honestly? I could spend hours playing with this test. Posting the resulting videos to social media is very satisfying!
01:14 - Minimap proof of concept. There's a better (more accessible) test of this sort of functionality later on.
01:32 - Wide gamut colors. Needs to be tested on a display/browser combination that supports wide gamut colors - which is not the case in this run through.
01:36 - Label entitys. Making sure the label text gets copied into the canvas shadow DOM (for accessibility) is an essential part of these tests.
04:39 - Enhanced label entitys. Here begins some of the most complex tests in the entire suite. So, so many parameters available for manipulation! And so much room for error when using non-western fonts. I have no idea how I could even start to write unit tests for this sort of thing - especially when it comes to line breaking, styling, letter/word/line spacing, text-direction (including vertical), character pivoting, underlines, etc, etc ...
09:17 - Filter tests. This extensive set of tests allows us to play with all the parameters for each filter. The important thing to remember here is that testing the filter on just one image is not enough - I have a library of images available to drag onto the test canvas, which is very useful for making sure the filters work in line with their design across a wide range of images.
16:30 - Reduce palette filter. This is - by a long stretch - the most dangerous test of them all! I can lose myself for hours playing with the parameters as I drag over different images and try to get them to display well with just a 12 color palette.
18:02 - Compound filter effects. Not so much a test, more me showing off what the SC filters system is capable of achieving.
Testing results - part 4 (12mins 52secs)
00:00 - Emitter particle entity tests. Particles are fun, but very heavy on the CPU if not well-coded.
01:58 - Net particle entity tests. A good place to test a particle system to destruction.
04:38 - Mesh entitys. Useful for distorting images.
05:15 - DOM-based testing. Because canvas elements are part of the DOM. There's some weird and wacky tests in this section - particularly later on.
08:09 - 3D rotated canvas test. Tracking the mouse cursor across a 3D rotated canvas used to be a huge problem across browsers. Happily things have settled down a lot over the past few years.
10:09 - Newer Web APIs. Getting the canvas element, with SC, to play nicely with some of the more recently introduced Web APIs - such as the Fullscreen, Screen capture and Popover APIs - has been the focus of my more recent development work. Failures in some of these tests can always be expected!
12:24 - Packets tests. Yes, these tests are really boring. But they test stuff which is fundamental to the way the whole library works. Endure the tests as they run - much more interesting tests will follow along shortly …
Testing results - part 5 (9mins 15secs)
When I reach the first modules test, I know I'm in sight of the finishing line. Even so, some of these tests can be very distracting. Resolves must be stiffened as we race down the home stretch!
00:00 - Accessible, responsive charts proof-of-concept test. Testing with the keyboard is essential, as is checking the text rendered in the canvas shadow DOM (which I didn't do in this run through).
01:40 - Import and use Lottie files. Because: why not?
01:46 - Accessible canvas editing GUI proof of concept test.
02:30 - Scrollytelling proof of concept test. Just scroll and observe, then move on. And yes, “scrollytelling” is a real word (according to the New York Times).
04:24 - Accessible before/after slider infographic proof of concept test.
04:56 - Header text snippets proof of concept test. Pure fun to play with! Less fun to code up.
07:55 - Machine learning tests. In particular MediaPipe models. These tests use very old models which do their stuff in Wasm-land. But AI/ML Shiny is Hot Shiny nowadays so it's important for me to check that SC can play with such models across browsers.
Aaaaand ... we're done!
Conclusions
If you've made it this far - congratulations! Maybe you have the patience to contribute some effort to my mad endeavour? Or not. Whatever.
There are several obvious criticisms that can be made about my approach to testing Scrawl-canvas. I can see the imaginary line of Engineers forming disorderly already! For instance:
There's no excuse not to start introducing Typescript into the code using JSDoc.
Even a few unit tests in critical areas would help give people more confidence to use the library in their products.
A lot of this work could be automated with the help of modern integration testing suites such as Chromatic or BrowserStack or Shiny New AI-Based Integration Testing Solution Phone Us For Competitive Terms Today!
... I apologise. You're absolutely right - I have no excuses. Except: I like the way I test Scrawl-canvas.
Please indulge me as I knit my defence!
It's like this. As the library has evolved over the last decade, so my approach to working on it has evolved. Back in 2017 our relationship was not good. In fact it had become toxic: I had come to view SC as a parasite on my time and effort. Not just for the coding work, but also everything else that goes into maintaining a library: the documentation; the futile attempts at promotion; the costs of maintaining a website to showcase my little product in a tsunami of products ... it felt like hammers on my psyche.
So I took a hard decision and chose to abandon my obsession. Walk away from it; forget about its existence.
That breakup lasted for two years.
When I finally looked again upon my work, something sparked within me. Those two years of doing Other Things gave me a perspective on what I had liked, and didn't like, about the library - including some valuable insights on my working relationship with it.
In 2019 I restarted from scratch. Every line in the code base was new. And this time I took consideration for how I could maintain that new code in a more sustainable, life enhancing way.
The decision not to automate as much of the work as possible ... it was a deliberate decision. Because I stopped looking at the library as a product to be evangelised, but instead came to see it as a work of craft. An ongoing act of making which can, and does, add value to my self-worth.
I test by eye-hand because in that process of testing I can remember again the problems I've tackled and defeated. The issues overcome to get this functionality, or that effect, to work as it does today. It's important for me to regularly remind myself of those victories, value them, take Joy in them!
For even code can be poetry. And I am, above all, a Poet!