• Home
  • Blog
  • Projects
  • Bookshelf
  • About

Testing Software






Writing unit tests for code is surprisingly a hot button topic within software development. Many developers have very strong opinions in how tests should be approached, written, and implemented.

Over the past few months, several of our engineering departments have been pushing for more adoption of frontend tests for their code. Thanks to this push, our Frontend Platforms team has found that we need to put a lot more effort into the documentation around writing tests. Several engineers, from both our team and other feature teams, have been writing docs, pairing with others to write tests, and writing guides and presentations around unit testing.

Through all this work, we have slowly shifted our mindset from approaching testing as something that is a consistent, unit-based, solid foundation for an application to something that is a bit more fluid, a bit farther away from testing each unit individually, and something that changes as the application changes.

The key takeaway I have learned from a coworker is that the type and value of the tests you write depends on if you are writing the code for the application at the same time as the test or if you write the test after the feature is code-complete.

Writing tests before a code-complete feature is completely different from writing them when the feature is done

We have found that a lot of the pain points around writing unit tests are a result of attempting to write unit tests after the feature is code complete and deployed to production.

A lot of our other learnings have been so simply summarized by this tweet by Guillermo Rauch:

Twitter
Published
Last modified
....
....
← Back to all posts
Share this post on Bluesky
See discussion on Bluesky

Write tests. Not too many. Mostly integration. — Guillermo Rauch (@rauchg)

December 10, 2016

This short tweet is packed with so much wisdom, the core takeaways we have had as a team have been:

  • Don't focus on code coverage
  • Prefer integration tests over unit tests

Don't Focus on Code Coverage

Code coverage is one of those feel-good stats that we seek as developers, giving us a rush when we see the coverage report come back with a higher percentage than we had previously. Unfortunately it has absolutely no value to the end user of your application.

Users do not care what your code coverage percentage is

You never see consumer products showcase these stats to their users in their footers, nor have I ever seen another developer choose an npm dependency on their code coverage percentage.

Your code will change, and your application will change as well, you shouldn't worry about how well covered the code is, and instead focus on how well the user flow is covered. Do you have a test for that checkout flow on site? What about for the user login flow? These are the things worth capturing coverage for, real parts of the application and not the number of lines of code.

Prefer Integration Tests over Unit Tests

In my experience, preferring to write integration tests rather than unit tests is even more contentious than the point above about code coverage, many developers seem to extrapolate from TDD that tests must be focused to units of code rather than writen for the larger picture of the feature or application as a whole.

The tests that I have found that are worth keeping around over time are these integration tests, ones that aren't at all worried about the implementation of the feature but rather the user flow through the feature.

The most important goal in software development (keep in mind writing tests is part of this work) is to deliver a working, enjoyable experience for the customer. Your customer won't care about the implementation of the checkout button, or if you are using some middleware for authentication. The key, when writing code, is to think about the user. Sometimes your user may be another developer using your service to implement another feature, or it may be a customer looking to buy their favorite bed frame.

The Half-Life of Code and Tests

I think the key for getting value out of software testing is not that you must write integration tests through all stages of development, but rather that over time the tests that should remain in the codebase should be scoped as integration tests.

Many of the developers I have talked with consider tests as these solid, never changing pillars of a codebase; however in my experience the type of tests, and the value of these tests, changes dramatically depending on what I am doing. If I am working on a refactor of a component for example, I may want to setup some visual regression tests, then I can refactor with confidence, and when the refactor work is done I can tear down the tests.

Unfortunately many of our modern testing tools don't really communicate this ephemerality of testing software, either through their implementation or through their documentation. This leads new developers to approach writing tests as something to do once, ship it and then forget it.

Often unit tests, those written to accomplish a code coverage goal or test a particular implementation detail, add friction to development that so many developers worry about when they try to get into testing their code. A good mental model for tests is that often pure unit tests should have a short half-life within the codebase, meaning they only exist for a short amount of time (frequently only during the very early stages of a new feature or application), and the half-life of integration tests is much longer and should remain in the codebase for the lifetime of the feature.

Unit tests should have a short half-life within your codebase, focus on code that has a long half-life instead



Tags:

Web DevelopmentTesting

Loading...

Related Posts

Web Development

Website Redesign v11

....

I've once again updated my personal website, this time powered by my own RSC framework and a custom CMS solution as well!

Onboarding Your New AI Teammate

....

Are we reinventing the wheel when it comes to onboarding new AI agents to a codebase, when we already have primitives available for onboarding humans?

Quick Tip - Theme Aware Images

....

Have you ever found the need to change the image you render on a web page based on the current preferred color scheme of your theme?

Async Class Creation In JavaScript

....

Have you ever wanted to create a class in JavaScript or TypeScript but also have the initialization be async? Here's a quick tip on a pattern that I've used in the past!

Website Redesign v10

....

I recently launched a rewrite and redesign of this personal website, I figured I'd talk a bit about the changes and new features that I added along the way!

Server Side Rendering Compatible CSS Theming

....

A quick tip to implementing CSS theming that's compatible with Server Side Rendered applications!

Podcasting By Hand

....

A brief overview on how we launched The Bikeshed Podcast, including a deep dive in our recording and distribution workflows!

Quick Tip - Specific Local Module Declarations

....

A quick tip outlining how to provide specific TypeScript type definitions for a local module!

On File-System Routing Conventions

....

Some rough thoughts on building a file-system routing based web application

You're Building Software Wrong

....

Slicing software: why vertical is better than horizontal.

Single File Web Apps

....

What if you could author an entire web application in a single file?

Resetting Controlled Components in Forms

....

A quick way to handle resetting internal state in components when a parent form is submitted!

A Quick Look at Import Maps

....

A brief look at Import Maps and package.json#imports to support isomorphic JavaScript applications!

Recommended Tech Talks

....

A collection of tech talks that I regularly re-watch and also recommend to everyone!

Request for a (minimal) RSC Framework

....

Some features and functionality that I'd like within a React Server Component compatible framework.

Bluesky Tips and Tools

....

A (running) collection of Bluesky tips, tools, packages, and other misc things!

The Bookkeeping Pattern

....

A quick look at a small but powerful pattern I've been leveraging as of late!

TSLite

....

A proposal for a minimal variant of TypeScript!

Monorepo Tips and Tricks

....

Sharing a few core recommendations when working within monorepos to make your life easier!

Next.js with Deno v2

....

This is a quick post noting that Next.js should now work with Deno v2!

Don't Break the Implicit Prop Contract

....

React components have a fundamental contract that is often unstated in their implementation, and you should know about it!

A Better useSSR Implementation

....

Replace that old useState and useEffect combo for a new and better option!

My Current Dev Setup

....

A quick look at the applications and tools that I (generally) use day to day for web development!

Abstract Your API

....

Proposing a solution for sharing core "business" logic across services!

Tip: Request and Response Headers

....

There's a common gotcha when creating Web Request and Response instances with Headers!

Using Feature Toggles to De-risk Refactors

....

Feature toggles are often underused by most software development teams, and yet offer so much value during not only feature development but also refactors

Hohoro

....

A quick introduction to my new side project, hohoro. An incremental JS/TS library build tool!

Custom Favicon Recipes

....

Two neat tricks for enhancing your site's favicon!

Corporate Sponsored OSS

....

The various risks and pitfalls of open source software run by corporations.

The Library-Docs Monorepo Template

....

A monorepo template for managing a library and documentation together.

Building Better Beacon

....

How we solved an almost show-stopping production bug, and how you can avoid it in your own projects.

Project Deep Dive: Tails

....

A(nother) deep dive into one of my recent side projects; tails - a plain and simple cocktail recipe app.

Churn Anxiety

....

When did semver major changes become so scary?

Service Monitors and Observability

....

Leveraging service monitors properly to improve service observability.

On Adopting CSS-in-JS

....

A brief recap of how Wayfair changed it's CSS approach not once but twice in the span of 5 years!

Project Deep Dive: Microfibre

....

A deep dive into one of my recent side projects; microfibre - a minimal text posting application

Pair Programming

....

Pair programming can be good sometimes - but not all the time

Suspense Plus GraphQL

....

A few thoughts on using Suspense with GraphQL to optimize application data loading

You've Launched, Now What?

....

A few thoughts on what to do after you launch a new project

Taking a Break

....

A few quick thoughts on burn out and taking a break

Managing Complex UI Component State

....

A few thoughts on managing complex UI component state within React

Understanding React 16.3 Updates

....

A quick overview of the new lifecycle methods introduced in React 16.3

CSS in JS

....

A few thoughts and patterns for using styled-jsx or other CSS-in-JS solutions

Redesign v6

....

A few thoughts on the redesign of my personal site, adopting Next.js and deploying via Now

JavaScript Weirdness

....

A few weird things about JavaScript

Calendar

....

Building a calendar web application