Testing Applications

The spin test plugin allows you to run tests, written in WebAssembly, against a Spin application (where all Spin and WASI APIs are configurable mocks).

To use spin test you write test scenarios for your app in any language with WebAssembly component support, and mock out all interactions your app has with the outside world without requiring any code changes to the app itself. That means the code you test in development is the same code that runs in production.

Note: spin test is still under active development and so the details here may have changed since this post was first published. Check the spin-test repo for the latest information on installation and usage.

Prerequisites

The example below uses the Rust programming language and cargo components(a cargo subcommand for building WebAssembly components).

Installing the Plugin

To run spin test , you’ll first need to install the canary release of the plugin. As spin test matures, we’ll be making stable releases:

spin plugin install -u https://github.com/fermyon/spin-test/releases/download/canary/spin-test.json

This will install the plugin which can be invoked with spin test.

Creating App and Component

First, create an empty Spin app, change into that app folder and then add a component inside it:

$ spin new -t http-empty my-app --accept-defaults
$ cd my-app/
$ spin add -t http-rust my-component --accept-defaults

The above commands will scaffold out the application in the following format:

my-app/
├── my-component
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── spin.toml

Creating a Test Suite

There is currently first-class support for writing tests in Rust, but any language with support for writing WebAssembly components can be used as long as the fermyon:spin-test/test world is targeted. You can find the definition of this world here. For this example, we’ll use Rust.

We use cargo to create a test suite, and then change into our newly created tests directory:

$ cargo new tests --lib
$ cd tests

After running the cargo new command we will have the following application structure:

my-app/
├── my-component
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
├── spin.toml
└── tests
    ├── Cargo.toml
    └── src
        └── lib.rs

From within that test suite (from inside the tests directory), we then add the spin-test SDK reference:

$ cargo add spin-test-sdk --git https://github.com/fermyon/spin-test

Then, we open the Cargo.toml file from within in the tests directory and edit to add the crate-type of cdylib:

[lib]
crate-type = ["cdylib"]

The my-app/tests/Cargo.toml file will look like this after editing:

[package]
name = "tests"
version = "0.1.0"
edition = "2021"

[dependencies]
spin-test-sdk = { git = "https://github.com/fermyon/spin-test", version = "0.1.0" }

[lib]
crate-type = ["cdylib"]

Writing a Test

Next, create a test that spin test can run as a compiled WebAssembly component.

In this example, we will write some tests appropriate to a JSON API service for information about service users. Here are two such tests written in Rust using the Spin Test Rust SDK. The first test ensures that the Spin app responds properly to an HTTP request. The second test ensures that the Spin app responds properly when the user data is present in the key-value store - for testing purposes, simulated by inserting it into a “virtual” store.

Open the my-app/tests/src/lib.rs file and fill it with the following content:

use spin_test_sdk::{
    bindings::{fermyon::spin_test_virt, wasi, wasi::http},
    spin_test,
};

#[spin_test]
fn send_get_request_without_key() {
    // Perform the request
    let request = http::types::OutgoingRequest::new(http::types::Headers::new());
    request.set_path_with_query(Some("/")).unwrap();
    let response = spin_test_sdk::perform_request(request);

    // Assert response status and body is 404
    assert_eq!(response.status(), 404);
}

#[spin_test]
fn send_get_request_with_invalid_key() {
    // Perform the request
    let request = http::types::OutgoingRequest::new(http::types::Headers::new());
    request.set_path_with_query(Some("/x?123")).unwrap();
    let response = spin_test_sdk::perform_request(request);

    // Assert response status and body is 404
    assert_eq!(response.status(), 404);
}

#[spin_test]
fn send_get_request_with_invalid_key_id() {
    // Perform the request
    let request = http::types::OutgoingRequest::new(http::types::Headers::new());
    request.set_path_with_query(Some("/user?0")).unwrap();
    let response = spin_test_sdk::perform_request(request);

    // Assert response status and body is 404
    assert_eq!(response.status(), 404);
}

#[spin_test]
fn cache_hit() {
    let user_json = r#"{"id":123,"name":"Ryan"}"#;

    // Configure the app's virtualised 'default' key-value store ready for the test
    let key_value = spin_test_virt::key_value::Store::open("default");
    // Set a specific key with a specific value
    key_value.set("123", user_json.as_bytes());

    // Make the request against the Spin app
    let request = wasi::http::types::OutgoingRequest::new(wasi::http::types::Headers::new());
    request.set_path_with_query(Some("/user?123")).unwrap();
    let response = spin_test_sdk::perform_request(request);

    // Assert the response status and body
    assert_eq!(response.status(), 200);
    let body = response.body_as_string().unwrap();
    assert_eq!(body, user_json);

    // Assert the key-value store was queried
    assert_eq!(
        key_value.calls(),
        vec![spin_test_virt::key_value::Call::Get("123".to_owned())]
    );
}

The following points are intended to unpack the above example for your understanding:

  • Each function marked with #[spin_test] will be run as a separate test.
  • Each test can perform any setup to their environment by using the APIs available in spin_test_sdk::bindings::fermyon::spin_test_virt
  • After requests are made, you can use the APIs in spin_test_sdk::bindings::fermyon::spin_test_virt to make assertions that confirm that the request has been responded to (e.g. response status equals 200) or that expected Spin API calls (e.g. get to the key/value API) have been made.

The tests above run inside of WebAssembly. Calls, such as Key Value storage and retrieval, never actually leave the WebAssembly sandbox. This means your tests are quick and reproducible as you don’t need to rely on running an actual web server, and you don’t need to ensure any of your app’s dependencies are running. Everything your app interacts with is mocked for you. The isolation benefits from this mocking mean that your application’s actual data is never touched. There is also the added benefit of reproducibility whereby you never have to worry about leftover data from previous tests.

Configure spin-test

Before you can run the test, you’ll need to tell spin-test where your test lives and how to build it. You do this from inside our app’s manifest (the spin.toml file). We change back up into our application’s root directory:

$ cd ..

Then we edit the my-app application’s manifest (the spin.toml file) by adding the following block:

[component.my-component.tool.spin-test]
source = "tests/target/wasm32-wasip1/release/tests.wasm"
build = "cargo component build --release"
dir = "tests"

Updating the App to Pass the Tests

This section provides configuration and business logic at the application level.

Configure App Storage

We are using Key Value storage in the application and therefore need to configure the list of allowed key_value_stores in our spin.toml file (in this case we are just using the default). Go ahead and paste the key_value_stores configuration directly inside the [component.my-component] section, as shown below:

[component.my-component]
...
key_value_stores = ["default"]
...

If you would like to learn more about Key Value storage, see this tutorial.

After editing, the whole my-app/spin.toml file will look like the following:

spin_manifest_version = 2

[application]
name = "my-app"
version = "0.1.0"
authors = ["Fermyon Engineering <engineering@fermyon.com>"]
description = ""

[[trigger.http]]
route = "/..."
component = "my-component"

[component.my-component]
source = "my-component/target/wasm32-wasip1/release/my_component.wasm"
allowed_outbound_hosts = []
key_value_stores = ["default"]
[component.my-component.build]
command = "cargo build --target wasm32-wasip1 --release"
workdir = "my-component"
watch = ["src/**/*.rs", "Cargo.toml"]

[component.my-component.tool.spin-test]
source = "tests/target/wasm32-wasip1/release/tests.wasm"
build = "cargo component build --release"
dir = "tests"

Adding Business Logic

The goal of tests is to ensure that the business logic in our application works as intended. We haven’t yet updated our “business logic” frrom the “Hello, Fermyon” app. To make our new tests pass, copy and paste the following code into the application’s source file (located at my-app/my-component/src/lib.rs):

use spin_sdk::{
    http::{IntoResponse, Request, Response, Method},
    http_component,
    key_value::Store,
};

#[http_component]
fn handle_request(req: Request) -> anyhow::Result<impl IntoResponse> {
    // Open the default key-value store
    let store = Store::open_default()?;

    let (status, body) = match *req.method() {
        Method::Get => {
            // Get the value associated with the request URI, or return a 404 if it's not present
            match store.get(req.query())? {
                Some(value) => {
                    println!("Found value for the key {:?}", req.query());
                    (200, Some(value))
                }
                None => {
                    println!("No value found for the key {:?}", req.query());
                    (404, None)
                }
            }
        }
        // No other methods are currently supported
        _ => (405, None),
    };
    Ok(Response::new(status, body))
}

Building and Running the Test

With the application’s configuration and business logic done, we’re ready for our test to be run. We can do this by invoking spin-test from the application’s root directory (my-app):

$ spin build
$ spin test

running 4 tests
No value found for the key "123"
test request-with-invalid-key    ... ok
Found value for the key "123"
test cache-hit                   ... ok
No value found for the key "0"
test request-with-invalid-key-id ... ok
No value found for the key ""
test request-without-key         ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.08s

Next Steps

spin-test is still in the early days of development, so you’re likely to run into things that don’t quite work yet. We’d love to hear about your experience so we can prioritize which features and bugs to fix first. We’re excited about the future potential of using WebAssembly components for testing, and we look forward to hearing about your experiences as we continue the development of spin-test.

You might also like to learn about spin doctor which is a command-line tool that detects and helps fix issues preventing applications from building and running. For more information see the troubleshooting applications page.