Write tests using the Jaribu framework for .NET 10 file-based apps (runfiles)

Maintained in timewarp-jaribu · Canonical file: SKILL.md

Install

npx skills add TimeWarpEngineering/timewarp-jaribu --skill jaribu

Or copy the SKILL.md into your agent's skills directory.

Jaribu Testing Skill

Jaribu is an attribute-based testing framework for .NET 10 runfiles. Tests are standalone executables.

Architecture

Jaribu uses a sink-based architecture where all test output flows through ITestResultSink:

TestRunner.RunTestsAsync(sink)
    ├── TerminalSink → Console output (dotnet run single-file)
    ├── MtpSink → IMessageBus → VS/Rider/dotnet test
    └── NullSink → Silent (testing/benchmarking)

Core Types

Test File Template (Multi-Mode Compatible)

#!/usr/bin/dotnet --

#if !JARIBU_MULTI
return await RunAllTests();
#endif

namespace Your.Namespace.Here
{

[TestTag("CategoryName")]
public class YourTestClassName
{
    [ModuleInitializer]
    internal static void Register() => RegisterTests<YourTestClassName>();

    public static async Task Should_describe_expected_behavior()
    {
        // Arrange
        // Act
        // Assert
        await Task.CompletedTask;
    }
}

} // namespace Your.Namespace.Here

Test Methods

• Must be public static async Task
• Follow the naming convention below

Test Naming Convention

Use a hierarchical SUT_Action_Given_Should_Result pattern:

Full qualified test name: ShellBuilder_.CaptureAsync_Given_.EchoCommand_Should_CaptureStdout

Example Structure

namespace ShellBuilder_
{
  [TestTag("Core")]
  public class CaptureAsync_Given_
  {
    [ModuleInitializer]
    internal static void Register() => RegisterTests<CaptureAsync_Given_>();

    public static async Task EchoCommand_Should_CaptureStdout()
    {
      CommandOutput output = await Shell.Builder("echo").WithArguments("Hello").CaptureAsync();
      output.Stdout.Trim().ShouldBe("Hello");
    }

    public static async Task MultipleArgs_Should_PassAllArgs()
    {
      CommandOutput output = await Shell.Builder("echo").WithArguments("a", "b").CaptureAsync();
      output.Stdout.Trim().ShouldBe("a b");
    }
  }

  public class WithArguments_Given_
  {
    [ModuleInitializer]
    internal static void Register() => RegisterTests<WithArguments_Given_>();

    public static async Task EmptyArray_Should_ExecuteWithNoArgs()
    {
      // ...
    }
  }
}

Why This Convention?

• Grouping: Test explorers group by namespace/class, so related tests cluster together
• Readability: Full name reads as a sentence: "ShellBuilder CaptureAsync Given EchoCommand Should CaptureStdout"
• DRY: SUT and Action aren't repeated in every method name
• Discoverable: Easy to find all tests for a specific class or method

Setup and CleanUp

Called automatically before/after EACH test:

public static async Task Setup()
{
    // Called before EACH test
    await Task.CompletedTask;
}

public static async Task CleanUp()
{
    // Called after EACH test (even if test fails)
    await Task.CompletedTask;
}

Data-Driven Tests

[Input("value1")]
[Input("value2")]
[Input("value3")]
public static async Task Should_handle_input(string input)
{
    // Runs once per [Input] attribute
    await Task.CompletedTask;
}

Attributes

Shouldly Assertions

result.ShouldBe(expected);
result.ShouldNotBe(unexpected);
result.ShouldBeNull();
result.ShouldNotBeNull();
flag.ShouldBeTrue();
flag.ShouldBeFalse();
count.ShouldBeGreaterThan(0);
text.ShouldContain("substring");
text.ShouldStartWith("prefix");
text.ShouldBeEmpty();
list.ShouldContain(item);
Should.Throw<Exception>(() => ThrowingMethod());

File Naming

Format: {sut}.{action}.cs (kebab-case)

This aligns with the naming convention - one file per Action keeps files small and token-efficient.

Examples:

shell-builder.capture-async.cs    # ShellBuilder_.CaptureAsync_Given_.*
shell-builder.to-command-string.cs # ShellBuilder_.ToCommandString_Given_.*
shell-builder.pipe.cs             # ShellBuilder_.Pipe_Given_.*
shell-builder.run-async.cs        # ShellBuilder_.RunAsync_Given_.*

Why one file per Action?

• Token efficiency: Small, focused files reduce context when AI reads them
• Organization: Easy to find tests for specific functionality
• Maintainability: Changes to one action don't require reading unrelated tests

Running Tests

# Make executable and run (Runfile Mode)
chmod +x my-test.cs
./my-test.cs

# Or with dotnet
dotnet run my-test.cs

# Filter by tag
JARIBU_FILTER_TAG=Lexer ./my-test.cs

# M.T.P. Mode (IDE integration, dotnet test)
dotnet test

# List discovered tests
dotnet run -- --list-tests

Sink-Based API

// Simple usage - returns exit code (backward compatible)
int exitCode = await TestRunner.RunTests<MyTests>();

// Sink-based - get stats with console output
using TerminalSink sink = new();
TestRunStats stats = await TestRunner.RunTestsAsync<MyTests>(sink);

// Silent execution - for programmatic use
TestRunStats stats = await TestRunner.RunTestsAsync<MyTests>(NullSink.Instance);

// Non-generic - for reflection-based execution
TestRunStats stats = await TestRunner.RunTestsAsync(typeof(MyTests), sink);

Directory.Build.props Dependencies

<ItemGroup>
  <PackageReference Include="TimeWarp.Jaribu" />
  <PackageReference Include="Shouldly" />
</ItemGroup>

<ItemGroup>
  <Using Include="Shouldly" />
  <Using Include="System.Console" Static="true" />
  <Using Include="TimeWarp.Jaribu" />
  <Using Include="TimeWarp.Jaribu.TestRunner" Static="true" />
</ItemGroup>

Temporary Test Files

• Do NOT write to /tmp directory
• Use local tests/ directory in the project
• Prefix temporary files with temp- (e.g., temp-test-which.cs)

Testing Philosophy

Tests exist to expose bugs. A failing test is doing its job.

NEVER:

• Skip a failing test to make CI green - that hides bugs
• Work around test failures - fix the actual bug
• Delete tests because they fail - they're telling you something is broken

When a test fails:

1. Keep the test - it documents expected behavior
2. Create a kanban task - track the bug it exposed
3. Fix the bug - in the source code, not the test

Valid reasons to skip:

• Test requires unavailable infrastructure
• Feature intentionally not yet implemented (mark with issue #)
• Flaky due to timing - but prefer fixing the flakiness

Best Practices

DO:

• Use descriptive Should_ names
• Follow Arrange-Act-Assert pattern
• One behavior per test
• Use Setup/CleanUp for per-test state
• Use static initializers for expensive one-time setup

DON'T:

• Use xUnit/NUnit/MSTest
• Make tests depend on each other
• Put expensive operations in Setup (runs per test)