Tips for keeping Claude on the tracks

[perfectra1n]

Hey nizos, super cool tool, thanks for it. I'm developing in both TypeScript and Rust - but Claude is having a really hard time with maintaining the "knowledge" of piping the test output into tdd-guard (e.g. | tdd-guard-rust --project-root /Users/username/projects/my-app --passthrough) because it seemingly forgets even though I've put it in my initial message, try to steer it to do so during generation, etc.

It eventually just devolves into using sed / echo to replace code in the files to get around the hook's imposed limits.

I've also tried to do the same in TypeScript, and it seems to constantly "forget" to add the required reporter for Vitest, and then just devolves into the same state of using sed/echo.

Also, I'm trying to follow the docs that say to do this to disable tdd-guard :

Simply type one of these commands in your Claude Code prompt:

• tdd-guard on - Enables TDD Guard enforcement
• tdd-guard off - Disables TDD Guard enforcement

and Claude doesn't seem to be too happy about it or understand it:

Do you have any tips on how to enlighten Claude, and/or give it more "steering" so it knows how to provide the required information to the hook so that it passes? Also, how's best to temporarily disable the hook?

I thought maybe I was missing the instructions.md (which I am in fact missing), but how do I go about having it get generated?:

[nizos]

Hi @perfectra1n,

Thanks for reaching out!

Regarding the reporter:
We design the reporters as standalone tools rather than test framework wrappers to ensure user flexibility. The tradeoff however is that for some test frameworks, such as those that don't support a configuration file, the reporter has to be piped in the command.

One way to approach this is to create a simple test command that includes the reporter automatically. For example:

Makefile (with .PHONY):

.PHONY: test
test:
	cargo nextest run 2>&1 | tdd-guard-rust --project-root /absolute/path/to/project --passthrough

Cargo aliases (in .cargo/config.toml):

[alias]
test = "test -- -Z unstable-options --format json 2>&1 | tdd-guard-rust --project-root /absolute/path/to/project --passthrough"

Similar approaches work with Just, Cargo-make, and other build tools.

This way, the agent can run the test command using make test, just test, cargo make test, without having to deal with the full pipeline command and its flags.

Note: the --project-root should always point to your project's root directory, not the location of the Makefile or config file.

You can find more configuration details here: Rust reporter

For typescript, you need to make sure that you've added the reporter correctly

import { defineConfig } from 'vitest/config'
import { VitestReporter } from 'tdd-guard-vitest'
import path from 'path'

export default defineConfig({
  test: {
    reporters: ['default', new VitestReporter(path.resolve(__dirname))],
  },
})

and also have your test command run vitest tests:

{
  "scripts": {
    "test": "vitest run"
  }
}

This applies to both vitest and jest, you can find more information about their configurations here: Vitest reporter and Jest reporter

You can verify if your reporters are working correctly by running npm test, or cargo make test. The test command should create data in a .claude/tdd-guard/data/test.json file.

As for the agent running terminal commands to bypass validation, this usually happens when there is an issue with capturing/reporting the tests correctly. Nonetheless, you can prevent the agent from using those commands using permissions in your claude settings.

For example, in .claude/settings.json in your project root:

{
  "permissions": {
    "deny": [
      "Bash(echo:*)",
      "Bash(printf:*)",
      "Bash(sed:*)",
      "Bash(awk:*)",
      "Bash(perl:*)"
    ]
  }
}

You can read more about it here: Strengthening enforcement

Regarding the tdd-guard on/off commands, the issue seems to be that they are not being captured by tdd-guard and as such they are passed on to the agent. Adding the hook resolves this, you can find the instructions here: Quick Commands

Regarding the instructions.md file, this gets automatically created in .claude/tdd-guard/data/ when you have the SessionStart hook configured. See Session Clearing for setup and Custom Instructions for customization details.

Let me know if this solves the issues you're having or if there is anything else I can help with.

Thank you for bringing this to my attention, I realize now that I need to simplify/unify the configuration process so that it is easier to follow. It is now next on my list.

[rhokstar]

Yep... Claude Code is using sed to defeat tdd-guard. I'm quite upset about it because Anthropic advocates for TDD but Claude is essentially doing the opposite. I might as well uninstall and not do TDD any more until Anthropic can fix the problem.

[nizos]

Hi @rhokstar,

I understand your frustration. You can completely block Claude Code from using sed and any other terminal commands by denying it access to them. You can find the instructions on tdd-guard Strengthening enforcement guide and on Claude Code's settings documentation.

[rhokstar]

@nizos

Thanks. I've implemented this and now its using cat and other bash commands. I've implemented OS level file protections (write protecting permissions like chmod 444) but this adds more manual state changes I have to turn off/on in my dev environment.

I think I might just quit TDD for now because I'm spending too much time fighting a tool / AI adversary rather than being productive.

[nizos]

Hi @rhokstar,

No worries! Are you sure the issue is not that the test results aren't reported correctly to tdd-guard? This usually happens when the validation agent can't see any test results. Which programming language and framework are you using? I'd gladly resolve this issue if you have time to share more details. Thanks for reporting this!

[rhokstar]

I'm on Python/FastAPI. The problem is not your solution, its Anthropic.

I've had your solution in mind for quite sometime and decided to use this yesterday when I started to test Github's new spec-kit. When I got to the TDD portion of spec-kit, without tdd-guard Claude tries to be good but it defeats itself and breaks TDD. It even catches itself violating TDD and then succumbs to the dark side... laying wanton code destruction like an unchained beast!

And then, Claude starts installing unrelated libraries to bypass tdd-guard protections which I've had to intervene! I wouldn't be surprised if Claude started using exploits to bypass chmod 444 😒.

[perfectra1n]

Hey nizos,

1. Understood about the reporters, my issues were not with the reporters - it was with Claude just simply ignoring the instruction to use them (especially over time as the conversation continues). I was just curious if you know of any ways to customize a prompt, or maybe even the prompt that tdd-guard responds with to gently remind it to use what command/code needed to have the reporter be used?

2. Regarding tdd-guard on/off, I understand that the hook is required for it to work. However, my issues happen when the hook is already configured:

WindowsTerminal_TzXF76BMBz.mp4

3. Understood about the SessionClearing I wasn't aware that was a required part of the setup. I'll open a PR for the README.md (feat(readme): update readme to include session clearing setup step #63) :)

[nizos]

Hi @perfectra1n,

My bad! Sorry for the misunderstanding! :)

Just to be sure, do you have this hook configured?

{
  "hooks": {
    "userpromptsubmit": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "tdd-guard"
          }
        ]
      }
    ]
  }
}

Did you also try to restart the session? :)

I really appreciate the PR and I welcome contributions but I think the configuration guide as a whole needs to be revised. Thank you though! :)