Find Tfcompat Bug Skill for Claude

Find & prove the next tfcompat bug

Hunt one genuine runtime divergence between OpenTofu (tofu apply) and pulumi-hcl, and

prove it with a failing tfcompat test. That is the whole job of this skill: find a

real divergence and leave behind a test that fails on master for the right reason. You

stop at the proven failure — you do not fix it. To turn a proven failure into a

shipped fix, hand off to the fix-tfcompat-bug skill.

The divergence can be anywhere the two runtimes can disagree — an expression/function

result (the L1 seam), or resource, module, provider, or lifecycle behavior (the L2

seam). Functions are the easiest place to start, not the only place to look.

Direction matters — only one direction is a bug. A bug is an input that **works in

OpenTofu but does not work (or works differently) in pulumi-hcl**: OpenTofu produces a

value and pulumi-hcl errors or returns something else. That is the case a user migrating

real Terraform/OpenTofu configuration into pulumi-hcl actually hits. The reverse — an

input pulumi-hcl accepts that OpenTofu rejects (pulumi-hcl is more lenient) — is not a

bug to chase here: no OpenTofu config depends on it, and tightening pulumi-hcl to also

reject it removes capability without helping any migration. If the only divergence you can

find is pulumi-hcl being more permissive than OpenTofu, discard it and keep hunting.

Each invocation produces exactly one proven failing test. Don't bundle several.

Ground rules (from CLAUDE.md — do not violate)

• Confirm assumptions with research. Don't guess. Verify every behavioral claim

against OpenTofu source, the cty stdlib source, the docs, or an empirical tofu

probe — never from memory. A backwards memory of token semantics has already burned

us once.

• Faithfully match OpenTofu. The goal is parity, not "better". No hacks.
• The deliverable is new test files only — you ADD tests/tfcompat/ files. Don't edit

runtime source under pkg/ (that's the fixing skill's job), and don't touch staging the

user left.

Source-of-truth locations

The Go module cache root is !go env GOPATH/pkg/mod.

• OpenTofu funcs: !go env GOPATH/pkg/mod/github.com/pulumi/opentofu@*/lang/funcs/*.go
• OpenTofu binding map: .../opentofu@*/lang/functions.go
• OpenTofu docs: .../opentofu@*/website/docs/language/functions/*.mdx
• cty stdlib: !go env GOPATH/pkg/mod/github.com/zclconf/go-cty@*/cty/function/stdlib/
• pulumi-hcl runtime: pkg/hcl/ (expression eval, resource / module / provider handling)
• pulumi-hcl function registry: pkg/hcl/eval/functions.go

(func Functions(baseDir string) at the top binds every name to its impl)

• Function unit tests: pkg/hcl/eval/functions_test.go
• tfcompat harness + Case type: tests/testutil/tfcompat/harness.go
• Empirical probe: use the tofu on your PATH (!which tofu)

Step 1 — Find a candidate divergence

Look for any input the two runtimes can evaluate differently. Two seams, easiest first:

• L1 — expression / function: a function in pkg/hcl/eval/functions.go with a

hand-rolled Impl rather than a delegated stdlib.*Func. Hand-rolled impls are

where function divergences hide. Diff against the OpenTofu impl/docs, and where cheap,

probe both sides:

  # OpenTofu side — write a tiny main.tf and apply
  echo 'output "x" { value = <expr> }' > /tmp/probe/main.tf
  (cd /tmp/probe && tofu apply -auto-approve)

• L2 — resource / module / lifecycle: behavior around resources, modules,

providers, count/for_each, pre/postconditions, or the preview / destroy / replace

lifecycle. These exercise a provider (e.g. providers.SimpleProvider) — compare a

small program end-to-end through apply, and where relevant preview and destroy.

Good hunting grounds: encoding/escaping funcs, numeric edge cases, empty-collection

errors, date/time token tables, string normalization, unknowns in preview,

precondition/postcondition error messages, replace/destroy lifecycle, module/provider

passthrough. Look for: wrong error on edge input, precision loss, off-by-one token

mapping, charset handling, escaping rules, a field that diverges only in state, an

operation that succeeds on one path and errors on the other.

Keep the direction in mind (see the intro): the case you want is one where **OpenTofu

succeeds and pulumi-hcl errors or returns a different value**. When an `operation that

succeeds on one path and errors on the other` turns up, check which side errors — if

pulumi-hcl is the one that succeeds and OpenTofu errors, that is pulumi-hcl being more

lenient, which is not the bug you are looking for. Discard it and keep hunting.

Observability caveat: pulumi serializes stack-output numbers as float64. Integer

precision past ~16 significant digits is lost on BOTH paths, so it cannot be exposed via

output comparison. Don't build a case whose only difference is unobservable in outputs —

reach for AssertState/ExpectErr (below), or prove it with a unit test, or pick a

different bug.

Step 2 — Prove it with a failing test

Name the case l1_<name> for an expression/function bug or l2_<name> for a

resource/module/lifecycle bug, and create:

• tests/tfcompat/<l1|l2>_<name>_test.go
• tests/tfcompat/testdata/cases/<l1|l2>_<name>/main.tf

Existing cases (pick a fresh name):

!ls tests/tfcompat/testdata/cases

The test (copy the Apache header from any sibling _test.go):

// L1 — pure expression, no providers
func TestL1<Name>(t *testing.T) {
	t.Parallel()
	tfcompat.RunCase(t, "l1_<name>", tfcompat.Case{})
}

// L2 — register the provider(s) the program uses
func TestL2<Name>(t *testing.T) {
	t.Parallel()
	tfcompat.RunCase(t, "l2_<name>", tfcompat.Case{
		Providers: []tfcompat.Provider{
			{Name: "simple", Factory: providers.SimpleProvider},
		},
	})
}

RunCase loads every file under testdata/cases/<name>/ and runs it through both

tofu apply and pulumi up, asserting identical stack outputs. For an L1 case the

fixture is pure locals + output (one output per behavior). **Put the fixture on disk

under testdata/cases/<name>/ and pass an empty tfcompat.Case{} — this is the default,

including multi-file cases** (e.g. a templatefile case ships its main.tf *and* the

template it reads; RunCase loads every file in the directory). The Case struct gives

you more knobs when outputs alone can't show the divergence:

• Providers / Config — register TF providers / set input variables. Config is the

only channel for values not known until runtime (ports, temp paths): declare a variable

in the fixture and set it here — never generate program text in Go.

• AssertState — assert on resource fields not reachable via outputs (e.g. Protect).
• Stages — per-stage behavior matched positionally against the disk fixture: Mode

(StageApply / StagePreview / StageDestroy), ExpectErr (require BOTH runtimes to

fail with a matching error substring — the way to prove an error-behavior divergence),

and AssertOutput. A Stage carries behavior only; program files always come from the

case directory. A flat case dir with N Stages entries runs the same program N times

(e.g. preview then apply); a program that must *change* between stages uses numbered

subdirs testdata/cases/<name>/0/, 1/, ... with one Stages entry per subdir (or

none).

Confirm the bug is real on master:

PATH="$PWD/bin:$PATH" go test ./tests/tfcompat/ -run 'Test(L1|L2)<Name>' -count=1 -v

It must FAIL, and the failure must show the genuine OpenTofu-vs-pulumi difference (a real

output diff or a matched ExpectErr), not a harness/provider/fixture error. If it passes,

the bug isn't real (or isn't observable) — go back to Step 1.

Done — hand off

You now have a proven failing tfcompat test, left uncommitted in the working tree.

Report:

• The divergence — one line: what OpenTofu does vs. what pulumi-hcl does.
• The test — name + the file paths you created.
• The failure output — the captured diff/error proving it fails for the right reason.

To fix the divergence and ship it, continue with the

fix-tfcompat-bug skill. Do not fix it here.