[SOLVED!] My Text-Based Adventure Benchmark Cracked by Claude Opus 5

And when Alexander saw the breadth of his domain, he wept, for there were no more worlds to conquer

First, here are my previous articles on this subject:

You’re Standing in a Clearing in a Forest (Apr 10, 2026)

Text Adventure Benchmarks Revisited (Jun 14, 2026)

Testing Fable 5 on Text Adventure Games (Jul 5, 2026)

And a reminder of the domain. This is a small custom text-based adventure I created from scratch as a personal benchmark to run new models of LLMs against. It’s 10 rooms total and the goal is to collect 3 keys (brass, silver, and gold) and use them correctly on the final door in Room 3 to exit the dungeon.

The first and most challenging central puzzle is a rotating room (r5) operated by a crank mechanism in r4. The player must first find the handle to the crank in r2, carry it to r4, insert it, and turn it to align openings between r5 and its adjacent rooms.

The most difficult aspect of this puzzle seemed to be non-local causal reasoning combined with allocentric coordinates. The crank is two rooms away from the rotating room that it actually turns. When the player turns the crank a grinding sound nearby can be heard through the walls. There is an informational diagram on the wall in the same room as the wall, and it updates with each turn. Earlier models struggled to understand that the diagram was information, and not the final cause of cranking.

These types of puzzles require the player to fiddle with something in one place, then explore to see if something has changed elsewhere. I honestly thought this was going to be a blocker for a good long while.

Another difficulty with the puzzle is that the slots in the rotating room need to line up with gaps in adjacent rooms. E.g. the S slot of the rotating room has to align with the N gap in the foyer (r1) for the passage to be open. The player could just exhaustively experiment, but to be efficient, they’d need to see the puzzle from the perspective of the rotating room. Earlier models very often seemed to understand the spatial locations of r1 and r5, but still conclude that the gap in r5 and the slot in r1 needed to both be N in order to align. The latest agent figured this out once it had gathered the relevant information (more on this later).

Beyond the rotating room, there are three puzzles to attain each key. The brass key requires extinguishing your torch in order to reveal the container for the key outlined in glowing moss only visible in the dark. Riddles give hints to the user.

The silver key requires using a magnifying glass (found in r6) and concentrating light onto a panel to open it.

The gold key is attained by ringing a bell near a secret compartment.

All three keys must then be carried to r3 and used to unlock 3 locks.

I started in April, four months ago. I implemented an agentic harness and refined it since that time, adding:

• A map tool, so the agent can generate, update, and reference a map as they play.

• A hypothesis generation and update step. Each step, prior to taking an action, the agent is forced to reflect on what actions will gain the most information to help update any of their current hypotheses. After acting, they may create a new hypothesis or update an existing one (revise, discard, or validate).

• A causal map with nodes and edges, updated after each step and referenced prior to each action.

• Action history and world updates logged and accessible in context.

Until this morning, the domain remained unsolved. Last night I made a couple of tweaks to the harness and let it run. This morning Claude Opus 5 was the first model used with my agent harness that completed the full dungeon. Fable 5 had previously worked out the rotating room functionality 2 out of 3 trials, but was still unable to solve the whole thing.

Now, a little caveat. The way it got to the solution for the rotating room was a little weird. First it rotated the crank so that the diagram in r4 showed E as green. It had formed the hypothesis that the crank turned the foyer (r1). So it then moved E to check this. Finding nothing changed, it determined that the crank did not move the foyer arches.

It then formed a new hypothesis, that the crank somehow opened the gold lock leading from r2 to r3 when the diagram showed S as green. So it turned the crank, and went E again to check. As a secondary check, it also wrote it its notes that going E "also re-checks whether the north arch changed." So it moved to r2, saw that the gold lock remained unchanged, invalidating that hypothesis, then moved back to r1, and at that point it noticed the N passage open. Everything clicked, and it now updated its causal graph to correctly indicate how the crank affected the rotating room.

It used 76 turns to solve the whole dungeon. 48 is perfect execution. Here’s a breakdown of turns used for each puzzle:

The biggest struggle was how to use the magnifying glass correctly. It formed three incorrect hypotheses, had 7 failed turns at the puzzle, and at one point put down the lantern (which is the light source it needed to use along with the magnifying glass to solve the puzzle). It almost left the lantern behind, but went back for it, corrected, and solved the puzzle.

To me this behavior all sounds very familiar and very human. When I played these games, I almost never solved a given puzzle in a single shot. I flailed around. I formed different hypotheses and tried to validate or disconfirm them. This is the kind of behavior we would want to see out of an agent that we want to be able to explore the world, learn new things, and solve problems.

I would have been suspicious of a one-shot solution. But the first thing I did after the result was a contamination audit, making sure hints or solutions weren’t leaking between parts of the system. After a clean audit, I did a deep dive to analyze the behavior. Between the audit and the flailing→revising→solution loop for the magnifying glass puzzle, I’m satisfied this is a solid result.

I’m also simultaneously excited and bummed. I’m bummed because I thought I had my own benchmark in hand that would continue to allow me to test models over at least the medium term. When the agents failed the past, they seemed so clueless that I figured this domain would continue to thwart them for a good long while. Well, that was wrong.

I also experienced the common failure mode of unproductive looping. The crank in particular is the type of puzzle where an agent can burn a lot of time and energy spinning their wheels. I saw this quite a bit in previous trials. Not sure if the labs are using reinforcement learning to curtail this specific behavior, but as the models improved, they were less likely to chase their own tails.

So this little benchmark is saturated. I could do a number of follow-ups, but I don’t really see the point. I could expand the dungeon. I could randomize the paths. I thought of this because I was mildly worried about contamination (which is why I did a custom dungeon to begin with). But I think the basic capabilities that I set out to test are now demonstrated in a way that satisfies me. I don’t see much reason for further tinkering in the same space.

Between this and my recent little probe into action sequencing being saturated by Fable 5, I don’t have any live, unsolved benchmarks. Not sure where I want to go next.

It’s a very odd feeling creating a problem that these systems can’t solve, then within a very short period of time, watching them knock it down. But this is the future. To anyone who thinks we’ve hit a brick wall of some kind or diminishing returns, I’d have to say I certainly am not experiencing that based on the evidence I have.

If I do decide to try to implement a new probe or benchmark, you’ll be the first to know about it here. Until then, I’m closing up shop on these particular tests. Let me know your thoughts in the comments.