Ask a closed model the same question with Compare

For most people trying open models, the real question is whether one is good enough for the task in front of them. Benchmarks give a general answer. Your own question, put to an open model and a closed one, gives a more useful one.

That is what Compare is for. Since August 26, 2026, a finished reply on tryingopen has a small "Compare with a closed model" button under it. Press it, pick Claude Sonnet 5 or GPT-5.6 Terra, and the conversation up to that reply goes to the model you picked. Its answer appears under the open model's reply.

What happens when you press it

The button opens two choices, each with its maker's logo: Claude Sonnet 5 from Anthropic and GPT-5.6 Terra from OpenAI. Until you pick, a line reads "Pick one to see how it answers the same conversation."

The answer streams in underneath, set in the same type as the reply above it, because two answers read as a comparison only when they look alike. As soon as you pick, the row narrows to that model, so you can see who wrote the answer below it. Close the comparison and press the button again to try the other model.

The button appears only on a finished reply with text in it. Under every comparison sits a short note. With Claude it reads: "Closed model from Anthropic, sent this whole conversation and the same web results. No tools of its own, and outside this page's no-retention promise." The part about web results shows only when the open model had some.

Same question, same material

A comparison is worth reading only if both models were asked the same thing. When Compare shipped, the closed model got the text of the chat and nothing more. Since September 1, 2026, it also gets the attachments and web results the open model had.

"Summarize this" means nothing without the PDF it points at, so the pictures and PDFs you attached now travel with their messages (sending pictures and PDFs covers the limits). And when the open model searched the web, the closed model receives the same results. Without them, one side would answer from this morning's web and the other from its training data, and you would be measuring which one had a search tool.

The closed model answers fresh, without seeing the open model's answer. It gets no tools of its own, so it reads the same evidence and has no way to fetch more. Its instructions are the short style prompt the open models also start with, minus the lines about tryingopen and its tools, so both answers begin from the same plain request. How web search works on tryingopen explains where the open side's results come from.

Why the closed models stay out of the picker

We could have put Claude Sonnet 5 and GPT-5.6 Terra in the model picker. We kept them out on purpose.

The picker is a catalogue of open models, and the site makes claims based on it. The page description says "Try 23 open AI models", and the picker tells you how many of its models anyone can download and run (what counts as an open model explains how we count). A closed model in that list would make those statements false.

So the two closed models live apart from the catalogue, and the Compare button is the only way to reach them. Compare is a yardstick: it shows where an open model stands next to a closed one, on the question you actually asked. The catalogue stays open models, and a tour of the picker walks through all 24.

It runs only when you press it

Closed frontier models cost far more per token than the cheaper open models. On OpenRouter's listings as of October 2026, Claude Sonnet 5 costs $10 per million output tokens and GPT-5.6 Terra $12. GLM 5.3 Flash, the default cloud model behind our small model, is listed at $0.50. Asking a closed model on every message, for every visitor, would spend that money on a question most people never asked.

For the same reason, Compare has its own allowance, separate from the chat's. Using up your comparisons leaves ordinary chatting untouched, and the page tells you how many minutes remain until the allowance resets.

Where your conversation goes

Compare is the one feature of the chat that hands your conversation to a closed model's maker. Our privacy policy says:

"Only if you press it, the conversation up to that reply is sent to the closed model's maker (Anthropic or OpenAI) through ngrok's gateway, under that company's own terms."

That puts a comparison outside tryingopen's zero-retention rules. Many messages to open models on the site carry a requirement that the provider delete the message after answering and never train on it (what zero data retention covers explains the details and the exceptions). A comparison carries no such requirement. Anthropic's or OpenAI's own terms decide what happens to it, including any pictures, PDFs and web results that went along.

The note under each comparison repeats this, right where you read the answer. If a chat holds something you wouldn't send to Anthropic or OpenAI, leave the button alone. What tryingopen keeps lists what we hold on our side.

How Compare differs from the side-by-side

Tryingopen has a second comparison feature, the side-by-side, and the two point in opposite directions.

The side-by-side shows how much a 3B model on a palm-sized device can do next to your pick. Compare shows how your pick holds up against a closed frontier model. How the side-by-side compares our small model with your pick has the full story.

Getting a useful answer from it

• Ask the question you really have. Paste the error message you're stuck on, or attach the document you need summarized. No leaderboard can run that comparison for you.
• Try another open model if the first falls short. The "Answer again with a different model" button reruns the message on another open model, and you can press Compare on that reply too.
• Keep what you learn. If the open answer holds up, each cloud reply's "Get the code for this reply" gives you the API request to run it yourself (what each reply tells you).