Fine-tuning dataset validator

What it does

Open a JSON Lines training file, or paste a few lines, and choose the provider's format. The tool checks every line against what that provider's own material says a training example must be, and lists what is wrong, grouped by kind, with the line it is on. It also:

• finds examples that are empty, and examples that repeat an earlier one;
• counts each example's tokens, shows how they are spread, and flags examples longer than a limit you set;
• estimates the training cost, but only where the provider's pricing page states a training price;
• can write a copy of the file with the repeated examples left out.

The file never leaves your browser. It is read in a background thread as it streams, one line at a time, so a file of hundreds of megabytes is checked without freezing the page and without being held in memory. A shared link holds the settings, never the lines.

Salyro does not offer fine-tuning. This tool checks files for the providers' own fine-tuning services.

Try it: an OpenAI file with one broken line of each kind, a valid one with a tool call and weights, Gemini on Vertex AI, with a training price, or Claude in Amazon Bedrock with turns out of order.

What it checks

An error means the provider's own material states the rule. A warning means no provider page read here states it: it is likely a mistake, but the tool does not claim the provider will refuse it.

OpenAI, supervised, chat format. Each line is {"messages": [...]}, with optional tools. A message's role is system, user, assistant or tool. Every message carries text, or for an assistant, a tool call. Every example needs at least one assistant message. weight on an assistant message is 0 or 1. A file needs at least 10 examples, and may be up to 1 GB. Content may be text or a list of text and image_url parts.

OpenAI, preference (DPO). Each line is {"input": {"messages": [...]}, "preferred_output": [...], "non_preferred_output": [...]}, each output holding the assistant's reply. A pair whose two replies are the same is a warning: there is nothing to prefer.

Gemini supervised tuning on Vertex AI. Each line is {"contents": [...]}, with an optional systemInstruction. A turn's role is user or model, and its content is in parts. Every example needs a model turn, the output it learns. A dataset needs at least 100 examples.

Claude fine-tuning in Amazon Bedrock. Each line is {"system": "...", "messages": [...]}, where system is optional. There are at least two messages; the first is from the user and the last from the assistant; user and assistant alternate; and no other key is allowed.

For every format it also reports a line that is not JSON or not an object, a blank line, an empty example, and a repeated one. A single line of more than 16 MiB is not read at all: it is reported as not checked, and not counted as valid. Where an example carries tool calls or tool results, it runs the conversation linter's rules too — a tool result that answers no call, two calls with one id. Those rules are about the providers' request APIs, and no fine-tuning guide states them, so they are warnings here. A tool call in an example's last message is never reported as missing its result: an example that teaches the model to call a tool ends exactly there.

The findings are grouped by kind. Each group gives its count and lists its first lines, with "and N more" for the rest, so a file with a million broken lines is still one short list.

Tokens

For the OpenAI formats, tokens are counted exactly with o200k_base or cl100k_base, OpenAI's own encodings, running in your browser. Anthropic and Google count tokens through their own APIs, so for their formats the count is an estimate, at about four characters a token, and says so.

What is counted is the text of every message and tool call. Providers also add a few tokens around each message, and tool definitions count too; neither is included, so the real count is at least what is shown. A single run of hundreds of characters with no space in it — a long hash, a wall of one letter — is estimated rather than counted exactly, because exact counting slows down sharply on it, and the page says how many examples that applied to.

Epochs. The epochs field goes up to 50, OpenAI's stated maximum. For Vertex AI no source read here states a limit, so there 50 is only this page's bound.

Length limits. A provider limits how long a training example may be, and the limit depends on the model. The pages that state the current limits could not be read, so the tool applies none: set your own under "Flag examples over", and it is reported as a warning, as your limit.

Training cost

Training is billed on the dataset's tokens times the number of epochs — OpenAI's guide and Vertex AI's pricing page both say so. The cost is shown only where a training price was read off the provider's own page:

• Gemini on Vertex AI: priced, per base model, from Vertex AI's pricing page. The tokens are estimated, so the cost is an estimate too.
• OpenAI: unknown. Its pricing page could not be read, so no figure is shown.
• Claude in Amazon Bedrock: unknown. Training is billed by Amazon Bedrock, whose pricing page could not be read.

The arithmetic is Salyro's cost engine, the same function it prices usage with, applied to the stated rate.

What it does not know

• The current token limits per example, for any provider: the pages that state them could not be read. None is guessed.
• OpenAI's and Amazon Bedrock's training prices. Shown as unknown.
• xAI. No fine-tuning dataset format could be read for xAI — its documentation could not be reached, and its SDK has no fine-tuning call — so none is checked.
• Everything a provider checks on upload. The tool checks what the providers' material states. A provider may refuse a file for a reason its material does not give, and may accept a file with a warning here.
• Images. An image part is checked for its shape, not opened or measured.

FAQ

Is my file uploaded anywhere?

No. It is read in your browser, in a background thread, and nothing is sent. A shared link holds only the settings.

Why is OpenAI's documentation not the source?

OpenAI's, Google's and xAI's documentation sites could not be reached when these facts were compiled. Where the provider keeps the same material in a repository of its own on GitHub — OpenAI's cookbook and OpenAPI description, Google's generative-ai samples, Anthropic's cookbook — the facts were read from there, and each fact names where.

Does the copy without repeats change anything else?

No. It leaves out every example that repeats an earlier one, with its line ending, and keeps the first. Every other line is the file's own bytes: blank lines, CRLF line endings and anything that is not valid UTF-8 stay as they were. Examples are compared after parsing, so two lines that differ only in key order or spacing are the same example; a line with bytes that are not valid UTF-8 is compared by its exact bytes. A local hash only finds the candidates: an example is called a repeat after the earlier one is read back from your file and compared with it, so two different examples are never taken for each other. A line too long to read is kept in the copy as it is, unchecked. Most repeats are compared in memory; those whose earlier line is long are confirmed once the file has been read, by reading the earlier lines back in file order.

Sources

Every rule and price is recorded as Reference Data with the page it was read from and the date, and listed in the data freshness report: OpenAI's fine-tuning guide, data-preparation, DPO and vision fine-tuning notebooks in its cookbook, and its OpenAPI description; Google's Vertex AI tuning notebooks and its Vertex AI pricing page; Anthropic's Bedrock fine-tuning notebook and its announcement of fine-tuning in Amazon Bedrock. The token counts use OpenAI's encodings, held to OpenAI's own tiktoken.