Prompt Engineering & Fine-Tuning

Prompts and model tunes measured against a test set, not a demo.

What it is

Marketing copy in a house style, legal summaries that keep the defined terms, ticket classification, field extraction from invoices: tasks where the model’s exact behaviour matters and “usually right” is not good enough. GTDataWorks writes and tests prompts for GPT, Claude and Gemini against a held-out test set with deterministic grading, so the result is a before-and-after score rather than a convincing demo on three examples.

Your users, or your code, get consistent output: the same structure every time, the edge cases handled the way the test set says they should be, and a versioned prompt pack that drops into your application through a short integration snippet. Where prompting alone cannot reach the target, the studio fine-tunes an open model on your data with LoRA and measures it against the same test set.

What you get

  • Versioned prompt pack
  • Test set and grading script
  • Score report
  • Integration snippet

A score, not a demo

How the work runs

The build steps once the quote is agreed. The brief, scoping call and quote are the same for every service.

  1. Define correct

    Agree what a correct output is for this task: the format, the hard constraints, and 5 to 10 examples of acceptable and unacceptable output, written down.

  2. Build the test set

    Collect 50 to 200 real inputs with reference outputs. Split them into a development set used for writing prompts and a held-out set that is not looked at until the end.

  3. Baseline score

    Run your current prompt, or a plain one, against the held-out set with the deterministic grader. This is the before number, and it goes in the report unchanged.

  4. Iterate on the development set

    Prompt variants, few-shot examples, structured output, model choice; each variant is scored and kept in the pack’s version history. If the ceiling is too low, a LoRA fine-tune of an open model on your data.

  5. Final score and handover

    One run on the held-out set becomes the score report. You receive the prompt pack, the test set, the grading script and an integration snippet in your language.

Pricing

Fixed price

$500–$2,000

Per prompt pack or tuning project, including test set and score report.

Hourly

$80–$150/hr

Custom development and ongoing iteration.

Prices are in USD and are starting points. The final quote follows a scoping call.

Where it fits

A good fit when

  • The task is a transformation of text you can define: classify, extract, summarize, rewrite in a style.
  • You can supply, or help assemble, 50 or more real examples with known-good outputs.
  • The output goes into a system or in front of customers, so format and consistency matter.

Not a fit when

Questions

How is accuracy measured?

Deterministically wherever possible: exact match, schema validation, and rubric checks a script can run the same way every time. Where a human judgement is unavoidable, tone for instance, a short written rubric and a sampled human review are recorded next to the automated score. The grading script ships with the pack, so you can reproduce the number yourself.

Which models does this cover?

Prompting: GPT, Claude and Gemini, through your own API keys. Fine-tuning: open-weight models from the Hugging Face hub with LoRA through PEFT, trained on hardware you choose. The tuned weights are yours. Closed models are prompted, not tuned.

Is my data used to train anything?

Only when you ask for a fine-tune, and then only your model: the training data and the resulting weights stay yours. Prompt work uses your examples in the test set and nowhere else. Nothing is pooled across clients, and model calls run on your keys under your agreement with the provider.

What happens when the provider updates the model?

Scores can move. The test set and grading script exist so that you can re-run them against the new model version in minutes and see whether the prompt still holds. If it does not, the hourly tier covers the fix, and the versioned pack shows what changed.

Bring the task and ten examples

Ten real inputs with the output you wanted, and a sentence on what counts as wrong, are enough to scope the work. The test set is built from there.