Reasoningby LawtradesExplore projects Briefings
Menu

HOW IT WORKS

What you’ll hand back.

Projects can involve writing tasks and answers, editing documents or reviewing scoring criteria.

Mia McClenaghan

LEGAL ENGINEER AT LAWTRADES

Mia McClenaghan: from legal work to legal AI

Mia began as a paralegal in restructuring at Kirkland & Ellis before moving into real estate. She later trained as a software engineer and worked on CoCounsel at Thomson Reuters.

“Legal AI training appealed to me because it brings those two sides together.”

The assignment

In her fictional lease-review task, an AI must compare a draft lease with agreed heads of terms. One definition removes six months of rent-free time. Correcting the rent commencement date alone would also shift the tenant’s break date. A strong answer must restore the rent-free period and preserve the agreed break date.

The review

Mia received no revision requests for this task. She refined the scoring criteria so that each failure reflected a meaningful error.

Mia’s advice

“My advice to anyone considering Reasoning or this type of work is to really draw on your own experience: build scenarios around work you know well, and think about real mistakes you have made yourself or seen others make. You will often be able to replicate them with the models!”
Explore current projects
Task development
A legal task, supporting sources, a reference answer and scoring criteria.
Document editing
Edited fictional legal documents in Word, with an explanation of the changes.
Criteria review
Reviewed and refined scoring criteria for a supplied legal task and its sources.
01

Read the assignment.

Follow a fictional assignment from these three source records to the revised answer.

ILLUSTRATIVE TASKDocument analysis

Does this record establish that all requested messages were preserved?

  1. 01

    Preservation request

    4 May

    Preserve business messages from 1 March onward, including messages on personal devices.

  2. 02

    Admin log

    9 May

    Automatic message deletion disabled.

  3. 03

    Collection inventory

    10 May

    Export covers 1 April–10 May. Personal devices are not included.

02 · TASK DEVELOPMENT

Explain the answer.

A reference answer shows what the sources support, where they conflict and what remains unknown.

ANSWER UNDER REVIEW

“All requested messages were preserved, as confirmed by the preservation request.”
REVIEW FEEDBACK

A request states what should be preserved. It does not establish what was kept. Account for the five-day delay, the missing March coverage and the excluded personal devices before drawing a conclusion.

FINAL REFERENCE ANSWER

Complete preservation is not established.

The request was issued on 4 May, but automatic deletion was not disabled until 9 May [01, 02]. The export begins on 1 April, one month after the requested start date, and excludes personal devices [01, 03].

Check whether March and personal-device messages were preserved elsewhere, and whether messages were deleted before 9 May. These records do not establish complete preservation or prove that messages were lost.

Sources 01–03
03

Make the criteria precise.

Evaluation criteria need to distinguish a convincing answer from a correct one. State what the response must get right.

TOO BROAD

Mentions the preservation request.

REVISED CRITERION

Explains why the request is not proof of preservation.

  • Identifies the timing, date-range and device gaps.
  • Supports the conclusion with the relevant records.
  • Does not mistake missing evidence for proof of loss.
04

Check the finished work.

Here’s how the revised answer meets the criteria. Each judgment needs a reason.

  1. 01

    Separate the request from the action.

    The answer distinguishes the 4 May request from the 9 May change [01, 02].

  2. 02

    Identify what the collection leaves out.

    It names the March and personal-device gaps [01, 03].

  3. 03

    Keep the conclusion within the evidence.

    It asks for further checks without assuming either complete preservation or data loss.

Actual submissions follow the project’s instructions and may need further revision before acceptance.

What to expect as an expert