Skip to content

Commit 30dce61

Browse files
scosmanscosman
authored
Merge pull request #295 from Kiln-AI/backload_requirements
Usability update: don't ask for requirements when first getting setup.
2 parents 0436e78 + 33b2637 commit 30dce61

File tree

2 files changed

+94
-106
lines changed

2 files changed

+94
-106
lines changed

‎app/web_ui/src/routes/(fullscreen)/setup/(setup)/create_task/edit_task.svelte

Lines changed: 93 additions & 105 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@
2525
requirements: [],
2626
}
2727
28+
$: creating = !task.id
29+
$: editing = !creating
30+
$: show_requirements = editing || task.requirements.length > 0
31+
2832
// These have their own custom VM, which is translated back to the model on save
2933
let outputSchemaSection: SchemaSection
3034
let inputSchemaSection: SchemaSection
@@ -56,7 +60,6 @@
5660
5761
async function create_task() {
5862
try {
59-
const creating = !task.id
6063
saved = false
6164
if (!target_project_id) {
6265
error = new KilnError(
@@ -169,29 +172,7 @@
169172
description: "An example task from the KilnAI team.",
170173
instruction:
171174
"Generate a joke, given a theme. The theme will be provided as a word or phrase as the input to the model. The assistant should output a joke that is funny and relevant to the theme. If a style is provided, the joke should be in that style. The output should include a setup and punchline.",
172-
requirements: [
173-
{
174-
name: "Keep on topic",
175-
instruction:
176-
"Keep the joke on topic. If the user specifies a theme, the joke must be related to that theme.",
177-
priority: 1,
178-
type: "five_star",
179-
},
180-
{
181-
name: "Keep it clean",
182-
instruction:
183-
"Avoid any jokes that are offensive or inappropriate. Keep the joke clean and appropriate for all audiences.",
184-
priority: 2,
185-
type: "pass_fail",
186-
},
187-
{
188-
name: "Be funny",
189-
instruction:
190-
"Make the joke funny and engaging. It should be something that someone would want to tell to their friends. Something clever, not just a simple pun.",
191-
priority: 1,
192-
type: "five_star",
193-
},
194-
],
175+
requirements: [],
195176
input_json_schema: JSON.stringify({
196177
type: "object",
197178
properties: {
@@ -231,7 +212,7 @@
231212

232213
<div class="flex flex-col gap-2 w-full">
233214
<FormContainer
234-
submit_label={task.id ? "Save Task" : "Create Task"}
215+
submit_label={editing ? "Save Task" : "Create Task"}
235216
on:submit={create_task}
236217
bind:warn_before_unload
237218
bind:error
@@ -240,7 +221,7 @@
240221
>
241222
<div>
242223
<div class="text-xl font-bold">Part 1: Overview</div>
243-
{#if !task.id && !hide_example_task}
224+
{#if creating && !hide_example_task}
244225
<h3 class="text-sm mt-1">
245226
Just exploring?
246227
<button class="link text-primary" on:click={example_task}
@@ -257,6 +238,14 @@
257238
max_length={120}
258239
/>
259240

241+
<FormElement
242+
label="Task Instructions / Prompt"
243+
inputType="textarea"
244+
id="task_instructions"
245+
description="A prompt for the model to follow. You can override this later."
246+
bind:value={task.instruction}
247+
/>
248+
260249
<FormElement
261250
label="Task Description"
262251
inputType="textarea"
@@ -266,14 +255,6 @@
266255
bind:value={task.description}
267256
/>
268257

269-
<FormElement
270-
label="Task Instructions"
271-
inputType="textarea"
272-
id="task_instructions"
273-
description="This will form the basis of the model's prompt. Keep this high level, and define any details in the 'Requirements' section below."
274-
bind:value={task.instruction}
275-
/>
276-
277258
<FormElement
278259
label="'Thinking' Instructions"
279260
inputType="textarea"
@@ -284,93 +265,100 @@
284265
bind:value={task.thinking_instruction}
285266
/>
286267

287-
<div class="text-sm font-medium text-left pt-6 flex flex-col gap-1">
288-
<div class="text-xl font-bold" id="requirements_part">
289-
Part 2: Requirements
290-
</div>
291-
<div class="text-xs text-gray-500">
292-
Define any requirements for the task. These will become part of the
293-
prompt, but are also broken out for model evals and training.
268+
{#if show_requirements}
269+
<div class="text-sm font-medium text-left pt-6 flex flex-col gap-1">
270+
<div class="text-xl font-bold" id="requirements_part">
271+
Part 2: Requirements
272+
</div>
273+
<div class="text-xs text-gray-500">
274+
Define requirements you can use to rate the results of the model.
275+
These are used in the prompt, ratings, evals and training.
276+
<a
277+
href="https://docs.getkiln.ai/docs/reviewing-and-rating"
278+
target="_blank"
279+
class="link">Learn more</a
280+
>.
281+
</div>
294282
</div>
295-
</div>
296283

297-
<!-- Requirements Section -->
298-
<FormList
299-
content={task.requirements}
300-
content_label="Requirement"
301-
start_with_one={true}
302-
empty_content={{
303-
name: "",
304-
description: "",
305-
instructions: "",
306-
priority: 1,
307-
}}
308-
let:item_index
309-
>
310-
<div class="flex flex-col gap-3">
311-
<div class="flex flex-row gap-1">
312-
<div class="grow flex flex-col gap-1">
313-
<FormElement
314-
label="Requirement Name"
315-
id="requirement_name_{item_index}"
316-
light_label={true}
317-
bind:value={task.requirements[item_index].name}
318-
max_length={32}
319-
/>
320-
</div>
321-
<div class="flex flex-col gap-1">
322-
<FormElement
323-
label="Rating Type"
324-
inputType="select"
325-
id="requirement_type_{item_index}"
326-
light_label={true}
327-
select_options={[
328-
["five_star", "5 Star"],
329-
["pass_fail", "Pass / Fail"],
330-
["pass_fail_critical", "Pass / Fail / Critical"],
331-
]}
332-
bind:value={task.requirements[item_index].type}
333-
/>
284+
<!-- Requirements Section -->
285+
<FormList
286+
content={task.requirements}
287+
content_label="Requirement"
288+
start_with_one={false}
289+
empty_content={{
290+
name: "",
291+
description: "",
292+
instructions: "",
293+
priority: 1,
294+
}}
295+
let:item_index
296+
>
297+
<div class="flex flex-col gap-3">
298+
<div class="flex flex-row gap-1">
299+
<div class="grow flex flex-col gap-1">
300+
<FormElement
301+
label="Requirement Name"
302+
id="requirement_name_{item_index}"
303+
light_label={true}
304+
bind:value={task.requirements[item_index].name}
305+
max_length={32}
306+
/>
307+
</div>
308+
<div class="flex flex-col gap-1">
309+
<FormElement
310+
label="Rating Type"
311+
inputType="select"
312+
id="requirement_type_{item_index}"
313+
light_label={true}
314+
select_options={[
315+
["five_star", "5 Star"],
316+
["pass_fail", "Pass / Fail"],
317+
["pass_fail_critical", "Pass / Fail / Critical"],
318+
]}
319+
bind:value={task.requirements[item_index].type}
320+
/>
321+
</div>
322+
<div class="flex flex-col gap-1">
323+
<FormElement
324+
label="Priority"
325+
inputType="select"
326+
id="requirement_priority_{item_index}"
327+
light_label={true}
328+
select_options={[
329+
[0, "P0 - Critical"],
330+
[1, "P1 - High"],
331+
[2, "P2 - Medium"],
332+
[3, "P3 - Low"],
333+
]}
334+
bind:value={task.requirements[item_index].priority}
335+
/>
336+
</div>
334337
</div>
335-
<div class="flex flex-col gap-1">
338+
<div class="grow flex flex-col gap-1">
336339
<FormElement
337-
label="Priority"
338-
inputType="select"
339-
id="requirement_priority_{item_index}"
340+
label="Instructions"
341+
inputType="textarea"
342+
id="requirement_instructions_{item_index}"
340343
light_label={true}
341-
select_options={[
342-
[0, "P0 - Critical"],
343-
[1, "P1 - High"],
344-
[2, "P2 - Medium"],
345-
[3, "P3 - Low"],
346-
]}
347-
bind:value={task.requirements[item_index].priority}
344+
bind:value={task.requirements[item_index].instruction}
348345
/>
349346
</div>
350347
</div>
351-
<div class="grow flex flex-col gap-1">
352-
<FormElement
353-
label="Instructions"
354-
inputType="textarea"
355-
id="requirement_instructions_{item_index}"
356-
light_label={true}
357-
bind:value={task.requirements[item_index].instruction}
358-
/>
359-
</div>
360-
</div>
361-
</FormList>
348+
</FormList>
349+
{/if}
362350

363351
<div class="text-sm font-medium text-left pt-6 flex flex-col gap-1">
364352
<div class="text-xl font-bold" id="requirements_part">
365-
Part 3: Input Schema
353+
Part {show_requirements ? "3" : "2"}: Input Schema
366354
</div>
367355
<div class="text-xs text-gray-500">
368356
What kind of input will the model receive?
369357
</div>
370358
</div>
371359

372360
<div>
373-
{#if task.id}
361+
{#if editing}
374362
<div>
375363
<div class="text-sm mb-2 flex flex-col gap-1">
376364
<p>
@@ -401,15 +389,15 @@
401389

402390
<div class="text-sm font-medium text-left pt-6 flex flex-col gap-1">
403391
<div class="text-xl font-bold" id="requirements_part">
404-
Part 4: Output Schema
392+
Part {show_requirements ? "4" : "3"}: Output Schema
405393
</div>
406394
<div class="text-xs text-gray-500">
407395
What kind of output will the model produce?
408396
</div>
409397
</div>
410398

411399
<div>
412-
{#if task.id}
400+
{#if editing}
413401
<div>
414402
<div class="text-sm mb-2 flex flex-col gap-1">
415403
<p>

‎app/web_ui/src/routes/(fullscreen)/setup/(setup)/create_task/schema_section.svelte

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@
5252
value={true}
5353
bind:group={plaintext}
5454
/>
55-
<span class="label-text text-left grow">Plaintext</span>
55+
<span class="label-text text-left grow">Plain Text</span>
5656
</label>
5757
</div>
5858
<div class="form-control">

0 commit comments

Comments
 (0)