Skip to content

[ERP-2289] Add section explaining how to view job results #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 26, 2024
Merged

[ERP-2289] Add section explaining how to view job results #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 23 additions & 16 deletions src/components/tool/guide/EresearchJob/EresearchInstructions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
LyraStartInstructions,
RvdiStartInstructions,
} from "../Instructions/StartInstructions";
import { LyraStatusInstructions } from "../Instructions/StatusInstructions";
import { TeardownInstructions } from "../Instructions/TeardownInstructions";

const TEXT_PATTERN = /^[a-z0-9]*$/;
Expand Down Expand Up @@ -53,22 +54,28 @@ export function EresearchInstructions({ config }) {
{config.service === "Lyra" && (
<>
{batchJobName && (
<LyraStartInstructions
jobType={config.jobType}
jobName={batchJobName}
hardware={config.hardware}
cpuVendor={config.cpuVendor}
cpuModel={config.cpuModel}
cpuCores={config.cpuCores}
ram={config.ram}
gpuVendor={config.gpuVendor}
gpuModel={config.gpuModel}
gpuModules={config.gpuModules}
nodes={config.nodes}
wallTime={config.wallTime}
jobInstanceType={config.jobInstanceType}
arrayConfig={config.arrayConfig}
/>
<>
<LyraStartInstructions
jobType={config.jobType}
jobName={batchJobName}
hardware={config.hardware}
cpuVendor={config.cpuVendor}
cpuModel={config.cpuModel}
cpuCores={config.cpuCores}
ram={config.ram}
gpuVendor={config.gpuVendor}
gpuModel={config.gpuModel}
gpuModules={config.gpuModules}
nodes={config.nodes}
wallTime={config.wallTime}
jobInstanceType={config.jobInstanceType}
arrayConfig={config.arrayConfig}
/>
<LyraStatusInstructions
jobType={config.jobType}
jobName={batchJobName}
/>
</>
)}
</>
)}
Expand Down
7 changes: 6 additions & 1 deletion src/components/tool/guide/HuggingFacePipeline/HuggingFaceInstructions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Box } from "@chakra-ui/react";
import { Box, Code } from "@chakra-ui/react";

import { useState } from "react";

Expand All @@ -15,6 +15,7 @@ import {
RvdiStartInstructions,
} from "../Instructions/StartInstructions";
import { TeardownInstructions } from "../Instructions/TeardownInstructions";
import InstructionText from "../Instructions/components/InstructionText";

const TEXT_PATTERN = /^[a-z0-9]*$/;

Expand Down Expand Up @@ -55,6 +56,10 @@ export function HuggingFaceInstructions({ task, model, config, port }) {
jobInstanceType={config.jobInstanceType}
arrayConfig={config.arrayConfig}
/>
<InstructionText>
Wait for the job to start, and take note of the node you are on (eg.{" "}
<Code>cl5n042</Code>).
</InstructionText>
<InstructionInput
label="Node name"
placeholder="node"
Expand Down
101 changes: 1 addition & 100 deletions src/components/tool/guide/Instructions/StartInstructions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,4 @@
import {
Code,
Flex,
Kbd,
Radio,
RadioGroup,
Table,
Tbody,
Td,
Th,
Thead,
Tr,
} from "@chakra-ui/react";
import { Code, Flex, Kbd, Radio, RadioGroup } from "@chakra-ui/react";

import { useState } from "react";

Expand Down Expand Up @@ -158,93 +146,6 @@ export function LyraStartInstructions({
press <Kbd>Ctrl</Kbd> + <Kbd>D</Kbd> to submit the job.
</InstructionText>
)}
<InstructionHeading>Job status</InstructionHeading>
<InstructionText>
The job has been scheduled and will start running when resources are
available.
</InstructionText>
{jobType === "Interactive" && (
<>
<InstructionText>
Wait for the job to start, and take note of the node you are on (eg.{" "}
<Code>cl5n042</Code>).
</InstructionText>
</>
)}
{jobType === "Batch" && (
<>
<InstructionText>
You can check the status of your jobs by running the following
command:
</InstructionText>
<CopyBox>qstat -xu $USER</CopyBox>
<InstructionText>
The output will look similar to the following:
</InstructionText>
<CopyBox wrap={false}>
{`pbs:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
----------- -------- ----- -------- ------ --- --- ------ ----- - -----
1234567.pbs username quick job-name -- 1 4 32gb 01:00 Q --`}
</CopyBox>
<InstructionText>
The job status will be shown in the <Code>S</Code> column.
<Table size="sm">
<Thead>
<Tr>
<Th>Status</Th>
<Th>Description</Th>
</Tr>
</Thead>
<Tbody>
<Tr>
<Td>
<Code>Q</Code>
</Td>
<Td>Queued</Td>
</Tr>
<Tr>
<Td>
<Code>R</Code>
</Td>
<Td>Running</Td>
</Tr>
<Tr>
<Td>
<Code>B</Code>
</Td>
<Td>Array job has at least one subjob running</Td>
</Tr>
<Tr>
<Td>
<Code>H</Code>
</Td>
<Td>Held</Td>
</Tr>
<Tr>
<Td>
<Code>S</Code>
</Td>
<Td>Suspended</Td>
</Tr>
<Tr>
<Td>
<Code>E</Code>
</Td>
<Td>Exiting</Td>
</Tr>
<Tr>
<Td>
<Code>F</Code>
</Td>
<Td>Finished</Td>
</Tr>
</Tbody>
</Table>
</InstructionText>
</>
)}
</>
);
}
Expand Down
201 changes: 201 additions & 0 deletions src/components/tool/guide/Instructions/StatusInstructions.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
import { Code, Table, Tbody, Td, Text, Th, Thead, Tr } from "@chakra-ui/react";

import { useState } from "react";

import CopyBox from "../../../output/CopyBox";
import InstructionInput from "./InstructionInput";
import InstructionHeading from "./components/InstructionHeading";
import InstructionText from "./components/InstructionText";

export function LyraStatusInstructions({ jobType, jobName }) {
const [jobId, setJobId] = useState("");
const jobIdOrPlaceholder = jobId || "1234567";
return (
<>
<InstructionText>
The job has been scheduled and will start running when resources are
available.
</InstructionText>
{jobType === "Batch" && (
<>
<InstructionInput
label="Job ID"
placeholder="1234567"
value={jobId}
onChange={setJobId}
pattern={/^\d*$/}
helperText="Optionally enter the job ID so that it can be pre-populated in the commands below."
errorText="Job ID must only contain numbers"
/>
<InstructionHeading>Job status</InstructionHeading>
<InstructionText>
You can check the status of your jobs by running the following
command:
</InstructionText>
<CopyBox>qstat -xu $USER</CopyBox>
<InstructionText>
The output will look similar to the following:
</InstructionText>
<CopyBox wrap={false}>
{`pbs:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
----------- -------- ----- -------- ------ --- --- ------ ----- - -----
1234567.pbs username quick job-name -- 1 4 32gb 01:00 Q --`}
</CopyBox>
<InstructionText>
The job status will be shown in the <Code>S</Code> column.
</InstructionText>
<Table size="sm" variant="striped">
<Thead>
<Tr>
<Th>Status</Th>
<Th>Description</Th>
</Tr>
</Thead>
<Tbody>
<Tr>
<Td>
<Text as="b">Q</Text>
</Td>
<Td>Queued</Td>
</Tr>
<Tr>
<Td>
<Text as="b">R</Text>
</Td>
<Td>Running</Td>
</Tr>
<Tr>
<Td>
<Text as="b">B</Text>
</Td>
<Td>Array job has at least one subjob running</Td>
</Tr>
<Tr>
<Td>
<Text as="b">H</Text>
</Td>
<Td>Held</Td>
</Tr>
<Tr>
<Td>
<Text as="b">S</Text>
</Td>
<Td>Suspended</Td>
</Tr>
<Tr>
<Td>
<Text as="b">E</Text>
</Td>
<Td>Exiting</Td>
</Tr>
<Tr>
<Td>
<Text as="b">F</Text>
</Td>
<Td>Finished</Td>
</Tr>
</Tbody>
</Table>
<InstructionHeading>Job results</InstructionHeading>
<InstructionText>
The output of the job will be saved in a file named{" "}
<Code>
{jobName}.o{jobIdOrPlaceholder}
</Code>{" "}
in the same directory that the job was submitted from.
</InstructionText>
<CopyBox wrap={false}>
{`$ ls
${jobName}.e${jobIdOrPlaceholder} ${jobName}.o${jobIdOrPlaceholder}

$ cat ${jobName}.o${jobIdOrPlaceholder}
Hello, world!
PBS Job ${jobIdOrPlaceholder}.pbs
CPU time : 00:00:00
Wall time : 00:00:01
Mem usage : 0b

$ cat ${jobName}.e${jobIdOrPlaceholder}
Data written to stderr`}
</CopyBox>
<InstructionText>
Use the following command to get more detailed information about the
job, including resource usage.
</InstructionText>
<CopyBox>{`qstat -fx ${jobIdOrPlaceholder}`}</CopyBox>
<InstructionText>
Observe the resource usage of the job and adjust the requested
resources in future job submissions accordingly. The output will
look similar to the following:
</InstructionText>
<CopyBox wrap={false}>
{`$ qstat -fx ${jobIdOrPlaceholder}
Job Id: ${jobIdOrPlaceholder}.pbs
...
resources_used.cpupercent = 99
resources_used.cput = 00:39:22
resources_used.mem = 5671036kb
resources_used.ncpus = 1
resources_used.vmem = 2755392kb
resources_used.walltime = 00:40:51
...`}
</CopyBox>
<Table size="sm" my={4} variant="striped">
<Thead>
<Tr>
<Th>Resource</Th>
<Th>Description</Th>
</Tr>
</Thead>
<Tbody>
<Tr>
<Td>
<Text as="b">cpupercent</Text>
</Td>
<Td>
Sum of the percentage of CPU time used by each requested core.
(eg. 75% of each of 4 cores = 300)
</Td>
</Tr>
<Tr>
<Td>
<Text as="b">cput</Text>
</Td>
<Td>
Sum of the CPU time used by each requested core. (eg. 1 hour
using 4 cores = 04:00:00)
</Td>
</Tr>
<Tr>
<Td>
<Text as="b">mem</Text>
</Td>
<Td>Maximum memory used by the job.</Td>
</Tr>
<Tr>
<Td>
<Text as="b">ncpus</Text>
</Td>
<Td>Number of cores used by the job.</Td>
</Tr>
<Tr>
<Td>
<Text as="b">vmem</Text>
</Td>
<Td>Maximum virtual memory used by the job.</Td>
</Tr>
<Tr>
<Td>
<Text as="b">walltime</Text>
</Td>
<Td>Actual time the job was running.</Td>
</Tr>
</Tbody>
</Table>
</>
)}
</>
);
}
Loading