Running Jobs

The polly_jobs class contains functions which can be used to create, cancel and monitor polly jobs. Polly CLI jobs can now be initiated, managed and have a status-checked for from Polly Python. This lets users run jobs on the Polly cloud infrastructure by scaling computation resources as per need. Users can start and stop tasks and even monitor jobs.

Parameters:

Name	Type	Description	Default
`token`	`str`	token copy from polly.	`None`

Usage

from polly.jobs import jobs

jobs = jobs(token)

`cancel_job(project_id, job_id)`

cancel a polly job.

Parameters:

Name	Type	Description	Default
`project_id`	`str`	workspace id	required
`job_id`	`str`	job id	required

`job_logs(project_id, job_id, mode='all')`

get logs of job

Parameters:

Name	Type	Description	Default
`project_id`	`str`	workspace_id	required
`job_id`	`str`	job_id	required

Other Parameters:

Name	Type	Description
`mode`	`str`	either 'latest' or 'all' logs (default: {"all"})

Raises:

Type	Description
`err`	RequestException

`job_status(project_id, job_id='', internalCalls=False)`

Get the status of a job given the rproject id and job id. If no job id given, gives status for all the jobs in the provided workspace.

Parameters:

Name	Type	Description	Default
`project_id`	`str`	workspace id	required
`job_id`	`str`	job id	`''`

Returns:

Name	Type	Description
`DataFrame`	`dict`	Contains job id, job name and status sorted as per created timestamp

`submit_job(project_id, job_file)`

Submits a polly cli job in the given workspace. A message will be displayed on the status of the operation.

Parameters:

Name	Type	Description	Default
`project_id`	`str`	workspace id	required
`job_file`	`str`	required configuration json file path	required

Examples

import os
from polly.auth import Polly
from polly.jobs import jobs
token = os.environ['POLLY_REFRESH_TOKEN']
Polly.auth(token)
job = jobs()

# Making a JSON job file with all the required params
job_map = {
    "cpu":"1",
    "memory":"1Gi",
    "image": "ubuntu",
    "tag": "latest",
    "name": "This is a job for testing bust mode",
    "command": ["/bin/bash","-c", "TERM=xterm free -h; echo '\nnumber of vCPU';nproc;sleep 30"]
  }

import json
json_object = json.dumps(job_map, indent=4)
with open ("/import/test_job.json", "w") as input:
    input.write(json_object)

submit_job()

#job submission
workspace_id = "12345"
job_file = "/import/test_job.json"
job.submit_job(workspace_id,job_file)

   Workspace ID                            Job ID
0         12345  1f445778c0034b44a799a0e72d48ab94

job_status()

#getting job status immediately after running 
job.job_status(workspace_id,"1f445778c0034b44a799a0e72d48ab94")

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   RUNNING

#post job completion
job.job_status(workspace_id,"1f445778c0034b44a799a0e72d48ab94")

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   Success

#getting status of all jobs
job.job_status(workspace_id)

                              Job ID                             Job Name  \
0   cc1ad0d41eb64cc68dd12c4ddbefad9a  This is a job for testing bust mode   
1   9d9895b453fe42fcb13bb7902e5de457  This is a job for testing bust mode   
2   d9e7c71c6a154a2bb018458c07db0733  This is a job for testing bust mode   
3   6d9617e623444ec2888ded77c1affa2a  This is a job for testing bust mode   

    Job State  
0     Success  
1     Success  
2     Success  
3     Success

#workspace id can be int or string
workspace_id = 12345
job.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

# invalid workspace id 
workspace_id = 12345
job.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   Success

# invalid job id provided while status
workspace_id = 12345
job.job_status("12345", "1f445778c0034b44a799a0e72d4adac")

Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception

cancel_job()

#cancelling a job post success 
job.cancel_job(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

{'errors': [{'status': '400', 'code': 'bad_req', 'title': 'Bad Request', 'detail': 'Cannot cancel a job in Success state'}]}
Failed to cancel the job.: Cannot cancel a job in Success state

#cancelling jobs before success
job.submit_job(workspace_id,job_file)
job.cancel_job(workspace_id, "a51187d93aea402b8f40789404398bfb")

Cancelled job ID a51187d93aea402b8f40789404398bfb successfully!

job_logs()

# logs are not generated yet 
job.job_logs("workspace_id","a8e49e1d545c493f9cddf60f90baac2d")

Error: Not able to find the logs. It seems to be not generated yet