Running Jobs

The polly_jobs class contains functions which can be used to create, cancel and monitor polly jobs. Polly CLI jobs can now be initiated, managed and have a status-checked for from Polly Python. This lets users run jobs on the Polly cloud infrastructure by scaling computation resources as per need. Users can start and stop tasks and even monitor jobs.

Note:- User must use TOKEN instead of KEYS to work with the job module of polly-python.

Parameters:

• token (str, default: None ) –

  token copy from polly.

Usage

from polly.jobs import jobs

jobs = jobs(token)

cancel_job

cancel_job(project_id, job_id)

cancel a polly job.

Parameters:

• project_id (str) –

  workspace id

• job_id (str) –

  job id

Raises:

• InvalidParameterException –

  parameter passed are invalid

job_logs

job_logs(project_id, job_id, mode='all')

get logs of job

Parameters:

• project_id (str) –

  workspace_id

• job_id (str) –

  job_id

Other Parameters:

• mode (str) –

  either 'latest' or 'all' logs (default: {"all"})

Raises:

• err –

  RequestException

job_status

job_status(project_id, job_id='', internalCalls=False)

Get the status of a job given the rproject id and job id. If no job id given, gives status for all the jobs in the provided workspace.

Parameters:

• project_id (str) –

  workspace id

• job_id (str, default: '' ) –

  job id

Returns:

• DataFrame ( dict ) –

  Contains job id, job name and status sorted as per created timestamp

submit_job

submit_job(project_id, job_file=None, job_dict=None)

Submits a polly cli job in the given workspace. A message will be displayed on the status of the operation. Job configuration can be provided either as a json file using job_file parameter or as a dictionary using job_dict parameter

Parameters:

• project_id (str) –

  workspace id

• job_file (str, default: None ) –

  job configuration json file path, defaults to None

• job_dict (dict, default: None ) –

  job configuration dictionary, defaults to None

Examples

import os
from polly.auth import Polly
from polly.jobs import jobs
token = os.environ['POLLY_REFRESH_TOKEN']
Polly.auth(token)
jobs = jobs()

# Making a JSON job file with all the required params
job_map = {
    "cpu":"1",
    "memory":"1Gi",
    "image": "ubuntu",
    "tag": "latest",
    "name": "This is a job for testing bust mode",
    "command": ["/bin/bash","-c", "TERM=xterm free -h; echo '\nnumber of vCPU';nproc;sleep 30"]
  }

import json
json_object = json.dumps(job_map, indent=4)
with open ("/import/test_job.json", "w") as input:
    input.write(json_object)

submit_job()

#job submission
workspace_id = "12345"
job_file = "/import/test_job.json"
jobs.submit_job(workspace_id,job_file)

   Workspace ID                            Job ID
0         12345  1f445778c0034b44a799a0e72d48ab94

job_status()

#getting job status immediately after running 
jobs.job_status(workspace_id,"1f445778c0034b44a799a0e72d48ab94")

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   RUNNING  

#post job completion
jobs.job_status(workspace_id,"1f445778c0034b44a799a0e72d48ab94")

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   Success  

#getting status of all jobs
jobs.job_status(workspace_id)

                              Job ID                             Job Name  \
0   cc1ad0d41eb64cc68dd12c4ddbefad9a  This is a job for testing bust mode   
1   9d9895b453fe42fcb13bb7902e5de457  This is a job for testing bust mode   
2   d9e7c71c6a154a2bb018458c07db0733  This is a job for testing bust mode   
3   6d9617e623444ec2888ded77c1affa2a  This is a job for testing bust mode   

    Job State  
0     Success  
1     Success  
2     Success  
3     Success  

#workspace id can be int or string
workspace_id = 12345
jobs.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

# invalid workspace id 
workspace_id = 12345
jobs.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception

                             Job ID                             Job Name  \
0  1f445778c0034b44a799a0e72d48ab94  This is a job for testing bust mode   

  Job State  
0   Success  

# invalid job id provided while status
workspace_id = 12345
jobs.job_status("12345", "1f445778c0034b44a799a0e72d4adac")

Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception

cancel_job()

#cancelling a job post success 
jobs.cancel_job(workspace_id, "1f445778c0034b44a799a0e72d48ab94")

{'errors': [{'status': '400', 'code': 'bad_req', 'title': 'Bad Request', 'detail': 'Cannot cancel a job in Success state'}]}
Failed to cancel the job.: Cannot cancel a job in Success state

#cancelling jobs before success
jobs.submit_job(workspace_id,job_file)
jobs.cancel_job(workspace_id, "a51187d93aea402b8f40789404398bfb")

Cancelled job ID a51187d93aea402b8f40789404398bfb successfully!

job_logs()

# logs are not generated yet 
jobs.job_logs("workspace_id","a8e49e1d545c493f9cddf60f90baac2d")

Error: Not able to find the logs. It seems to be not generated yet