backend_clients Package

backend_client

class tvb.core.services.backend_clients.backend_client.BackendClient

Bases: object

Interface for a backend client that runs operations asynchronously on a specific environment

abstract static execute(operation_id, user_name_label, adapter_instance)

Start operation asynchronously

abstract static stop_operation(operation_id)

Stop the thread for a given operation id

cluster_scheduler_client

class tvb.core.services.backend_clients.cluster_scheduler_client.ClusterSchedulerClient

Bases: BackendClient

Simple class, to mimic the same behavior we are expecting from StandAloneClient, but firing behind the cluster job scheduling process..

static execute(operation_id, user_name_label, adapter_instance)

Call the correct system command to submit a job to the cluster.

static stop_operation(operation_id)

Stop a thread for a given operation id

hpc_scheduler_client

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCJobStatus(value)

Bases: Enum

An enumeration.

FAILED = 'FAILED'

QUEUED = 'QUEUED'

READY = 'READY'

STAGINGIN = 'STAGINGIN'

STAGINGOUT = 'STAGINGOUT'

SUCCESSFUL = 'SUCCESSFUL'

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCOperationThread(operation_id, *args, **kwargs)

Bases: Thread

stop()

stopped()

class tvb.core.services.backend_clients.hpc_scheduler_client.HPCSchedulerClient

Bases: BackendClient

Simple class, to mimic the same behavior we are expecting from StandAloneClient, but firing the operation on an HPC node. Define TVB_BIN_ENV_KEY and CSCS_LOGIN_TOKEN_ENV_KEY as environment variables before running on HPC.

CONTAINER_INPUT_FOLDER = '/home/tvb_user/.data'

CSCS_DATA_FOLDER = 'data'

CSCS_LOGIN_TOKEN_ENV_KEY = 'CSCS_LOGIN_TOKEN'

CSCS_PROJECT = 'CSCS_PROJECT'

HOME_FOLDER_MOUNT = '/HOME_FOLDER'

OUTPUT_FOLDER = 'output'

TVB_BIN_ENV_KEY = 'TVB_BIN'

static compute_available_disk_space(operation: Operation) → int

static execute(operation_id: int, user_name_label: None, adapter_instance: None) → None

Call the correct system command to submit a job to HPC.

static stage_out_to_operation_folder(working_dir: Storage, operation: Operation, simulator_gid: Union[uuid.UUID, str])

static stop_operation(operation_id)

Stop the thread for a given operation id

storage_interface = <tvb.storage.storage_interface.StorageInterface object>

update_datatype_groups()

static update_db_with_results(operation: Operation, sim_h5_filenames: list, metric_operation: Operation, metric_h5_filename: str)

Generate corresponding Index entities for the resulted H5 files and insert them in DB.

tvb.core.services.backend_clients.hpc_scheduler_client.get_op_thread(op_id: int) → HPCOperationThread

standalone_client

class tvb.core.services.backend_clients.standalone_client.OperationExecutor(op_id)

Bases: Thread

Thread in charge for starting an operation, used both on cluster and with stand-alone installations.

run()

Get the required data from the operation queue and launch the operation.

static stop_pid(pid)

Stop a process specified by PID. :returns: True when specified Process was stopped in here, False in case of exception(e.g. process stopped in advance).

stopped()

Check if current thread was marked for stop.

class tvb.core.services.backend_clients.standalone_client.StandAloneClient

Bases: BackendClient

Instead of communicating with a back-end cluster, fire locally a new thread.

static execute(operation_id, user_name_label, adapter_instance)

Start asynchronous operation locally

static process_queued_operations()

static start_operation(operation_id)

static stop_operation(operation_id)

Stop a thread for a given operation id

static stop_operation_process(operation_id, notify_pods=False)