Toil Documentation¶
Toil is an open-source pure-Python workflow engine that lets people write better pipelines.
Check out our website for a comprehensive list of Toil’s features and read our paper to learn what Toil can do in the real world. Please subscribe to our low-volume announce mailing list and feel free to also join us on GitHub and Gitter.
If using Toil for your research, please cite
Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., … Paten, B. (2017). Toil enables reproducible, open source, big biomedical data analyses. Nature Biotechnology, 35(4), 314–316. http://doi.org/10.1038/nbt.3772
Getting Started
Common Workflow Language (CWL)
Workflow Description Language (WDL)
Advanced Toil Usage
- Introduction
- Commandline Options
- Toil Utilities
- Toil Debugging
- Running in the Cloud
- Cloud Platforms
- HPC Environments
- Workflow Execution Service (WES)
Toil Python API
- Developing a Python Workflow
- Scripting Quick Start
- Job Basics
- Invoking a Workflow
- Specifying Commandline Arguments
- Resuming a Workflow
- Functions and Job Functions
- Workflows with Multiple Jobs
- Dynamic Job Creation
- Promises
- Promised Requirements
- FileID
- Managing files within a workflow
- Using Docker Containers in Toil
- Services
- Checkpoints
- Encapsulation
- Depending on Toil
- Best Practices for Dockerizing Toil Workflows
- Toil Class API
- Job Store API
- Toil Job API
- Job Methods API
- Job.Runner API
- job.fileStore API
- Batch System API
- Job.Service API
- Exceptions API
Contributing to Toil
Appendices
- Toil Architecture
- Minimum AWS IAM permissions
- Auto-Deployment
- Environment Variables
- API Reference
- toil
- Submodules
- toil.batchSystems
- Submodules
- toil.batchSystems.abstractBatchSystem
- Attributes
- Exceptions
- Classes
- Module Contents
loggerEXIT_STATUS_UNAVAILABLE_VALUEBatchJobExitReasonUpdatedBatchJobInfoWorkerCleanupInfoAbstractBatchSystemAbstractBatchSystem.supportsAutoDeployment()AbstractBatchSystem.supportsWorkerCleanup()AbstractBatchSystem.setUserScript()AbstractBatchSystem.set_message_bus()AbstractBatchSystem.issueBatchJob()AbstractBatchSystem.killBatchJobs()AbstractBatchSystem.getIssuedBatchJobIDs()AbstractBatchSystem.getRunningBatchJobIDs()AbstractBatchSystem.getUpdatedBatchJob()AbstractBatchSystem.getSchedulingStatusMessage()AbstractBatchSystem.shutdown()AbstractBatchSystem.setEnv()AbstractBatchSystem.add_options()AbstractBatchSystem.setOptions()AbstractBatchSystem.getWorkerContexts()
BatchSystemSupportBatchSystemSupport.configBatchSystemSupport.maxCoresBatchSystemSupport.maxMemoryBatchSystemSupport.maxDiskBatchSystemSupport.environmentBatchSystemSupport.check_resource_request()BatchSystemSupport.setEnv()BatchSystemSupport.set_message_bus()BatchSystemSupport.get_batch_logs_dir()BatchSystemSupport.format_std_out_err_path()BatchSystemSupport.format_std_out_err_glob()BatchSystemSupport.workerCleanup()
NodeInfoAbstractScalableBatchSystemInsufficientSystemResourcesAcquisitionTimeoutExceptionResourcePoolResourceSet
- toil.batchSystems.abstractGridEngineBatchSystem
- Attributes
- Exceptions
- Classes
- Module Contents
loggerJobTupleExceededRetryAttemptsAbstractGridEngineBatchSystemAbstractGridEngineBatchSystem.GridEngineThreadExceptionAbstractGridEngineBatchSystem.GridEngineThreadAbstractGridEngineBatchSystem.GridEngineThread.bossAbstractGridEngineBatchSystem.GridEngineThread.newJobsQueueAbstractGridEngineBatchSystem.GridEngineThread.updatedJobsQueueAbstractGridEngineBatchSystem.GridEngineThread.killQueueAbstractGridEngineBatchSystem.GridEngineThread.killedJobsQueueAbstractGridEngineBatchSystem.GridEngineThread.waitingJobsAbstractGridEngineBatchSystem.GridEngineThread.runningJobsAbstractGridEngineBatchSystem.GridEngineThread.runningJobsLockAbstractGridEngineBatchSystem.GridEngineThread.batchJobIDsAbstractGridEngineBatchSystem.GridEngineThread.exceptionAbstractGridEngineBatchSystem.GridEngineThread.getBatchSystemID()AbstractGridEngineBatchSystem.GridEngineThread.forgetJob()AbstractGridEngineBatchSystem.GridEngineThread.createJobs()AbstractGridEngineBatchSystem.GridEngineThread.killJobs()AbstractGridEngineBatchSystem.GridEngineThread.checkOnJobs()AbstractGridEngineBatchSystem.GridEngineThread.run()AbstractGridEngineBatchSystem.GridEngineThread.coalesce_job_exit_codes()AbstractGridEngineBatchSystem.GridEngineThread.prepareSubmission()AbstractGridEngineBatchSystem.GridEngineThread.submitJob()AbstractGridEngineBatchSystem.GridEngineThread.getRunningJobIDs()AbstractGridEngineBatchSystem.GridEngineThread.killJob()AbstractGridEngineBatchSystem.GridEngineThread.getJobExitCode()
AbstractGridEngineBatchSystem.configAbstractGridEngineBatchSystem.currentJobsAbstractGridEngineBatchSystem.newJobsQueueAbstractGridEngineBatchSystem.updatedJobsQueueAbstractGridEngineBatchSystem.killQueueAbstractGridEngineBatchSystem.killedJobsQueueAbstractGridEngineBatchSystem.background_threadAbstractGridEngineBatchSystem.supportsAutoDeployment()AbstractGridEngineBatchSystem.count_needed_gpus()AbstractGridEngineBatchSystem.issueBatchJob()AbstractGridEngineBatchSystem.killBatchJobs()AbstractGridEngineBatchSystem.getIssuedBatchJobIDs()AbstractGridEngineBatchSystem.getRunningBatchJobIDs()AbstractGridEngineBatchSystem.getUpdatedBatchJob()AbstractGridEngineBatchSystem.shutdown()AbstractGridEngineBatchSystem.setEnv()AbstractGridEngineBatchSystem.getWaitDuration()AbstractGridEngineBatchSystem.sleepSeconds()AbstractGridEngineBatchSystem.with_retries()
- toil.batchSystems.awsBatch
- Attributes
- Classes
- Module Contents
loggerSTATE_TO_EXIT_REASONMAX_POLL_COUNTMIN_REQUESTABLE_MIBMIN_REQUESTABLE_CORESAWSBatchBatchSystemAWSBatchBatchSystem.supportsAutoDeployment()AWSBatchBatchSystem.regionAWSBatchBatchSystem.clientAWSBatchBatchSystem.queueAWSBatchBatchSystem.job_role_arnAWSBatchBatchSystem.owner_tagAWSBatchBatchSystem.worker_work_dirAWSBatchBatchSystem.user_scriptAWSBatchBatchSystem.docker_imageAWSBatchBatchSystem.job_definitionAWSBatchBatchSystem.bs_id_to_aws_idAWSBatchBatchSystem.aws_id_to_bs_idAWSBatchBatchSystem.killed_job_aws_idsAWSBatchBatchSystem.setUserScript()AWSBatchBatchSystem.issueBatchJob()AWSBatchBatchSystem.getUpdatedBatchJob()AWSBatchBatchSystem.shutdown()AWSBatchBatchSystem.getIssuedBatchJobIDs()AWSBatchBatchSystem.getRunningBatchJobIDs()AWSBatchBatchSystem.killBatchJobs()AWSBatchBatchSystem.add_options()AWSBatchBatchSystem.setOptions()
- toil.batchSystems.cleanup_support
- toil.batchSystems.contained_executor
- toil.batchSystems.gridengine
- Attributes
- Classes
- Module Contents
loggerGridEngineBatchSystemGridEngineBatchSystem.GridEngineThreadGridEngineBatchSystem.GridEngineThread.getRunningJobIDs()GridEngineBatchSystem.GridEngineThread.killJob()GridEngineBatchSystem.GridEngineThread.prepareSubmission()GridEngineBatchSystem.GridEngineThread.submitJob()GridEngineBatchSystem.GridEngineThread.getJobExitCode()GridEngineBatchSystem.GridEngineThread.prepareQsub()
GridEngineBatchSystem.getWaitDuration()
- toil.batchSystems.htcondor
- Attributes
- Classes
- Module Contents
loggerJobTupleschedd_lockHTCondorBatchSystemHTCondorBatchSystem.GridEngineThreadHTCondorBatchSystem.GridEngineThread.createJobs()HTCondorBatchSystem.GridEngineThread.prepareSubmission()HTCondorBatchSystem.GridEngineThread.submitJob()HTCondorBatchSystem.GridEngineThread.getRunningJobIDs()HTCondorBatchSystem.GridEngineThread.killJob()HTCondorBatchSystem.GridEngineThread.getJobExitCode()HTCondorBatchSystem.GridEngineThread.connectSchedd()HTCondorBatchSystem.GridEngineThread.duplicate_quotes()HTCondorBatchSystem.GridEngineThread.getEnvString()
HTCondorBatchSystem.issueBatchJob()
- toil.batchSystems.kubernetes
- Attributes
- Classes
- Functions
- Module Contents
loggerretryable_kubernetes_errorsis_retryable_kubernetes_error()KeyValuesListKubernetesBatchSystemKubernetesBatchSystem.supportsAutoDeployment()KubernetesBatchSystem.credential_timeKubernetesBatchSystem.namespaceKubernetesBatchSystem.host_pathKubernetesBatchSystem.service_accountKubernetesBatchSystem.pod_timeoutKubernetesBatchSystem.unique_idKubernetesBatchSystem.job_prefixKubernetesBatchSystem.finished_job_ttlKubernetesBatchSystem.user_scriptKubernetesBatchSystem.docker_imageKubernetesBatchSystem.worker_work_dirKubernetesBatchSystem.aws_secret_nameKubernetesBatchSystem.enable_watchingKubernetesBatchSystem.run_idKubernetesBatchSystem.resource_sourcesKubernetesBatchSystem.schedulingThreadKubernetesBatchSystem.DecoratorWrapperKubernetesBatchSystem.ItemTKubernetesBatchSystem.CovItemTKubernetesBatchSystem.PKubernetesBatchSystem.RKubernetesBatchSystem.setUserScript()KubernetesBatchSystem.PlacementKubernetesBatchSystem.FakeResponseKubernetesBatchSystem.TKubernetesBatchSystem.issueBatchJob()KubernetesBatchSystem.getUpdatedBatchJob()KubernetesBatchSystem.shutdown()KubernetesBatchSystem.getIssuedBatchJobIDs()KubernetesBatchSystem.getRunningBatchJobIDs()KubernetesBatchSystem.killBatchJobs()KubernetesBatchSystem.get_default_kubernetes_owner()KubernetesBatchSystem.KubernetesConfigKubernetesBatchSystem.add_options()KubernetesBatchSystem.OptionTypeKubernetesBatchSystem.setOptions()
- toil.batchSystems.local_support
- Attributes
- Classes
- Module Contents
loggerBatchSystemLocalSupportBatchSystemLocalSupport.localBatchBatchSystemLocalSupport.handleLocalJob()BatchSystemLocalSupport.killLocalJobs()BatchSystemLocalSupport.getIssuedLocalJobIDs()BatchSystemLocalSupport.getRunningLocalJobIDs()BatchSystemLocalSupport.getUpdatedLocalJob()BatchSystemLocalSupport.getNextJobID()BatchSystemLocalSupport.shutdownLocal()
- toil.batchSystems.lsf
- Attributes
- Classes
- Module Contents
loggerLSFBatchSystemLSFBatchSystem.GridEngineThreadLSFBatchSystem.GridEngineThread.getRunningJobIDs()LSFBatchSystem.GridEngineThread.fallbackRunningJobIDs()LSFBatchSystem.GridEngineThread.killJob()LSFBatchSystem.GridEngineThread.prepareSubmission()LSFBatchSystem.GridEngineThread.submitJob()LSFBatchSystem.GridEngineThread.coalesce_job_exit_codes()LSFBatchSystem.GridEngineThread.getJobExitCode()LSFBatchSystem.GridEngineThread.parse_bjobs_record()LSFBatchSystem.GridEngineThread.getJobExitCodeBACCT()LSFBatchSystem.GridEngineThread.fallbackGetJobExitCode()LSFBatchSystem.GridEngineThread.prepareBsub()LSFBatchSystem.GridEngineThread.parseBjobs()LSFBatchSystem.GridEngineThread.parseMaxMem()
LSFBatchSystem.getWaitDuration()
- toil.batchSystems.lsfHelper
- Attributes
- Functions
- Module Contents
LSB_PARAMS_FILENAMELSF_CONF_FILENAMELSF_CONF_ENVDEFAULT_LSF_UNITSDEFAULT_RESOURCE_UNITSLSF_JSON_OUTPUT_MIN_VERSIONloggerfind()find_first_match()get_conf_file()apply_conf_file()per_core_reserve_from_stream()get_lsf_units_from_stream()tokenize_conf_stream()apply_bparams()apply_lsadmin()get_lsf_units()parse_mem_and_cmd_from_output()get_lsf_version()check_lsf_json_output_supported()parse_memory()per_core_reservation()
- toil.batchSystems.mesos
- Submodules
- toil.batchSystems.mesos.batchSystem
- Attributes
- Classes
- Module Contents
logMesosBatchSystemMesosBatchSystem.supportsAutoDeployment()MesosBatchSystem.supportsWorkerCleanup()MesosBatchSystem.ExecutorInfoMesosBatchSystem.userScriptMesosBatchSystem.jobQueuesMesosBatchSystem.mesos_endpointMesosBatchSystem.mesos_nameMesosBatchSystem.mesos_timeoutMesosBatchSystem.killedJobIdsMesosBatchSystem.killJobIdsMesosBatchSystem.intendedKillMesosBatchSystem.hostToJobIDsMesosBatchSystem.nodeFilterMesosBatchSystem.runningJobMapMesosBatchSystem.taskResourcesMesosBatchSystem.updatedJobsQueueMesosBatchSystem.driverMesosBatchSystem.frameworkIdMesosBatchSystem.executorsMesosBatchSystem.agentsByIDMesosBatchSystem.nonPreemptibleNodesMesosBatchSystem.executorMesosBatchSystem.lastTimeOfferLoggedMesosBatchSystem.logPeriodMesosBatchSystem.ignoredNodesMesosBatchSystem.setUserScript()MesosBatchSystem.ignoreNode()MesosBatchSystem.unignoreNode()MesosBatchSystem.issueBatchJob()MesosBatchSystem.killBatchJobs()MesosBatchSystem.getIssuedBatchJobIDs()MesosBatchSystem.getRunningBatchJobIDs()MesosBatchSystem.getUpdatedBatchJob()MesosBatchSystem.nodeInUse()MesosBatchSystem.getWaitDuration()MesosBatchSystem.shutdown()MesosBatchSystem.registered()MesosBatchSystem.error()MesosBatchSystem.resourceOffers()MesosBatchSystem.statusUpdate()MesosBatchSystem.frameworkMessage()MesosBatchSystem.getNodes()MesosBatchSystem.reregistered()MesosBatchSystem.executorLost()MesosBatchSystem.get_default_mesos_endpoint()MesosBatchSystem.add_options()MesosBatchSystem.setOptions()
- toil.batchSystems.mesos.conftest
- toil.batchSystems.mesos.executor
- Attributes
- Classes
- Functions
- Module Contents
logMesosExecutorMesosExecutor.popenLockMesosExecutor.runningTasksMesosExecutor.workerCleanupInfoMesosExecutor.addressMesosExecutor.idMesosExecutor.registered()MesosExecutor.reregistered()MesosExecutor.disconnected()MesosExecutor.killTask()MesosExecutor.shutdown()MesosExecutor.error()MesosExecutor.launchTask()MesosExecutor.frameworkMessage()
main()
- toil.batchSystems.mesos.test
- toil.batchSystems.mesos.batchSystem
- Classes
- Package Contents
- Submodules
- toil.batchSystems.options
- toil.batchSystems.registry
- Attributes
- Functions
- Module Contents
loggeradd_batch_system_factory()get_batch_systems()get_batch_system()DEFAULT_BATCH_SYSTEMaws_batch_batch_system_factory()gridengine_batch_system_factory()lsf_batch_system_factory()single_machine_batch_system_factory()mesos_batch_system_factory()slurm_batch_system_factory()torque_batch_system_factory()htcondor_batch_system_factory()kubernetes_batch_system_factory()
- toil.batchSystems.singleMachine
- Attributes
- Classes
- Module Contents
loggerSingleMachineBatchSystemSingleMachineBatchSystem.supportsAutoDeployment()SingleMachineBatchSystem.supportsWorkerCleanup()SingleMachineBatchSystem.numCoresSingleMachineBatchSystem.minCoresSingleMachineBatchSystem.physicalMemorySingleMachineBatchSystem.configSingleMachineBatchSystem.physicalDiskSingleMachineBatchSystem.scaleSingleMachineBatchSystem.debugWorkerSingleMachineBatchSystem.jobIndexSingleMachineBatchSystem.jobIndexLockSingleMachineBatchSystem.jobsSingleMachineBatchSystem.inputQueueSingleMachineBatchSystem.outputQueueSingleMachineBatchSystem.runningJobsSingleMachineBatchSystem.childrenSingleMachineBatchSystem.childToJobSingleMachineBatchSystem.accelerator_identitiesSingleMachineBatchSystem.resource_sourcesSingleMachineBatchSystem.schedulingStatusMessageSingleMachineBatchSystem.shuttingDownSingleMachineBatchSystem.daddyThreadSingleMachineBatchSystem.daddyExceptionSingleMachineBatchSystem.daddy()SingleMachineBatchSystem.getSchedulingStatusMessage()SingleMachineBatchSystem.check_resource_request()SingleMachineBatchSystem.issueBatchJob()SingleMachineBatchSystem.killBatchJobs()SingleMachineBatchSystem.getIssuedBatchJobIDs()SingleMachineBatchSystem.getRunningBatchJobIDs()SingleMachineBatchSystem.shutdown()SingleMachineBatchSystem.getUpdatedBatchJob()SingleMachineBatchSystem.add_options()SingleMachineBatchSystem.setOptions()
Info
- toil.batchSystems.slurm
- Attributes
- Classes
- Functions
- Module Contents
loggerTERMINAL_STATESNONTERMINAL_STATESparse_slurm_time()option_detector()any_option_detector()SlurmBatchSystemSlurmBatchSystem.PartitionInfoSlurmBatchSystem.PartitionSetSlurmBatchSystem.GridEngineThreadSlurmBatchSystem.GridEngineThread.bossSlurmBatchSystem.GridEngineThread.getRunningJobIDs()SlurmBatchSystem.GridEngineThread.killJob()SlurmBatchSystem.GridEngineThread.prepareSubmission()SlurmBatchSystem.GridEngineThread.submitJob()SlurmBatchSystem.GridEngineThread.coalesce_job_exit_codes()SlurmBatchSystem.GridEngineThread.getJobExitCode()SlurmBatchSystem.GridEngineThread.prepareSbatch()
SlurmBatchSystem.partitionsSlurmBatchSystem.start_timeSlurmBatchSystem.issueBatchJob()SlurmBatchSystem.add_options()SlurmBatchSystem.OptionTypeSlurmBatchSystem.setOptions()
- toil.batchSystems.torque
- Attributes
- Classes
- Module Contents
loggerTorqueBatchSystemTorqueBatchSystem.GridEngineThreadTorqueBatchSystem.GridEngineThread.getRunningJobIDs()TorqueBatchSystem.GridEngineThread.getUpdatedBatchJob()TorqueBatchSystem.GridEngineThread.killJob()TorqueBatchSystem.GridEngineThread.prepareSubmission()TorqueBatchSystem.GridEngineThread.submitJob()TorqueBatchSystem.GridEngineThread.getJobExitCode()TorqueBatchSystem.GridEngineThread.prepareQsub()TorqueBatchSystem.GridEngineThread.generateTorqueWrapper()
- toil.batchSystems.abstractBatchSystem
- Exceptions
- Package Contents
- Submodules
- toil.bus
- Attributes
- Classes
- Functions
- Module Contents
loggerNamesget_job_kind()JobIssuedMessageJobUpdatedMessageJobCompletedMessageJobFailedMessageJobMissingMessageJobAnnotationMessageExternalBatchIdMessageQueueSizeMessageClusterSizeMessageClusterDesiredSizeMessagemessage_to_bytes()MessageTypebytes_to_message()MessageBusMessageBusClientMessageInboxMessageOutboxMessageBusConnectionJobStatusreplay_message_bus()gen_message_bus_path()
- toil.common
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
UUID_LENGTHloggerget_default_config_path()ConfigConfig.logFileConfig.logRotatingConfig.cleanWorkDirConfig.max_jobsConfig.max_local_jobsConfig.manualMemArgsConfig.run_local_jobs_on_workersConfig.coalesceStatusCallsConfig.mesos_endpointConfig.mesos_framework_idConfig.mesos_roleConfig.mesos_nameConfig.kubernetes_host_pathConfig.kubernetes_ownerConfig.kubernetes_service_accountConfig.kubernetes_pod_timeoutConfig.kubernetes_privilegedConfig.kubernetes_pod_security_contextConfig.kubernetes_security_contextConfig.tes_endpointConfig.tes_userConfig.tes_passwordConfig.tes_bearer_tokenConfig.aws_batch_regionConfig.aws_batch_queueConfig.aws_batch_job_role_arnConfig.scaleConfig.batchSystemConfig.batch_logs_dirConfig.statePollingWaitConfig.state_polling_timeoutConfig.disableAutoDeploymentConfig.workflowIDConfig.workflowAttemptNumberConfig.jobStoreConfig.logLevelConfig.colored_logsConfig.workDirConfig.coordination_dirConfig.noStdOutErrConfig.statsConfig.cleanConfig.clusterStatsConfig.restartConfig.cachingConfig.symlinkImportsConfig.moveOutputsConfig.symlink_job_store_readsConfig.provisionerConfig.nodeTypesConfig.minNodesConfig.maxNodesConfig.targetTimeConfig.betaInertiaConfig.scaleIntervalConfig.preemptibleCompensationConfig.nodeStorageConfig.nodeStorageOverridesConfig.metricsConfig.assume_zero_overheadConfig.maxPreemptibleServiceJobsConfig.maxServiceJobsConfig.deadlockWaitConfig.deadlockCheckIntervalConfig.defaultMemoryConfig.defaultCoresConfig.defaultDiskConfig.defaultPreemptibleConfig.defaultAcceleratorsConfig.maxCoresConfig.maxMemoryConfig.maxDiskConfig.retryCountConfig.stop_on_first_failureConfig.enableUnlimitedPreemptibleRetriesConfig.doubleMemConfig.maxJobDurationConfig.rescueJobsFrequencyConfig.job_store_timeoutConfig.maxLogFileSizeConfig.writeLogsConfig.writeLogsGzipConfig.writeLogsFromAllJobsConfig.write_messagesConfig.realTimeLoggingConfig.publish_workflow_metricsConfig.environmentConfig.disableChainingConfig.disableJobStoreChecksumVerificationConfig.sseKeyConfig.servicePollingIntervalConfig.useAsyncConfig.forceDockerApplianceConfig.statusWaitConfig.disableProgressConfig.readGlobalFileMutableByDefaultConfig.debugWorkerConfig.disableWorkerOutputCaptureConfig.badWorkerConfig.badWorkerFailIntervalConfig.kill_polling_intervalConfig.cwlConfig.memory_is_productConfig.set_from_default_config()Config.prepare_start()Config.prepare_restart()Config.setOptions()Config.check_configuration_consistency()Config.__eq__()Config.__hash__()
ensure_config()generate_config()update_config()parser_with_common_options()addOptions()getNodeID()ToilToil.configToil.optionsToil.__enter__()Toil.__exit__()Toil.start()Toil.restart()Toil.JOB_STORE_TYPESToil.getJobStore()Toil.parseLocator()Toil.buildLocator()Toil.canonical_locator()Toil.resumeJobStore()Toil.createBatchSystem()Toil.url_exists()Toil.importFile()Toil.import_file()Toil.exportFile()Toil.export_file()Toil.normalize_uri()Toil.getToilWorkDir()Toil.get_toil_coordination_dir()Toil.get_workflow_path_component()Toil.getLocalWorkflowDir()Toil.get_local_workflow_coordination_dir()
ToilRestartExceptionToilContextManagerExceptionToilMetricsToilMetrics.mtailImageToilMetrics.grafanaImageToilMetrics.prometheusImageToilMetrics.nodeExporterProcToilMetrics.startDashboard()ToilMetrics.add_prometheus_data_source()ToilMetrics.log()ToilMetrics.logClusterSize()ToilMetrics.logClusterDesiredSize()ToilMetrics.logQueueSize()ToilMetrics.logMissingJob()ToilMetrics.logIssuedJob()ToilMetrics.logFailedJob()ToilMetrics.logCompletedJob()ToilMetrics.shutdown()
cacheDirName()getDirSizeRecursively()getFileSystemSize()safeUnpickleFromStream()
- toil.cwl
- Submodules
- toil.cwl.conftest
- toil.cwl.cwltoil
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerDEFAULT_TMPDIRDEFAULT_TMPDIR_PREFIXcwltoil_was_removed()UnresolvedDictSkipNullfilter_skip_null()ensure_no_collisions()try_prepull()ConditionalResolveSourceStepValueFromDefaultWithSourceJustAValueresolve_dict_w_promises()simplify_list()ToilPathMapperToilSingleJobExecutorToilToolToilCommandLineToolToilExpressionTooltoil_make_tool()MISSING_FILEToilFsAccesstoil_get_file()import_file_through_cache()path_to_loc()extract_file_uri_once()VFileVisitFuncDirectoryVisitFuncV2visit_files()upload_directory()ensure_file_imported()writeGlobalFileWrapper()remove_empty_listings()CWLNamedJobResolveIndirecttoilStageFiles()CWLJobWrapperCWLJobget_container_engine()makeRootJob()makeJob()CWLScatterCWLGatherSelfJobProcessTyperemove_pickle_problems()CWLWorkflowCWLInstallImportsJobCWLInstallImportsJob.initialized_job_orderCWLInstallImportsJob.toolCWLInstallImportsJob.basedirCWLInstallImportsJob.skip_remoteCWLInstallImportsJob.bypass_file_storeCWLInstallImportsJob.leader_importsCWLInstallImportsJob.worker_importsCWLInstallImportsJob.fill_in_files()CWLInstallImportsJob.run()
CWLImportWrapperCWLStartJobextract_workflow_inputs()import_workflow_inputs()TvisitSteps()rm_unprocessed_secondary_files()filtered_secondary_files()scan_for_unsupported_requirements()determine_load_listing()NoAvailableJobStoreExceptiongenerate_default_job_store()usage_messageget_options()main()find_default_container()
- toil.cwl.utils
- Attributes
- Exceptions
- Functions
- Module Contents
loggerCWL_UNSUPPORTED_REQUIREMENT_EXIT_CODECWLUnsupportedExceptionCWL_UNSUPPORTED_REQUIREMENT_EXCEPTIONvisit_top_cwl_class()DownReturnTypeUpReturnTypevisit_cwl_class_and_reduce()DirectoryStructureget_from_structure()download_structure()trim_mounts_op_down()sniff_location()trim_mounts_op_up()remove_redundant_mounts()
- Functions
- Package Contents
- Submodules
- toil.deferred
- Attributes
- Classes
- Module Contents
loggerDeferredFunctionDeferredFunctionManagerDeferredFunctionManager.STATE_DIR_STEMDeferredFunctionManager.PREFIXDeferredFunctionManager.WIP_SUFFIXDeferredFunctionManager.stateDirDeferredFunctionManager.stateFileNameDeferredFunctionManager.stateFileOutDeferredFunctionManager.stateFileInDeferredFunctionManager.__del__()DeferredFunctionManager.open()DeferredFunctionManager.cleanupWorker()
- toil.exceptions
- toil.fileStores
- Submodules
- toil.fileStores.abstractFileStore
- Attributes
- Classes
- Module Contents
loggerAbstractFileStoreAbstractFileStore.jobStoreAbstractFileStore.jobDescAbstractFileStore.localTempDirAbstractFileStore.workflow_dirAbstractFileStore.coordination_dirAbstractFileStore.jobNameAbstractFileStore.waitForPreviousCommitAbstractFileStore.logging_messagesAbstractFileStore.logging_user_streamsAbstractFileStore.filesToDeleteAbstractFileStore.createFileStore()AbstractFileStore.shutdownFileStore()AbstractFileStore.open()AbstractFileStore.get_disk_usage()AbstractFileStore.getLocalTempDir()AbstractFileStore.getLocalTempFile()AbstractFileStore.getLocalTempFileName()AbstractFileStore.writeGlobalFile()AbstractFileStore.writeGlobalFileStream()AbstractFileStore.logAccess()AbstractFileStore.readGlobalFile()AbstractFileStore.readGlobalFileStream()AbstractFileStore.getGlobalFileSize()AbstractFileStore.deleteLocalFile()AbstractFileStore.deleteGlobalFile()AbstractFileStore.importFile()AbstractFileStore.import_file()AbstractFileStore.exportFile()AbstractFileStore.export_file()AbstractFileStore.log_to_leader()AbstractFileStore.logToMaster()AbstractFileStore.log_user_stream()AbstractFileStore.startCommit()AbstractFileStore.waitForCommit()AbstractFileStore.shutdown()
- toil.fileStores.cachingFileStore
- Attributes
- Exceptions
- Classes
- Module Contents
loggerSQLITE_TIMEOUT_SECSCacheErrorCacheUnbalancedErrorIllegalDeletionCacheErrorInvalidSourceCacheErrorCachingFileStoreCachingFileStore.forceNonFreeCachingCachingFileStore.forceDownloadDelayCachingFileStore.contentionBackoffCachingFileStore.localCacheDirCachingFileStore.jobNameCachingFileStore.jobIDCachingFileStore.jobDiskBytesCachingFileStore.workflowAttemptNumberCachingFileStore.dbPathCachingFileStore.process_identity_lockCachingFileStore.commitThreadCachingFileStore.as_process()CachingFileStore.conCachingFileStore.curCachingFileStore.getCacheLimit()CachingFileStore.getCacheUsed()CachingFileStore.getCacheExtraJobSpace()CachingFileStore.getCacheAvailable()CachingFileStore.getSpaceUsableForJobs()CachingFileStore.getCacheUnusedJobRequirement()CachingFileStore.adjustCacheLimit()CachingFileStore.fileIsCached()CachingFileStore.getFileReaderCount()CachingFileStore.cachingIsFree()CachingFileStore.open()CachingFileStore.writeGlobalFile()CachingFileStore.readGlobalFile()CachingFileStore.readGlobalFileStream()CachingFileStore.deleteLocalFile()CachingFileStore.deleteGlobalFile()CachingFileStore.exportFile()CachingFileStore.export_file()CachingFileStore.waitForCommit()CachingFileStore.startCommit()CachingFileStore.startCommitThread()CachingFileStore.shutdown()CachingFileStore.__del__()
- toil.fileStores.nonCachingFileStore
- Attributes
- Classes
- Module Contents
loggerNonCachingFileStoreNonCachingFileStore.jobStateFileNonCachingFileStore.localFileMapNonCachingFileStore.check_for_coordination_corruption()NonCachingFileStore.check_for_state_corruption()NonCachingFileStore.open()NonCachingFileStore.writeGlobalFile()NonCachingFileStore.readGlobalFile()NonCachingFileStore.readGlobalFileStream()NonCachingFileStore.exportFile()NonCachingFileStore.export_file()NonCachingFileStore.deleteLocalFile()NonCachingFileStore.deleteGlobalFile()NonCachingFileStore.waitForCommit()NonCachingFileStore.startCommit()NonCachingFileStore.__del__()NonCachingFileStore.shutdown()
- toil.fileStores.abstractFileStore
- Classes
- Package Contents
- Submodules
- toil.job
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerJobPromiseConstraintErrorConflictingPredecessorErrorDebugStoppingPointReachedFilesDownloadedStoppingPointReachedTemporaryIDAcceleratorRequirementparse_accelerator()accelerator_satisfies()accelerators_fully_satisfy()RequirementsDictREQUIREMENT_NAMESParsedRequirementParseableIndivisibleResourceParseableDivisibleResourceParseableFlagParseableAcceleratorRequirementParseableRequirementRequirerJobBodyReferenceJobDescriptionJobDescription.localJobDescription.jobNameJobDescription.unitNameJobDescription.displayNameJobDescription.jobStoreIDJobDescription.filesToDeleteJobDescription.predecessorNumberJobDescription.predecessorsFinishedJobDescription.childIDsJobDescription.followOnIDsJobDescription.successor_phasesJobDescription.serviceTreeJobDescription.logJobStoreFileIDJobDescription.files_to_useJobDescription.get_names()JobDescription.get_chain()JobDescription.serviceHostIDsInBatches()JobDescription.successorsAndServiceHosts()JobDescription.allSuccessors()JobDescription.successors_by_phase()JobDescription.servicesJobDescription.has_body()JobDescription.attach_body()JobDescription.detach_body()JobDescription.get_body()JobDescription.nextSuccessors()JobDescription.filterSuccessors()JobDescription.filterServiceHosts()JobDescription.clear_nonexistent_dependents()JobDescription.clear_dependents()JobDescription.is_subtree_done()JobDescription.replace()JobDescription.assert_is_not_newer_than()JobDescription.is_updated_by()JobDescription.addChild()JobDescription.addFollowOn()JobDescription.addServiceHostJob()JobDescription.hasChild()JobDescription.hasFollowOn()JobDescription.hasServiceHostJob()JobDescription.renameReferences()JobDescription.addPredecessor()JobDescription.onRegistration()JobDescription.setupJobAfterFailure()JobDescription.getLogFileHandle()JobDescription.remainingTryCountJobDescription.clearRemainingTryCount()JobDescription.__str__()JobDescription.__repr__()JobDescription.reserve_versions()JobDescription.pre_update_hook()
ServiceJobDescriptionCheckpointJobDescriptionJobJob.userModuleJob.__str__()Job.check_initialized()Job.jobStoreIDJob.descriptionJob.diskJob.memoryJob.coresJob.acceleratorsJob.preemptibleJob.preemptable()Job.checkpointJob.files_to_useJob.add_to_files_to_use()Job.remove_from_files_to_use()Job.assignConfig()Job.run()Job.addChild()Job.hasChild()Job.addFollowOn()Job.hasPredecessor()Job.hasFollowOn()Job.addService()Job.hasService()Job.addChildFn()Job.addFollowOnFn()Job.addChildJobFn()Job.addFollowOnJobFn()Job.tempDirJob.log()Job.wrapFn()Job.wrapJobFn()Job.encapsulate()Job.rv()Job.registerPromise()Job.prepareForPromiseRegistration()Job.checkJobGraphForDeadlocks()Job.getRootJobs()Job.checkJobGraphConnected()Job.checkJobGraphAcylic()Job.checkNewCheckpointsAreLeafVertices()Job.defer()Job.RunnerJob.ServiceJob.getUserScript()Job.getTopologicalOrderingOfJobs()Job.saveBody()Job.saveAsRootJob()Job.loadJob()Job.set_debug_flag()Job.has_debug_flag()Job.files_downloaded_hook()
JobExceptionJobGraphDeadlockExceptionFunctionWrappingJobJobFunctionWrappingJobPromisedRequirementFunctionWrappingJobPromisedRequirementJobFunctionWrappingJobEncapsulatedJobServiceHostJobFileMetadatapotential_absolute_uris()get_file_sizes()CombineImportsJobWorkerImportJobImportsJobPromiseTPromisedunwrap()unwrap_all()PromisedRequirementUnfulfilledPromiseSentinel
- toil.jobStores
- Submodules
- toil.jobStores.abstractJobStore
- Attributes
- Exceptions
- Classes
- Module Contents
loggerProxyConnectionErrorLocatorExceptionInvalidImportExportUrlExceptionNoSuchJobExceptionConcurrentFileModificationExceptionNoSuchFileExceptionNoSuchJobStoreExceptionJobStoreExistsExceptionAbstractJobStoreAbstractJobStore.initialize()AbstractJobStore.writeConfig()AbstractJobStore.write_config()AbstractJobStore.resume()AbstractJobStore.configAbstractJobStore.locatorAbstractJobStore.rootJobStoreIDFileNameAbstractJobStore.setRootJob()AbstractJobStore.set_root_job()AbstractJobStore.loadRootJob()AbstractJobStore.load_root_job()AbstractJobStore.createRootJob()AbstractJobStore.create_root_job()AbstractJobStore.getRootJobReturnValue()AbstractJobStore.get_root_job_return_value()AbstractJobStore.importFile()AbstractJobStore.import_file()AbstractJobStore.exportFile()AbstractJobStore.export_file()AbstractJobStore.destroy()AbstractJobStore.getEnv()AbstractJobStore.get_env()AbstractJobStore.clean()AbstractJobStore.assignID()AbstractJobStore.assign_job_id()AbstractJobStore.create()AbstractJobStore.create_job()AbstractJobStore.exists()AbstractJobStore.job_exists()AbstractJobStore.publicUrlExpirationAbstractJobStore.getPublicUrl()AbstractJobStore.get_public_url()AbstractJobStore.getSharedPublicUrl()AbstractJobStore.get_shared_public_url()AbstractJobStore.load()AbstractJobStore.load_job()AbstractJobStore.update()AbstractJobStore.batch()AbstractJobStore.update_job()AbstractJobStore.delete()AbstractJobStore.delete_job()AbstractJobStore.jobs()AbstractJobStore.writeFile()AbstractJobStore.write_file()AbstractJobStore.writeFileStream()AbstractJobStore.write_file_stream()AbstractJobStore.getEmptyFileStoreID()AbstractJobStore.get_empty_file_store_id()AbstractJobStore.readFile()AbstractJobStore.read_file()AbstractJobStore.readFileStream()AbstractJobStore.read_file_stream()AbstractJobStore.deleteFile()AbstractJobStore.delete_file()AbstractJobStore.fileExists()AbstractJobStore.file_exists()AbstractJobStore.getFileSize()AbstractJobStore.get_file_size()AbstractJobStore.updateFile()AbstractJobStore.update_file()AbstractJobStore.updateFileStream()AbstractJobStore.update_file_stream()AbstractJobStore.sharedFileNameRegexAbstractJobStore.writeSharedFileStream()AbstractJobStore.write_shared_file_stream()AbstractJobStore.readSharedFileStream()AbstractJobStore.read_shared_file_stream()AbstractJobStore.writeStatsAndLogging()AbstractJobStore.write_logs()AbstractJobStore.readStatsAndLogging()AbstractJobStore.read_logs()AbstractJobStore.write_leader_pid()AbstractJobStore.read_leader_pid()AbstractJobStore.write_leader_node_id()AbstractJobStore.read_leader_node_id()AbstractJobStore.write_kill_flag()AbstractJobStore.read_kill_flag()AbstractJobStore.default_caching()
JobStoreSupport
- toil.jobStores.aws
- Submodules
- toil.jobStores.aws.jobStore
- Attributes
- Classes
- Functions
- Module Contents
DEFAULT_AWS_PART_SIZEloggerAWSJobStoreAWSJobStore.s3_resourceAWSJobStore.s3_clientAWSJobStore.part_sizeAWSJobStore.bucketAWSJobStore.job_key_prefixAWSJobStore.job_associations_key_prefixAWSJobStore.content_key_prefixAWSJobStore.shared_key_prefixAWSJobStore.logs_key_prefixAWSJobStore.initialize()AWSJobStore.resume()AWSJobStore.destroy()AWSJobStore.is_in_bucket()AWSJobStore.write_to_bucket()AWSJobStore.read_from_bucket()AWSJobStore.assign_job_id()AWSJobStore.create_job()AWSJobStore.job_exists()AWSJobStore.jobs()AWSJobStore.load_job()AWSJobStore.update_job()AWSJobStore.delete_job()AWSJobStore.associate_job_with_file()AWSJobStore.write_file()AWSJobStore.find_s3_key_from_file_id()AWSJobStore.write_file_stream()AWSJobStore.update_file_stream()AWSJobStore.write_shared_file_stream()AWSJobStore.update_file()AWSJobStore.file_exists()AWSJobStore.get_file_size()AWSJobStore.read_file()AWSJobStore.read_file_stream()AWSJobStore.read_shared_file_stream()AWSJobStore.delete_file()AWSJobStore.get_public_url()AWSJobStore.get_shared_public_url()AWSJobStore.get_empty_file_store_id()AWSJobStore.write_logs()AWSJobStore.read_logs()
parse_jobstore_identifier()
- toil.jobStores.aws.utils
- toil.jobStores.aws.jobStore
- Submodules
- toil.jobStores.conftest
- toil.jobStores.fileJobStore
- Attributes
- Classes
- Module Contents
loggerFileJobStoreFileJobStore.validDirsFileJobStore.validDirsSetFileJobStore.JOB_DIR_PREFIXFileJobStore.JOB_NAME_DIR_PREFIXFileJobStore.BUFFER_SIZEFileJobStore.LOG_TEMP_SUFFIXFileJobStore.LOG_PREFIXFileJobStore.default_caching()FileJobStore.jobStoreDirFileJobStore.jobsDirFileJobStore.statsDirFileJobStore.stats_inboxFileJobStore.stats_archiveFileJobStore.filesDirFileJobStore.jobFilesDirFileJobStore.sharedFilesDirFileJobStore.fanOutFileJobStore.linkImportsFileJobStore.moveExportsFileJobStore.symlink_job_store_readsFileJobStore.__repr__()FileJobStore.initialize()FileJobStore.resume()FileJobStore.destroy()FileJobStore.assign_job_id()FileJobStore.create_job()FileJobStore.batch()FileJobStore.job_exists()FileJobStore.get_public_url()FileJobStore.get_shared_public_url()FileJobStore.load_job()FileJobStore.update_job()FileJobStore.delete_job()FileJobStore.jobs()FileJobStore.write_file()FileJobStore.write_file_stream()FileJobStore.get_empty_file_store_id()FileJobStore.update_file()FileJobStore.read_file()FileJobStore.delete_file()FileJobStore.file_exists()FileJobStore.get_file_size()FileJobStore.update_file_stream()FileJobStore.read_file_stream()FileJobStore.write_shared_file_stream()FileJobStore.read_shared_file_stream()FileJobStore.list_all_file_names()FileJobStore.write_logs()FileJobStore.read_logs()
- toil.jobStores.googleJobStore
- Attributes
- Classes
- Functions
- Module Contents
logGOOGLE_STORAGEMAX_BATCH_SIZEgoogle_retry_predicate()google_retry()permission_error_reporter()GoogleJobStoreGoogleJobStore.nodeServiceAccountJsonGoogleJobStore.projectIDGoogleJobStore.bucketNameGoogleJobStore.bucketGoogleJobStore.statsBaseIDGoogleJobStore.statsReadPrefixGoogleJobStore.readStatsBaseIDGoogleJobStore.sseKeyGoogleJobStore.create_client()GoogleJobStore.initialize()GoogleJobStore.resume()GoogleJobStore.destroy()GoogleJobStore.assign_job_id()GoogleJobStore.batch()GoogleJobStore.create_job()GoogleJobStore.job_exists()GoogleJobStore.get_public_url()GoogleJobStore.get_shared_public_url()GoogleJobStore.load_job()GoogleJobStore.update_job()GoogleJobStore.delete_job()GoogleJobStore.get_env()GoogleJobStore.jobs()GoogleJobStore.write_file()GoogleJobStore.write_file_stream()GoogleJobStore.get_empty_file_store_id()GoogleJobStore.read_file()GoogleJobStore.read_file_stream()GoogleJobStore.delete_file()GoogleJobStore.file_exists()GoogleJobStore.get_file_size()GoogleJobStore.update_file()GoogleJobStore.update_file_stream()GoogleJobStore.write_shared_file_stream()GoogleJobStore.read_shared_file_stream()GoogleJobStore.write_logs()GoogleJobStore.read_logs()
- toil.jobStores.utils
- toil.jobStores.abstractJobStore
- Submodules
- toil.leader
- Attributes
- Classes
- Module Contents
loggerLeaderLeader.configLeader.jobStoreLeader.jobStoreLocatorLeader.toilStateLeader.batchSystemLeader.issued_jobs_by_batch_system_idLeader.preemptibleJobsIssuedLeader.serviceJobsIssuedLeader.serviceJobsToBeIssuedLeader.preemptibleServiceJobsIssuedLeader.preemptibleServiceJobsToBeIssuedLeader.timeSinceJobsLastRescuedLeader.reissueMissingJobs_missingHashLeader.provisionerLeader.clusterScalerLeader.serviceManagerLeader.statsAndLoggingLeader.potentialDeadlockedJobsLeader.potentialDeadlockTimeLeader.toilMetricsLeader.debugJobNamesLeader.deadlockThrottlerLeader.statusThrottlerLeader.kill_throttlerLeader.progress_overallLeader.progress_failedLeader.GOOD_COLORLeader.BAD_COLORLeader.PROGRESS_BAR_FORMATLeader.recommended_fail_exit_codeLeader.run()Leader.create_status_sentinel_file()Leader.innerLoop()Leader.checkForDeadlocks()Leader.feed_deadlock_watchdog()Leader.issueJob()Leader.issueJobs()Leader.issueServiceJob()Leader.issueQueingServiceJobs()Leader.getNumberOfJobsIssued()Leader.removeJob()Leader.getJobs()Leader.killJobs()Leader.reissueOverLongJobs()Leader.reissueMissingJobs()Leader.processRemovedJob()Leader.process_finished_job()Leader.process_finished_job_description()Leader.getSuccessors()Leader.processTotallyFailedJob()
- toil.lib
- Submodules
- toil.lib.accelerators
- toil.lib.aws
- Submodules
- toil.lib.aws.ami
- toil.lib.aws.config
- toil.lib.aws.iam
- Attributes
- Functions
- Module Contents
loggerCLUSTER_LAUNCHING_PERMISSIONSAllowedActionCollectiondelete_iam_instance_profile()delete_iam_role()create_iam_role()init_action_collection()add_to_action_collection()policy_permissions_allow()permission_matches_any()get_actions_from_policy_document()allowed_actions_attached()allowed_actions_roles()collect_policy_actions()allowed_actions_user()allowed_actions_group()get_policy_permissions()get_aws_account_num()
- toil.lib.aws.s3
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerAWS_MAX_MULTIPART_COUNTAWS_MAX_CHUNK_SIZEAWS_MIN_CHUNK_SIZEDEFAULT_AWS_CHUNK_SIZEAWSKeyNotFoundErrorAWSKeyAlreadyExistsErrorAWSBadEncryptionKeyErrorcreate_s3_bucket()delete_s3_bucket()bucket_exists()head_s3_object()list_multipart_uploads()copy_s3_to_s3()copy_local_to_s3()copy_s3_to_local()MultiPartPipeparse_s3_uri()list_s3_items()upload_to_s3()download_stream()download_fileobject()s3_key_exists()get_s3_object()put_s3_object()generate_presigned_url()create_public_url()get_s3_bucket_region()
- toil.lib.aws.session
- toil.lib.aws.utils
- Attributes
- Exceptions
- Functions
- Module Contents
loggerTHROTTLED_ERROR_CODESdelete_sdb_domain()connection_reset()connection_error()retryable_s3_errors()retry_s3()delete_s3_bucket()create_s3_bucket()enable_public_objects()NoBucketLocationErrorget_bucket_region()get_bucket_region_if_available()region_to_bucket_location()bucket_location_to_region()get_object_for_url()list_objects_for_url()flatten_tags()boto3_pager()
- Attributes
- Functions
- Package Contents
- Submodules
- toil.lib.bioio
- toil.lib.checksum
- toil.lib.compatibility
- toil.lib.conversions
- toil.lib.directory
- toil.lib.docker
- toil.lib.dockstore
- toil.lib.ec2
- Attributes
- Exceptions
- Functions
- Module Contents
a_short_timea_long_timeloggeris_base64()UserErrornot_found()inconsistencies_detected()INCONSISTENCY_ERRORSretry_ec2()UnexpectedResourceStatewait_transition()wait_instances_running()wait_spot_requests_active()create_spot_instances()create_ondemand_instances()increase_instance_hop_limit()prune()iam_clientwait_until_instance_profile_arn_exists()create_instances()create_launch_template()create_auto_scaling_group()
- toil.lib.ec2nodes
- toil.lib.encryption
- toil.lib.exceptions
- toil.lib.expando
- toil.lib.ftp_utils
- toil.lib.generatedEC2Lists
- toil.lib.history
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerHistoryDatabaseSchemaTooNewErrorWorkflowSummaryWorkflowAttemptSummaryWorkflowAttemptSummary.workflow_idWorkflowAttemptSummary.attempt_numberWorkflowAttemptSummary.succeededWorkflowAttemptSummary.start_timeWorkflowAttemptSummary.runtimeWorkflowAttemptSummary.submitted_to_dockstoreWorkflowAttemptSummary.batch_systemWorkflowAttemptSummary.cachingWorkflowAttemptSummary.toil_versionWorkflowAttemptSummary.python_versionWorkflowAttemptSummary.platform_systemWorkflowAttemptSummary.platform_machineWorkflowAttemptSummary.workflow_job_storeWorkflowAttemptSummary.workflow_trs_spec
JobAttemptSummaryRTdb_retry()HistoryManagerHistoryManager.enabled()HistoryManager.enabled_job()HistoryManager.database_path_overrideHistoryManager.database_path()HistoryManager.connection()HistoryManager.no_transaction()HistoryManager.ensure_tables()HistoryManager.record_workflow_creation()HistoryManager.record_workflow_metadata()HistoryManager.record_job_attempt()HistoryManager.record_workflow_attempt()HistoryManager.summarize_workflows()HistoryManager.get_submittable_workflow_attempts()HistoryManager.get_workflow_attempts_with_submittable_job_attempts()HistoryManager.get_workflow_attempt()HistoryManager.get_unsubmitted_job_attempts()HistoryManager.mark_workflow_attempt_submitted()HistoryManager.mark_job_attempts_submitted()HistoryManager.count_workflows()HistoryManager.count_workflow_attempts()HistoryManager.count_job_attempts()HistoryManager.get_fully_submitted_workflow_ids()HistoryManager.get_oldest_workflow_ids()HistoryManager.delete_workflow()HistoryManager.get_database_byte_size()HistoryManager.compact_database()HistoryManager.enforce_byte_size_limit()HistoryManager.database_dump_lines()
- toil.lib.history_submission
- Attributes
- Classes
- Functions
- Module Contents
loggerworkflow_execution_id()workflow_task_set_execution_id()job_execution_id()get_parsed_trs_spec()Submissioncreate_history_submission()create_current_submission()KeyTypedialog_tkinter()display_dialog_tkinter()dialog_tui()DecisionDIALOG_TITLEDIALOG_TEXTDIALOG_OPTIONSDIALOG_TIMEOUTask_user_about_publishing_metrics()
- toil.lib.humanize
- toil.lib.io
- Attributes
- Classes
- Functions
- Module Contents
loggerget_toil_home()TOIL_URI_SCHEMESTANDARD_SCHEMESREMOTE_SCHEMESALL_SCHEMESis_standard_url()is_remote_url()is_any_url()is_url_with_scheme()is_toil_url()is_toil_file_url()is_toil_dir_url()is_file_url()is_directory_url()mkdtemp()robust_rmtree()atomic_tmp_file()atomic_install()AtomicFileCreate()atomic_copy()atomic_copyobj()make_public_dir()try_path()WriteWatchingStreamReadableFileObjfile_digest()
- toil.lib.memoize
- toil.lib.misc
- toil.lib.objects
- toil.lib.pipes
- toil.lib.plugins
- toil.lib.resources
- toil.lib.retry
- Attributes
- Classes
- Functions
- Module Contents
SUPPORTED_HTTP_ERRORSkubernetesbotocoreloggerErrorConditionRTretry()return_status_code()get_error_code()get_error_message()get_error_status()get_error_body()meets_error_message_condition()meets_error_code_condition()meets_boto_error_code_condition()error_meets_conditions()DEFAULT_DELAYSDEFAULT_TIMEOUTEold_retry()retry_flaky_test
- toil.lib.threading
- Attributes
- Classes
- Functions
- Module Contents
- toil.lib.throttle
- toil.lib.trs
- toil.lib.url
- toil.lib.web
- Submodules
- toil.options
- toil.provisioners
- Submodules
- toil.provisioners.abstractProvisioner
- Attributes
- Exceptions
- Classes
- Module Contents
a_short_timeloggerManagedNodesNotSupportedExceptionShapeAbstractProvisionerAbstractProvisioner.LEADER_HOME_DIRAbstractProvisioner.cloudAbstractProvisioner.clusterNameAbstractProvisioner.clusterTypeAbstractProvisioner.enable_fuseAbstractProvisioner.supportedClusterTypes()AbstractProvisioner.createClusterSettings()AbstractProvisioner.readClusterSettings()AbstractProvisioner.setAutoscaledNodeTypes()AbstractProvisioner.hasAutoscaledNodeTypes()AbstractProvisioner.getAutoscaledInstanceShapes()AbstractProvisioner.retryPredicate()AbstractProvisioner.launchCluster()AbstractProvisioner.addNodes()AbstractProvisioner.addManagedNodes()AbstractProvisioner.terminateNodes()AbstractProvisioner.getLeader()AbstractProvisioner.getProvisionedWorkers()AbstractProvisioner.getNodeShape()AbstractProvisioner.destroyCluster()AbstractProvisioner.InstanceConfigurationAbstractProvisioner.InstanceConfiguration.filesAbstractProvisioner.InstanceConfiguration.unitsAbstractProvisioner.InstanceConfiguration.sshPublicKeysAbstractProvisioner.InstanceConfiguration.addFile()AbstractProvisioner.InstanceConfiguration.addUnit()AbstractProvisioner.InstanceConfiguration.addSSHRSAKey()AbstractProvisioner.InstanceConfiguration.toIgnitionConfig()
AbstractProvisioner.getBaseInstanceConfiguration()AbstractProvisioner.addVolumesService()AbstractProvisioner.addNodeExporterService()AbstractProvisioner.toil_service_env_options()AbstractProvisioner.add_toil_service()AbstractProvisioner.getKubernetesValues()AbstractProvisioner.addKubernetesServices()AbstractProvisioner.getKubernetesAutoscalerSetupCommands()AbstractProvisioner.getKubernetesCloudProvider()AbstractProvisioner.addKubernetesLeader()AbstractProvisioner.addKubernetesWorker()
- toil.provisioners.aws
- Submodules
- toil.provisioners.aws.awsProvisioner
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerawsRetryPredicate()expectedShutdownErrors()FawsRetry()awsFilterImpairedNodes()InvalidClusterStateExceptioncollapse_tags()AWSProvisionerAWSProvisioner.cloudAWSProvisioner.awsAWSProvisioner.s3_bucket_nameAWSProvisioner.supportedClusterTypes()AWSProvisioner.createClusterSettings()AWSProvisioner.readClusterSettings()AWSProvisioner.launchCluster()AWSProvisioner.toil_service_env_options()AWSProvisioner.getKubernetesAutoscalerSetupCommands()AWSProvisioner.getKubernetesCloudProvider()AWSProvisioner.getNodeShape()AWSProvisioner.retryPredicate()AWSProvisioner.destroyCluster()AWSProvisioner.terminateNodes()AWSProvisioner.addNodes()AWSProvisioner.addManagedNodes()AWSProvisioner.getProvisionedWorkers()AWSProvisioner.getLeader()AWSProvisioner.full_policy()AWSProvisioner.kubernetes_policy()
- toil.provisioners.aws.awsProvisioner
- Attributes
- Classes
- Functions
- Package Contents
- Submodules
- toil.provisioners.clusterScaler
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerEVICTION_THRESHOLDRESERVE_SMALL_LIMITRESERVE_SMALL_AMOUNTRESERVE_BREAKPOINTSRESERVE_FRACTIONSOS_SIZEFailedConstraintBinPackedFitNodeReservationadjustEndingReservationForJob()split()binPacking()ClusterScalerClusterScaler.provisionerClusterScaler.leaderClusterScaler.configClusterScaler.staticClusterScaler.on_too_bigClusterScaler.jobNameToAvgRuntimeClusterScaler.jobNameToNumCompletedClusterScaler.totalAvgRuntimeClusterScaler.totalJobsCompletedClusterScaler.targetTimeClusterScaler.betaInertiaClusterScaler.nodeShapeToTypeClusterScaler.instance_typesClusterScaler.nodeShapesClusterScaler.ignoredNodesClusterScaler.preemptibleNodeDeficitClusterScaler.previousWeightedEstimateClusterScaler.minNodesClusterScaler.maxNodesClusterScaler.node_shapes_after_overheadClusterScaler.without_overheadClusterScaler.getAverageRuntime()ClusterScaler.addCompletedJob()ClusterScaler.setStaticNodes()ClusterScaler.getStaticNodes()ClusterScaler.smoothEstimate()ClusterScaler.getEstimatedNodeCounts()ClusterScaler.updateClusterSize()ClusterScaler.setNodeCount()ClusterScaler.filter_out_static_nodes()ClusterScaler.getNodes()ClusterScaler.shutDown()
JobTooBigErrorScalerThreadClusterStats
- toil.provisioners.gceProvisioner
- Attributes
- Classes
- Module Contents
loggerGCEProvisionerGCEProvisioner.NODE_BOTO_PATHGCEProvisioner.SOURCE_IMAGEGCEProvisioner.cloudGCEProvisioner.supportedClusterTypes()GCEProvisioner.createClusterSettings()GCEProvisioner.readClusterSettings()GCEProvisioner.launchCluster()GCEProvisioner.getNodeShape()GCEProvisioner.retryPredicate()GCEProvisioner.destroyCluster()GCEProvisioner.terminateNodes()GCEProvisioner.addNodes()GCEProvisioner.getProvisionedWorkers()GCEProvisioner.getLeader()GCEProvisioner.DEFAULT_TASK_COMPLETION_TIMEOUTGCEProvisioner.ex_create_multiple_nodes()
- toil.provisioners.node
- Attributes
- Classes
- Module Contents
a_short_timeloggerNodeNode.maxWaitTimeNode.publicIPNode.privateIPNode.nameNode.nodeTypeNode.preemptibleNode.tagsNode.__str__()Node.__repr__()Node.__hash__()Node.remainingBillingInterval()Node.waitForNode()Node.copySshKeys()Node.injectFile()Node.extractFile()Node.sshAppliance()Node.sshInstance()Node.coreSSH()Node.coreRsync()
- toil.provisioners.abstractProvisioner
- Attributes
- Exceptions
- Functions
- Package Contents
- Submodules
- toil.realtimeLogger
- toil.resource
- Attributes
- Exceptions
- Classes
- Module Contents
loggerResourceFileResourceDirectoryResourceVirtualEnvResourceModuleDescriptorModuleDescriptor.dirPathModuleDescriptor.nameModuleDescriptor.forModule()ModuleDescriptor.belongsToToilModuleDescriptor.saveAsResourceTo()ModuleDescriptor.localize()ModuleDescriptor.globalize()ModuleDescriptor.toCommand()ModuleDescriptor.fromCommand()ModuleDescriptor.makeLoadable()ModuleDescriptor.load()
ResourceException
- toil.server
- Submodules
- toil.server.api_spec
- toil.server.app
- toil.server.celery_app
- toil.server.cli
- toil.server.utils
- Attributes
- Classes
- Functions
- Module Contents
HAVE_S3loggerget_iso_time()link_file()download_file_from_internet()download_file_from_s3()get_file_class()safe_read_file()safe_write_file()MemoryStateCacheAbstractStateStoreMemoryStateStoreFileStateStoreS3StateStorestate_store_cacheconnect_to_state_store()WorkflowStateStoreconnect_to_workflow_state_store()TERMINAL_STATESMAX_CANCELING_SECONDSWorkflowStateMachineWorkflowStateMachine.send_enqueue()WorkflowStateMachine.send_initialize()WorkflowStateMachine.send_run()WorkflowStateMachine.send_cancel()WorkflowStateMachine.send_canceled()WorkflowStateMachine.send_complete()WorkflowStateMachine.send_executor_error()WorkflowStateMachine.send_system_error()WorkflowStateMachine.get_current_state()
- toil.server.wes
- Submodules
- toil.server.wes.abstract_backend
- toil.server.wes.amazon_wes_utils
- toil.server.wes.tasks
- Attributes
- Classes
- Functions
- Module Contents
loggerWAIT_FOR_DEATH_TIMEOUTToilWorkflowRunnerToilWorkflowRunner.scratch_dirToilWorkflowRunner.storeToilWorkflowRunner.state_machineToilWorkflowRunner.requestToilWorkflowRunner.engine_optionsToilWorkflowRunner.wf_typeToilWorkflowRunner.versionToilWorkflowRunner.exec_dirToilWorkflowRunner.out_dirToilWorkflowRunner.default_job_storeToilWorkflowRunner.job_storeToilWorkflowRunner.write_scratch_file()ToilWorkflowRunner.get_state()ToilWorkflowRunner.write_workflow()ToilWorkflowRunner.sort_options()ToilWorkflowRunner.initialize_run()ToilWorkflowRunner.call_cmd()ToilWorkflowRunner.run()ToilWorkflowRunner.write_output_files()
run_wes_task()run_wescancel_run()TaskRunnerMultiprocessingTaskRunner
- toil.server.wes.toil_backend
- Attributes
- Classes
- Module Contents
loggerToilWorkflowToilWorkflow.base_scratch_dirToilWorkflow.state_store_urlToilWorkflow.run_idToilWorkflow.scratch_dirToilWorkflow.exec_dirToilWorkflow.storeToilWorkflow.state_machineToilWorkflow.fetch_state()ToilWorkflow.fetch_scratch()ToilWorkflow.exists()ToilWorkflow.get_state()ToilWorkflow.check_on_run()ToilWorkflow.set_up_run()ToilWorkflow.clean_up()ToilWorkflow.queue_run()ToilWorkflow.get_output_files()ToilWorkflow.get_stdout_path()ToilWorkflow.get_stderr_path()ToilWorkflow.get_messages_path()ToilWorkflow.get_task_logs()
ToilBackendToilBackend.run_id_prefixToilBackend.task_runnerToilBackend.wes_dialectToilBackend.dest_bucket_baseToilBackend.work_dirToilBackend.supported_versionsToilBackend.get_runs()ToilBackend.get_state()ToilBackend.get_service_info()ToilBackend.list_runs()ToilBackend.run_workflow()ToilBackend.get_run_log()ToilBackend.cancel_run()ToilBackend.get_run_status()ToilBackend.get_stdout()ToilBackend.get_stderr()ToilBackend.get_health()ToilBackend.get_homepage()
- Submodules
- toil.server.wsgi_app
- Submodules
- toil.serviceManager
- Attributes
- Classes
- Module Contents
loggerServiceManagerServiceManager.services_are_starting()ServiceManager.get_job_count()ServiceManager.start()ServiceManager.put_client()ServiceManager.get_ready_client()ServiceManager.get_unservable_client()ServiceManager.get_startable_service()ServiceManager.kill_services()ServiceManager.is_active()ServiceManager.is_running()ServiceManager.check()ServiceManager.shutdown()
- toil.statsAndLogging
- toil.test
- Submodules
- toil.test.batchSystems
- Submodules
- toil.test.batchSystems.batchSystemTest
- Attributes
- Classes
- Functions
- Module Contents
loggernumCorespreemptibledefaultRequirementsBatchSystemPluginTesthiddenhidden.AbstractBatchSystemTesthidden.AbstractBatchSystemTest.createBatchSystem()hidden.AbstractBatchSystemTest.supportsWallTime()hidden.AbstractBatchSystemTest.createConfig()hidden.AbstractBatchSystemTest.setUpClass()hidden.AbstractBatchSystemTest.setUp()hidden.AbstractBatchSystemTest.tearDown()hidden.AbstractBatchSystemTest.get_max_startup_seconds()hidden.AbstractBatchSystemTest.test_available_cores()hidden.AbstractBatchSystemTest.test_run_jobs()hidden.AbstractBatchSystemTest.test_set_env()hidden.AbstractBatchSystemTest.test_set_job_env()hidden.AbstractBatchSystemTest.testCheckResourceRequest()hidden.AbstractBatchSystemTest.testScalableBatchSystem()
hidden.AbstractGridEngineBatchSystemTest
AbstractBatchSystemJobTestKubernetesBatchSystemTestKubernetesBatchSystemBenchTestAWSBatchBatchSystemTestMesosBatchSystemTestwrite_temp_file()SingleMachineBatchSystemTestMaxCoresSingleMachineBatchSystemTestparentJob()childJob()grandChildJob()greatGrandChild()ServiceGridEngineBatchSystemTestSlurmBatchSystemTestLSFBatchSystemTestTorqueBatchSystemTestHTCondorBatchSystemTestTestSingleMachineBatchSystemJobTestMesosBatchSystemJobmeasureConcurrency()count()getCounters()resetCounters()get_omp_threads()
- toil.test.batchSystems.batch_system_plugin_test
- Attributes
- Classes
- Module Contents
loggerFakeBatchSystemFakeBatchSystem.supportsAutoDeployment()FakeBatchSystem.issueBatchJob()FakeBatchSystem.killBatchJobs()FakeBatchSystem.getIssuedBatchJobIDs()FakeBatchSystem.getRunningBatchJobIDs()FakeBatchSystem.getUpdatedBatchJob()FakeBatchSystem.shutdown()FakeBatchSystem.add_options()FakeBatchSystem.setOptions()
BatchSystemPluginTest
- toil.test.batchSystems.test_gridengine
- toil.test.batchSystems.test_lsf_helper
- toil.test.batchSystems.test_slurm
- Attributes
- Classes
- Functions
- Module Contents
loggerJOB_BASE_TIMEcall_either()call_sacct()call_scontrol()call_sacct_raises()call_sinfo()FakeBatchSystemSlurmTestSlurmTest.setUp()SlurmTest.test_getJobDetailsFromSacct_one_exists()SlurmTest.test_getJobDetailsFromSacct_one_not_exists()SlurmTest.test_getJobDetailsFromSacct_many_all_exist()SlurmTest.test_getJobDetailsFromSacct_many_some_exist()SlurmTest.test_getJobDetailsFromSacct_many_none_exist()SlurmTest.test_getJobDetailsFromSacct_argument_list_too_big()SlurmTest.test_getJobDetailsFromScontrol_one_exists()SlurmTest.test_getJobDetailsFromScontrol_one_not_exists()SlurmTest.test_getJobDetailsFromScontrol_many_all_exist()SlurmTest.test_getJobDetailsFromScontrol_many_some_exist()SlurmTest.test_getJobDetailsFromScontrol_many_none_exist()SlurmTest.test_getJobExitCode_job_exists()SlurmTest.test_getJobExitCode_job_not_exists()SlurmTest.test_getJobExitCode_sacct_raises_job_exists()SlurmTest.test_getJobExitCode_sacct_raises_job_not_exists()SlurmTest.test_coalesce_job_exit_codes_one_exists()SlurmTest.test_coalesce_job_exit_codes_one_not_exists()SlurmTest.test_coalesce_job_exit_codes_many_all_exist()SlurmTest.test_coalesce_job_exit_codes_mix_sacct_scontrol()SlurmTest.test_coalesce_job_exit_codes_some_exists()SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_exists()SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_not_exists()SlurmTest.test_PartitionSet_get_partition()SlurmTest.test_PartitionSet_default_gpu_partition()SlurmTest.test_prepareSbatch_partition()SlurmTest.test_prepareSbatch_time()SlurmTest.test_prepareSbatch_export()SlurmTest.test_option_detector()SlurmTest.test_any_option_detector()
- toil.test.batchSystems.batchSystemTest
- Submodules
- toil.test.cactus
- toil.test.conftest
- toil.test.cwl
- Submodules
- toil.test.cwl.conftest
- toil.test.cwl.cwlTest
- Attributes
- Classes
- Functions
- Module Contents
pkg_rootlogCONFORMANCE_TEST_TIMEOUTrun_conformance_tests()TesterFuncTypeTestCWLWorkflowTestCWLWorkflow.test_cwl_cmdline_input()TestCWLWorkflow.revsort()TestCWLWorkflow.revsort_no_checksum()TestCWLWorkflow.download()TestCWLWorkflow.load_contents()TestCWLWorkflow.download_directory()TestCWLWorkflow.download_subdirectory()TestCWLWorkflow.test_mpi()TestCWLWorkflow.test_s3_as_secondary_file()TestCWLWorkflow.test_run_revsort()TestCWLWorkflow.test_run_revsort_nochecksum()TestCWLWorkflow.test_run_revsort_no_container()TestCWLWorkflow.test_run_revsort2()TestCWLWorkflow.test_run_revsort_debug_worker()TestCWLWorkflow.test_run_colon_output()TestCWLWorkflow.test_run_dockstore_trs()TestCWLWorkflow.test_glob_dir_bypass_file_store()TestCWLWorkflow.test_required_input_condition_protection()TestCWLWorkflow.test_slurm_node_memory()TestCWLWorkflow.test_download_s3()TestCWLWorkflow.test_download_http()TestCWLWorkflow.test_download_https()TestCWLWorkflow.test_download_https_reference()TestCWLWorkflow.test_download_file()TestCWLWorkflow.test_download_file_worker_import()TestCWLWorkflow.test_download_file_uri()TestCWLWorkflow.test_download_file_uri_worker_import()TestCWLWorkflow.test_download_file_uri_no_hostname()TestCWLWorkflow.test_download_file_uri_no_hostname_worker_import()TestCWLWorkflow.test_download_directory_s3()TestCWLWorkflow.test_download_directory_s3_reference()TestCWLWorkflow.test_download_directory_file()TestCWLWorkflow.test_download_subdirectory_s3()TestCWLWorkflow.test_download_subdirectory_file()TestCWLWorkflow.test_load_contents_s3()TestCWLWorkflow.test_load_contents_http()TestCWLWorkflow.test_load_contents_https()TestCWLWorkflow.test_load_contents_file()TestCWLWorkflow.test_bioconda()TestCWLWorkflow.test_default_args()TestCWLWorkflow.test_biocontainers()TestCWLWorkflow.test_cuda()TestCWLWorkflow.test_restart()TestCWLWorkflow.test_caching()TestCWLWorkflow.test_streamable()TestCWLWorkflow.test_streamable_reference()TestCWLWorkflow.test_preemptible()TestCWLWorkflow.test_preemptible_expression()TestCWLWorkflow.test_missing_import()TestCWLWorkflow.test_optional_secondary_files_exists()TestCWLWorkflow.test_optional_secondary_files_missing()
cwl_v1_0_spec()TestCWLv10ConformanceTestCWLv10Conformance.test_run_conformance_with_caching()TestCWLv10Conformance.test_run_conformance()TestCWLv10Conformance.test_lsf_cwl_conformance()TestCWLv10Conformance.test_slurm_cwl_conformance()TestCWLv10Conformance.test_torque_cwl_conformance()TestCWLv10Conformance.test_gridengine_cwl_conformance()TestCWLv10Conformance.test_mesos_cwl_conformance()TestCWLv10Conformance.test_kubernetes_cwl_conformance()TestCWLv10Conformance.test_lsf_cwl_conformance_with_caching()TestCWLv10Conformance.test_slurm_cwl_conformance_with_caching()TestCWLv10Conformance.test_torque_cwl_conformance_with_caching()TestCWLv10Conformance.test_gridengine_cwl_conformance_with_caching()TestCWLv10Conformance.test_mesos_cwl_conformance_with_caching()TestCWLv10Conformance.test_kubernetes_cwl_conformance_with_caching()
cwl_v1_1_spec()TestCWLv11Conformancecwl_v1_2_spec()TestCWLv12ConformanceTestCWLv12Conformance.test_run_conformance()TestCWLv12Conformance.test_run_conformance_with_caching()TestCWLv12Conformance.test_run_conformance_with_task_caching()TestCWLv12Conformance.test_run_conformance_with_in_place_update()TestCWLv12Conformance.test_kubernetes_cwl_conformance()TestCWLv12Conformance.test_kubernetes_cwl_conformance_with_caching()TestCWLv12Conformance.test_wes_server_cwl_conformance()
test_workflow_echo_string_scatter_stderr_log_dir()test_log_dir_echo_no_output()test_log_dir_echo_stderr()test_filename_conflict_resolution()test_filename_conflict_resolution_3_or_more()test_filename_conflict_detection()test_filename_conflict_detection_at_root()test_pick_value_with_one_null_value()test_workflow_echo_string()test_workflow_echo_string_scatter_capture_stdout()test_visit_top_cwl_class()test_visit_cwl_class_and_reduce()test_trim_mounts_op_nonredundant()test_trim_mounts_op_redundant()test_trim_mounts_op_partially_redundant()test_trim_mounts_op_mixed_urls_and_paths()test_trim_mounts_op_decodable_paths()test_trim_mounts_op_multiple_encodings()test_download_structure()test_import_on_workers()test_missing_tmpdir_and_tmp_outdir()ImportWorkersMessageHandler
- Submodules
- toil.test.docs
- Submodules
- toil.test.docs.scriptsTest
- Attributes
- Classes
- Module Contents
pkg_rootToilDocumentationTestToilDocumentationTest.setUpClass()ToilDocumentationTest.tearDown()ToilDocumentationTest.checkExitCode()ToilDocumentationTest.checkExpectedOut()ToilDocumentationTest.checkExpectedPattern()ToilDocumentationTest.testStats()ToilDocumentationTest.testDynamic()ToilDocumentationTest.testEncapsulation()ToilDocumentationTest.testEncapsulation2()ToilDocumentationTest.testHelloworld()ToilDocumentationTest.testInvokeworkflow()ToilDocumentationTest.testInvokeworkflow2()ToilDocumentationTest.testJobFunctions()ToilDocumentationTest.testManaging()ToilDocumentationTest.testManaging2()ToilDocumentationTest.testMultiplejobs()ToilDocumentationTest.testMultiplejobs2()ToilDocumentationTest.testMultiplejobs3()ToilDocumentationTest.testPromises2()ToilDocumentationTest.testQuickstart()ToilDocumentationTest.testRequirements()ToilDocumentationTest.testArguments()ToilDocumentationTest.testDocker()ToilDocumentationTest.testPromises()ToilDocumentationTest.testServices()ToilDocumentationTest.testStaging()
- toil.test.docs.scriptsTest
- Submodules
- toil.test.jobStores
- Submodules
- toil.test.jobStores.jobStoreTest
- Attributes
- Classes
- Functions
- Module Contents
google_retry()loggertearDownModule()AbstractJobStoreTestAbstractJobStoreTest.TestAbstractJobStoreTest.Test.setUpClass()AbstractJobStoreTest.Test.setUp()AbstractJobStoreTest.Test.tearDown()AbstractJobStoreTest.Test.testInitialState()AbstractJobStoreTest.Test.testJobCreation()AbstractJobStoreTest.Test.testConfigEquality()AbstractJobStoreTest.Test.testJobLoadEquality()AbstractJobStoreTest.Test.testChildLoadingEquality()AbstractJobStoreTest.Test.testPersistantFilesToDelete()AbstractJobStoreTest.Test.testUpdateBehavior()AbstractJobStoreTest.Test.testJobDeletions()AbstractJobStoreTest.Test.testSharedFiles()AbstractJobStoreTest.Test.testReadWriteSharedFilesTextMode()AbstractJobStoreTest.Test.testReadWriteFileStreamTextMode()AbstractJobStoreTest.Test.testPerJobFiles()AbstractJobStoreTest.Test.testStatsAndLogging()AbstractJobStoreTest.Test.testWriteLogFiles()AbstractJobStoreTest.Test.testBatchCreate()AbstractJobStoreTest.Test.testGrowingAndShrinkingJob()AbstractJobStoreTest.Test.externalStoreCacheAbstractJobStoreTest.Test.cleanUpExternalStores()AbstractJobStoreTest.Test.mpTestPartSizeAbstractJobStoreTest.Test.makeImportExportTests()AbstractJobStoreTest.Test.testImportHttpFile()AbstractJobStoreTest.Test.testImportFtpFile()AbstractJobStoreTest.Test.testFileDeletion()AbstractJobStoreTest.Test.testMultipartUploads()AbstractJobStoreTest.Test.testZeroLengthFiles()AbstractJobStoreTest.Test.testLargeFile()AbstractJobStoreTest.Test.fetch_url()AbstractJobStoreTest.Test.assertUrl()AbstractJobStoreTest.Test.testCleanCache()AbstractJobStoreTest.Test.testPartialReadFromStream()AbstractJobStoreTest.Test.testDestructionOfCorruptedJobStore()AbstractJobStoreTest.Test.testDestructionIdempotence()AbstractJobStoreTest.Test.testEmptyFileStoreIDIsReadable()
AbstractEncryptedJobStoreTestFileJobStoreTestGoogleJobStoreTestAWSJobStoreTestInvalidAWSJobStoreTestEncryptedAWSJobStoreTestStubHttpRequestHandler
- toil.test.jobStores.jobStoreTest
- Submodules
- toil.test.lib
- Submodules
- toil.test.lib.aws
- toil.test.lib.dockerTest
- Attributes
- Classes
- Module Contents
loggerTestDockerTestDocker.dockerTestLogLevelTestDocker.testDockerClean()TestDocker.testDockerClean_CRx_FORGO()TestDocker.testDockerClean_CRx_STOP()TestDocker.testDockerClean_CRx_RM()TestDocker.testDockerClean_CRx_None()TestDocker.testDockerClean_CxD_FORGO()TestDocker.testDockerClean_CxD_STOP()TestDocker.testDockerClean_CxD_RM()TestDocker.testDockerClean_CxD_None()TestDocker.testDockerClean_Cxx_FORGO()TestDocker.testDockerClean_Cxx_STOP()TestDocker.testDockerClean_Cxx_RM()TestDocker.testDockerClean_Cxx_None()TestDocker.testDockerClean_xRx_FORGO()TestDocker.testDockerClean_xRx_STOP()TestDocker.testDockerClean_xRx_RM()TestDocker.testDockerClean_xRx_None()TestDocker.testDockerClean_xxD_FORGO()TestDocker.testDockerClean_xxD_STOP()TestDocker.testDockerClean_xxD_RM()TestDocker.testDockerClean_xxD_None()TestDocker.testDockerClean_xxx_FORGO()TestDocker.testDockerClean_xxx_STOP()TestDocker.testDockerClean_xxx_RM()TestDocker.testDockerClean_xxx_None()TestDocker.testDockerPipeChain()TestDocker.testDockerPipeChainErrorDetection()TestDocker.testNonCachingDockerChain()TestDocker.testNonCachingDockerChainErrorDetection()TestDocker.testDockerLogs()TestDocker.testDockerLogs_Stream()TestDocker.testDockerLogs_Demux()TestDocker.testDockerLogs_Demux_Stream()
- toil.test.lib.test_conversions
- toil.test.lib.test_ec2
- toil.test.lib.test_history
- toil.test.lib.test_misc
- toil.test.lib.test_trs
- Attributes
- Classes
- Module Contents
loggerDockstoreLookupTestDockstoreLookupTest.read_result()DockstoreLookupTest.test_lookup_from_page_url()DockstoreLookupTest.test_lookup_from_trs_with_version()DockstoreLookupTest.test_lookup_from_trs_no_version()DockstoreLookupTest.test_get()DockstoreLookupTest.test_get_from_trs_cached()DockstoreLookupTest.test_lookup_from_trs_nonexistent_workflow()DockstoreLookupTest.test_lookup_from_trs_nonexistent_workflow_bad_format()DockstoreLookupTest.test_lookup_from_trs_nonexistent_version()DockstoreLookupTest.test_get_nonexistent_workflow()DockstoreLookupTest.test_get_nonexistent_version()DockstoreLookupTest.test_get_nonexistent_workflow_bad_format()
- toil.test.lib.test_url
- toil.test.lib.url_plugin_test
- Submodules
- toil.test.mesos
- toil.test.options
- toil.test.provisioners
- Submodules
- toil.test.provisioners.aws
- Submodules
- toil.test.provisioners.aws.awsProvisionerTest
- Attributes
- Classes
- Functions
- Module Contents
logaws_zone()aws_region()ec2_client()TestAWSProvisionerBenchTestAbstractAWSAutoscaleTestAbstractAWSAutoscaleTest.instanceTypesAbstractAWSAutoscaleTest.clusterNameAbstractAWSAutoscaleTest.numWorkersAbstractAWSAutoscaleTest.numSamplesAbstractAWSAutoscaleTest.spotBidAbstractAWSAutoscaleTest.scriptDirAbstractAWSAutoscaleTest.venvDirAbstractAWSAutoscaleTest.dataDirAbstractAWSAutoscaleTest.scriptNameAbstractAWSAutoscaleTest.script()AbstractAWSAutoscaleTest.data()AbstractAWSAutoscaleTest.rsyncUtil()AbstractAWSAutoscaleTest.getRootVolID()AbstractAWSAutoscaleTest.putScript()
AWSAutoscaleTestAWSAutoscaleTest.clusterNameAWSAutoscaleTest.requestedLeaderStorageAWSAutoscaleTest.scriptNameAWSAutoscaleTest.setUp()AWSAutoscaleTest.launchCluster()AWSAutoscaleTest.getRootVolID()AWSAutoscaleTest.testAutoScale()AWSAutoscaleTest.testSpotAutoScale()AWSAutoscaleTest.testSpotAutoScaleBalancingTypes()
AWSStaticAutoscaleTestAWSManagedAutoscaleTestAWSAutoscaleTestMultipleNodeTypesAWSRestartTestPreemptibleDeficitCompensationTest
- toil.test.provisioners.aws.awsProvisionerTest
- Submodules
- toil.test.provisioners.clusterScalerTest
- Attributes
- Classes
- Module Contents
loggerc4_8xlarge_preemptiblec4_8xlarger3_8xlarger5_2xlarger5_4xlarget2_microBinPackingTestBinPackingTest.setUp()BinPackingTest.testPackingOneShape()BinPackingTest.testSorting()BinPackingTest.testAddingInitialNode()BinPackingTest.testLowTargetTime()BinPackingTest.testHighTargetTime()BinPackingTest.testZeroResourceJobs()BinPackingTest.testLongRunningJobs()BinPackingTest.run1000JobsOnMicros()BinPackingTest.testPathologicalCase()BinPackingTest.testJobTooLargeForAllNodes()
ClusterScalerTestClusterScalerTest.setUp()ClusterScalerTest.testRounding()ClusterScalerTest.testMaxNodes()ClusterScalerTest.testMinNodes()ClusterScalerTest.testPreemptibleDeficitResponse()ClusterScalerTest.testPreemptibleDeficitIsSet()ClusterScalerTest.testNoLaunchingIfDeltaAlreadyMet()ClusterScalerTest.testBetaInertia()ClusterScalerTest.test_overhead_accounting_large()ClusterScalerTest.test_overhead_accounting_small()ClusterScalerTest.test_overhead_accounting_observed()
ScalerThreadTestMockBatchSystemAndProvisionerMockBatchSystemAndProvisioner.configMockBatchSystemAndProvisioner.secondsPerJobMockBatchSystemAndProvisioner.provisionerMockBatchSystemAndProvisioner.batchSystemMockBatchSystemAndProvisioner.jobQueueMockBatchSystemAndProvisioner.updatedJobsQueueMockBatchSystemAndProvisioner.jobBatchSystemIDToIssuedJobMockBatchSystemAndProvisioner.totalJobsMockBatchSystemAndProvisioner.totalWorkerTimeMockBatchSystemAndProvisioner.toilMetricsMockBatchSystemAndProvisioner.nodesToWorkerMockBatchSystemAndProvisioner.workersMockBatchSystemAndProvisioner.maxWorkersMockBatchSystemAndProvisioner.runningMockBatchSystemAndProvisioner.leaderThreadMockBatchSystemAndProvisioner.toilStateMockBatchSystemAndProvisioner.start()MockBatchSystemAndProvisioner.shutDown()MockBatchSystemAndProvisioner.nodeInUse()MockBatchSystemAndProvisioner.ignoreNode()MockBatchSystemAndProvisioner.unignoreNode()MockBatchSystemAndProvisioner.supportedClusterTypes()MockBatchSystemAndProvisioner.createClusterSettings()MockBatchSystemAndProvisioner.readClusterSettings()MockBatchSystemAndProvisioner.setAutoscaledNodeTypes()MockBatchSystemAndProvisioner.getProvisionedWorkers()MockBatchSystemAndProvisioner.terminateNodes()MockBatchSystemAndProvisioner.remainingBillingInterval()MockBatchSystemAndProvisioner.addJob()MockBatchSystemAndProvisioner.getNumberOfJobsIssued()MockBatchSystemAndProvisioner.getJobs()MockBatchSystemAndProvisioner.getNodes()MockBatchSystemAndProvisioner.addNodes()MockBatchSystemAndProvisioner.getNodeShape()MockBatchSystemAndProvisioner.getWorkersInCluster()MockBatchSystemAndProvisioner.launchCluster()MockBatchSystemAndProvisioner.destroyCluster()MockBatchSystemAndProvisioner.getLeader()MockBatchSystemAndProvisioner.getNumberOfNodes()
- toil.test.provisioners.clusterTest
- Attributes
- Classes
- Module Contents
logAbstractClusterTestAbstractClusterTest.keyNameAbstractClusterTest.clusterNameAbstractClusterTest.leaderNodeTypeAbstractClusterTest.clusterTypeAbstractClusterTest.zoneAbstractClusterTest.regionAbstractClusterTest.awsAbstractClusterTest.venvDirAbstractClusterTest.python()AbstractClusterTest.pip()AbstractClusterTest.destroyCluster()AbstractClusterTest.setUp()AbstractClusterTest.tearDown()AbstractClusterTest.sshUtil()AbstractClusterTest.rsync_util()AbstractClusterTest.createClusterUtil()AbstractClusterTest.launchCluster()
CWLOnARMTest
- toil.test.provisioners.gceProvisionerTest
- Attributes
- Classes
- Module Contents
logAbstractGCEAutoscaleTestAbstractGCEAutoscaleTest.projectIDAbstractGCEAutoscaleTest.sshUtil()AbstractGCEAutoscaleTest.rsyncUtil()AbstractGCEAutoscaleTest.destroyClusterUtil()AbstractGCEAutoscaleTest.createClusterUtil()AbstractGCEAutoscaleTest.cleanJobStoreUtil()AbstractGCEAutoscaleTest.keyNameAbstractGCEAutoscaleTest.botoDirAbstractGCEAutoscaleTest.googleZoneAbstractGCEAutoscaleTest.leaderInstanceTypeAbstractGCEAutoscaleTest.instanceTypesAbstractGCEAutoscaleTest.numWorkersAbstractGCEAutoscaleTest.numSamplesAbstractGCEAutoscaleTest.spotBidAbstractGCEAutoscaleTest.setUp()AbstractGCEAutoscaleTest.tearDown()AbstractGCEAutoscaleTest.launchCluster()
GCEAutoscaleTestGCEStaticAutoscaleTestGCEAutoscaleTestMultipleNodeTypesGCERestartTest
- toil.test.provisioners.provisionerTest
- toil.test.provisioners.restartScript
- toil.test.provisioners.aws
- Submodules
- toil.test.server
- Submodules
- toil.test.server.serverTest
- Attributes
- Classes
- Module Contents
loggerToilServerUtilsTesthiddenFileStateStoreTestFileStateStoreURLTestBucketUsingTestAWSStateStoreTestTrueDictAbstractToilWESServerTestToilWESServerBenchTestToilWESServerWorkflowTestToilWESServerWorkflowTest.run_zip_workflow()ToilWESServerWorkflowTest.test_run_workflow_relative_url_no_attachments_fails()ToilWESServerWorkflowTest.test_run_workflow_relative_url()ToilWESServerWorkflowTest.test_run_workflow_https_url()ToilWESServerWorkflowTest.test_run_workflow_single_file_zip()ToilWESServerWorkflowTest.test_run_workflow_multi_file_zip()ToilWESServerWorkflowTest.test_run_workflow_manifest_zip()ToilWESServerWorkflowTest.test_run_workflow_inputs_zip()ToilWESServerWorkflowTest.test_run_workflow_manifest_and_inputs_zip()ToilWESServerWorkflowTest.test_run_workflow_no_params_zip()ToilWESServerWorkflowTest.test_run_and_cancel_workflows()
ToilWESServerCeleryWorkflowTestToilWESServerCeleryS3StateWorkflowTest
- toil.test.server.serverTest
- Submodules
- toil.test.sort
- Submodules
- toil.test.sort.restart_sort
- toil.test.sort.sort
- toil.test.sort.sortTest
- Attributes
- Classes
- Functions
- Module Contents
loggerdefaultLineLendefaultLinesdefaultNrunMain()SortTestSortTest.setUp()SortTest.tearDown()SortTest.testAwsSingle()SortTest.testAwsMesos()SortTest.testFileMesos()SortTest.testGoogleSingle()SortTest.testGoogleMesos()SortTest.testFileSingle()SortTest.testFileSingleNonCaching()SortTest.testFileSingleCheckpoints()SortTest.testFileSingle10000()SortTest.testFileGridEngine()SortTest.testFileTorqueEngine()SortTest.testNoSortTest.testSort()SortTest.testMerge()SortTest.testCopySubRangeOfFile()SortTest.testGetMidPoint()
- Submodules
- toil.test.src
- Submodules
- toil.test.src.autoDeploymentTest
- Attributes
- Classes
- Module Contents
loggerAutoDeploymentTestAutoDeploymentTest.setUp()AutoDeploymentTest.sitePackagesAutoDeploymentTest.testRestart()AutoDeploymentTest.testSplitRootPackages()AutoDeploymentTest.testUserTypesInJobFunctionArgs()AutoDeploymentTest.testDeferralWithConcurrentEncapsulation()AutoDeploymentTest.testDeferralWithFailureAndEncapsulation()
- toil.test.src.busTest
- toil.test.src.checkpointTest
- toil.test.src.deferredFunctionTest
- Attributes
- Classes
- Functions
- Module Contents
loggeroptions()TestDeferredFunctionTestDeferredFunction.testDeferredFunctionRunsWithMethod()TestDeferredFunction.testDeferredFunctionRunsWithClassMethod()TestDeferredFunction.testDeferredFunctionRunsWithLambda()TestDeferredFunction.testDeferredFunctionRunsWithFailures()TestDeferredFunction.testNewJobsCanHandleOtherJobDeaths()TestDeferredFunction.testBatchSystemCleanupCanHandleWorkerDeaths()
- toil.test.src.dockerCheckTest
- Classes
- Module Contents
DockerCheckTestDockerCheckTest.testOfficialUbuntuRepo()DockerCheckTest.testBroadDockerRepo()DockerCheckTest.testBroadDockerRepoBadTag()DockerCheckTest.testNonexistentRepo()DockerCheckTest.testToilQuayRepo()DockerCheckTest.testBadQuayRepoNTag()DockerCheckTest.testBadQuayRepo()DockerCheckTest.testBadQuayTag()DockerCheckTest.testGoogleRepo()DockerCheckTest.testBadGoogleRepo()DockerCheckTest.testApplianceParser()
- toil.test.src.environmentTest
- toil.test.src.fileStoreTest
- Attributes
- Classes
- Module Contents
testingIsAutomaticloggerhiddenhidden.AbstractFileStoreTesthidden.AbstractFileStoreTest.jobStoreTypehidden.AbstractFileStoreTest.setUp()hidden.AbstractFileStoreTest.options()hidden.AbstractFileStoreTest.create_file()hidden.AbstractFileStoreTest.testToilIsNotBroken()hidden.AbstractFileStoreTest.testFileStoreLogging()hidden.AbstractFileStoreTest.testFileStoreOperations()hidden.AbstractFileStoreTest.testWriteReadGlobalFilePermissions()hidden.AbstractFileStoreTest.testWriteExportFileCompatibility()hidden.AbstractFileStoreTest.testImportReadFileCompatibility()hidden.AbstractFileStoreTest.testReadWriteFileStreamTextMode()
hidden.AbstractNonCachingFileStoreTesthidden.AbstractCachingFileStoreTesthidden.AbstractCachingFileStoreTest.options()hidden.AbstractCachingFileStoreTest.testExtremeCacheSetup()hidden.AbstractCachingFileStoreTest.testCacheEvictionPartialEvict()hidden.AbstractCachingFileStoreTest.testCacheEvictionTotalEvict()hidden.AbstractCachingFileStoreTest.testCacheEvictionFailCase()hidden.AbstractCachingFileStoreTest.testAsyncWriteWithCaching()hidden.AbstractCachingFileStoreTest.testWriteNonLocalFileToJobStore()hidden.AbstractCachingFileStoreTest.testWriteLocalFileToJobStore()hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithoutCachingReadFile()hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithCachingReadFile()hidden.AbstractCachingFileStoreTest.testReadCachHitFileFromJobStore()hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheHitGlobalFile()hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheMissGlobalFile()hidden.AbstractCachingFileStoreTest.testFileStoreExportFile()hidden.AbstractCachingFileStoreTest.testReturnFileSizes()hidden.AbstractCachingFileStoreTest.testReturnFileSizesWithBadWorker()hidden.AbstractCachingFileStoreTest.testControlledFailedWorkerRetry()hidden.AbstractCachingFileStoreTest.testRemoveLocalMutablyReadFile()hidden.AbstractCachingFileStoreTest.testRemoveLocalImmutablyReadFile()hidden.AbstractCachingFileStoreTest.testDeleteLocalFile()hidden.AbstractCachingFileStoreTest.testSimultaneousReadsUncachedStream()
NonCachingFileStoreTestWithFileJobStoreCachingFileStoreTestWithFileJobStoreNonCachingFileStoreTestWithAwsJobStoreCachingFileStoreTestWithAwsJobStoreNonCachingFileStoreTestWithGoogleJobStoreCachingFileStoreTestWithGoogleJobStore
- toil.test.src.helloWorldTest
- toil.test.src.importExportFileTest
- toil.test.src.jobDescriptionTest
- toil.test.src.jobEncapsulationTest
- toil.test.src.jobFileStoreTest
- toil.test.src.jobServiceTest
- toil.test.src.jobTest
- Classes
- Functions
- Module Contents
TestJobTestJob.testStatic()TestJob.testStatic2()TestJob.testTrivialDAGConsistency()TestJob.testDAGConsistency()TestJob.testSiblingDAGConsistency()TestJob.testDeadlockDetection()TestJob.testNewCheckpointIsLeafVertexNonRootCase()TestJob.testNewCheckpointIsLeafVertexRootCase()TestJob.runNewCheckpointIsLeafVertexTest()TestJob.runCheckpointVertexTest()TestJob.testEvaluatingRandomDAG()TestJob.getRandomEdge()TestJob.makeRandomDAG()TestJob.getAdjacencyList()TestJob.reachable()TestJob.addRandomFollowOnEdges()TestJob.makeJobGraph()TestJob.isAcyclic()
simpleJobFn()fn1Test()fn2Test()trivialParent()parent()diamond()child()errorChild()TrivialService
- toil.test.src.miscTests
- toil.test.src.promisedRequirementTest
- Attributes
- Classes
- Functions
- Module Contents
logAbstractPromisedRequirementsTestAbstractPromisedRequirementsTest.testConcurrencyDynamic()AbstractPromisedRequirementsTest.testConcurrencyStatic()AbstractPromisedRequirementsTest.getOptions()AbstractPromisedRequirementsTest.getCounterPath()AbstractPromisedRequirementsTest.testPromisesWithJobStoreFileObjects()AbstractPromisedRequirementsTest.testPromisesWithNonCachingFileStore()AbstractPromisedRequirementsTest.testPromiseRequirementRaceStatic()
maxConcurrency()getOne()getThirtyTwoMb()logDiskUsage()TestSingleMachinePromisedRequirementsTestMesosPromisedRequirements
- toil.test.src.promisesTest
- toil.test.src.realtimeLoggerTest
- toil.test.src.regularLogTest
- toil.test.src.resourceTest
- toil.test.src.restartDAGTest
- toil.test.src.resumabilityTest
- toil.test.src.retainTempDirTest
- toil.test.src.systemTest
- toil.test.src.threadingTest
- toil.test.src.toilContextManagerTest
- toil.test.src.userDefinedJobArgTypeTest
- toil.test.src.workerTest
- toil.test.src.autoDeploymentTest
- Submodules
- toil.test.utils
- Submodules
- toil.test.utils.toilDebugTest
- toil.test.utils.toilKillTest
- toil.test.utils.utilsTest
- Attributes
- Classes
- Functions
- Module Contents
loggerunsortedFile()correctSort()TestUtilsTestUtils.NTestUtils.toilMainTestUtils.cleanCommand()TestUtils.statsCommand()TestUtils.statusCommand()TestUtils.test_config_functionality()TestUtils.testAWSProvisionerUtils()TestUtils.testUtilsSort()TestUtils.testUtilsStatsSort()TestUtils.testUnicodeSupport()TestUtils.testMultipleJobsPerWorkerStats()TestUtils.check_status()TestUtils.testGetPIDStatus()TestUtils.testGetStatusFailedToilWF()TestUtils.testGetStatusFailedCWLWF()TestUtils.testGetStatusSuccessfulCWLWF()TestUtils.testPrintJobLog()TestUtils.testRestartAttribute()
printUnicodeCharacter()RunTwoJobsPerWorker
- Submodules
- toil.test.wdl
- Submodules
- toil.test.wdl.wdltoil_test
- Attributes
- Classes
- Functions
- Module Contents
loggerWDL_CONFORMANCE_TEST_REPOWDL_CONFORMANCE_TEST_COMMITWDL_CONFORMANCE_TESTS_UNSUPPORTED_BY_TOILWDL_11_UNIT_TESTS_UNSUPPORTED_BY_TOILWDL_12_UNIT_TESTS_UNSUPPORTED_BY_TOILwdl_conformance_test_repo()TestWDLConformanceTestWDLConformance.check()TestWDLConformance.test_unit_tests_v11()TestWDLConformance.test_unit_tests_v12()TestWDLConformance.test_conformance_tests_v10()TestWDLConformance.test_conformance_tests_v11()TestWDLConformance.test_conformance_tests_development()TestWDLConformance.test_conformance_tests_integration()
TestWDLTestWDL.base_commandTestWDL.test_MD5sum()TestWDL.test_file_uri_no_hostname()TestWDL.test_url_to_file()TestWDL.test_string_file_coercion()TestWDL.test_gather()TestWDL.test_wait()TestWDL.test_restart()TestWDL.test_workflow_file_deletion()TestWDL.test_all_call_outputs()TestWDL.test_croo_detection()TestWDL.test_caching()TestWDL.test_url_to_optional_file()TestWDL.test_missing_output_directory()TestWDL.test_miniwdl_self_test()TestWDL.test_miniwdl_self_test_by_reference()TestWDL.test_dockstore_trs()TestWDL.test_dockstore_metrics_publication()TestWDL.test_giraffe_deepvariant()TestWDL.test_giraffe()TestWDL.test_gs_uri()TestWDL.test_check()
TestWDLToilBench
- toil.test.wdl.wdltoil_test_kubernetes
- toil.test.wdl.wdltoil_test
- Submodules
- toil.test.batchSystems
- Attributes
- Exceptions
- Classes
- Functions
- Package Contents
ProxyConnectionErrorloggerget_data()ToilTestMTget_temp_file()needs_env_var()needs_rsync3()pneeds_rsync3needs_online()pneeds_onlineneeds_aws_s3()pneeds_aws_s3needs_aws_ec2()pneeds_aws_ec2needs_aws_batch()needs_google_storage()needs_google_project()needs_gridengine()pneeds_gridengineneeds_torque()pneeds_torqueneeds_kubernetes_installed()needs_kubernetes()pneeds_kubernetesneeds_mesos()pneeds_mesosneeds_slurm()pneeds_slurmneeds_htcondor()needs_lsf()pneeds_lsfneeds_java()needs_docker()pneeds_dockerneeds_singularity()needs_singularity_or_docker()needs_local_cuda()pneeds_local_cudaneeds_docker_cuda()pneeds_docker_cudaneeds_encryption()needs_cwl()pneeds_cwlneeds_wdl()pneeds_wdlneeds_server()needs_celery_broker()needs_wes_server()pneeds_wes_serverneeds_local_appliance()needs_fetchable_appliance()integrative()pintegrativeslow()pslowmethodNamePartRegextimeLimit()make_tests()ApplianceTestSupportApplianceTestSupport.ApplianceApplianceTestSupport.Appliance.lockApplianceTestSupport.Appliance.outerApplianceTestSupport.Appliance.mountsApplianceTestSupport.Appliance.cleanMountsApplianceTestSupport.Appliance.containerNameApplianceTestSupport.Appliance.popenApplianceTestSupport.Appliance.__enter__()ApplianceTestSupport.Appliance.__exit__()ApplianceTestSupport.Appliance.tryRun()ApplianceTestSupport.Appliance.runOnAppliance()ApplianceTestSupport.Appliance.writeToAppliance()ApplianceTestSupport.Appliance.deployScript()
ApplianceTestSupport.LeaderThreadApplianceTestSupport.WorkerThread
- Submodules
- toil.toilState
- Attributes
- Classes
- Module Contents
loggerToilStateToilState.busToilState.successor_to_predecessorsToilState.successorCountsToilState.service_to_clientToilState.servicesIssuedToilState.jobs_issuedToilState.totalFailedJobsToilState.hasFailedSuccessorsToilState.failedSuccessorsToilState.jobsToBeScheduledWithMultiplePredecessorsToilState.load_workflow()ToilState.job_exists()ToilState.get_job()ToilState.commit_job()ToilState.delete_job()ToilState.reset_job()ToilState.reset_job_expecting_change()ToilState.successors_pending()ToilState.successor_returned()ToilState.count_pending_successors()
- toil.utils
- Submodules
- toil.utils.toilClean
- toil.utils.toilConfig
- toil.utils.toilDebugFile
- toil.utils.toilDebugJob
- toil.utils.toilDestroyCluster
- toil.utils.toilKill
- toil.utils.toilLaunchCluster
- toil.utils.toilMain
- toil.utils.toilRsyncCluster
- toil.utils.toilServer
- toil.utils.toilSshCluster
- toil.utils.toilStats
- Attributes
- Classes
- Functions
- Module Contents
loggerCATEGORIESCATEGORY_UNITSTITLESTIME_CATEGORIESSPACE_CATEGORIESCOMPUTED_CATEGORIESLONG_FORMSColumnWidthspad_str()pretty_space()pretty_time()report_unit()report_time()report_space()report_number()report()sprint_tag()decorate_title()decorate_subheader()get()sort_jobs()report_pretty_data()compute_column_widths()update_column_widths()build_element()create_summary()get_stats()process_data()report_data()sort_category_choicessort_field_choicesadd_stats_options()main()
- toil.utils.toilStatus
- Attributes
- Classes
- Functions
- Module Contents
loggerToilStatusToilStatus.jobStoreNameToilStatus.jobStoreToilStatus.message_bus_pathToilStatus.print_dot_chart()ToilStatus.printJobLog()ToilStatus.printJobChildren()ToilStatus.printAggregateJobStats()ToilStatus.report_on_jobs()ToilStatus.getPIDStatus()ToilStatus.getStatus()ToilStatus.print_running_jobs()ToilStatus.fetchRootJob()ToilStatus.fetchUserJobs()ToilStatus.traverseJobGraph()
main()
- toil.utils.toilUpdateEC2Instances
- Submodules
- toil.version
- toil.wdl
- Submodules
- toil.wdl.utils
- toil.wdl.wdltoil
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
loggerWDLINodeAnyINodeINodeTransformis_inode()ReadableFileObjFileDigesterfile_digestWDLContextInsufficientMountDiskSpacewdl_error_reporter()Freport_wdl_errors()toil_read_source()virtualized_equal()WDLBindingscombine_bindings()log_bindings()get_supertype()for_each_node()recursive_dependencies()parse_disks()pack_toil_uri()unpack_toil_uri()SHARED_PATH_ATTRclone_metadata()make_inode()set_inode_value()set_inode_nonexistent()get_inode_nonexistent()set_inode_virtualized_value()get_inode_virtualized_value()get_shared_fs_path()set_shared_fs_path()view_shared_fs_paths()poll_execution_cache()fill_execution_cache()choose_human_readable_directory()evaluate_decls_to_bindings()NonDownloadingSizeextract_inode_values()extract_inode_virtualized_values()extract_toil_file_uris()virtualize_inodes_in_bindings()convert_remote_files()ToilWDLStdLibBaseToilWDLStdLibWorkflowToilWDLStdLibTaskCommandToilWDLStdLibTaskOutputsevaluate_named_expression()evaluate_decl()evaluate_call_inputs()evaluate_defaultable_decl()devirtualize_inodes()virtualize_inodes()delete_dead_files()all_parents()add_paths()drop_if_missing()missing_inode_dropper()drop_missing_files()get_paths_in_bindings()map_over_inodes_in_bindings()map_over_inodes_in_binding()remove_expr_from_value()map_over_typed_value()map_over_typed_inodes_in_value()ensure_null_inodes_are_nullable()WDLBaseJobWDLTaskWrapperJobWDLTaskJobWDLWorkflowNodeJobWDLWorkflowNodeListJobWDLCombineBindingsJobWDLWorkflowGraphWDLSectionJobWDLScatterJobWDLArrayBindingsJobWDLConditionalJobWDLWorkflowJobWDLOutputsJobWDLStartJobWDLInstallImportsJobWDLImportWrappermake_root_job()main()
- Submodules
- toil.worker
- toil.batchSystems
- Attributes
- Exceptions
- Functions
- Package Contents
logwhich()toilPackageDirPath()inVirtualEnv()resolveEntryPoint()physicalMemory()physicalDisk()applianceSelf()customDockerInitCmd()customInitCmd()lookupEnvVar()checkDockerImageExists()parseDockerAppliance()checkDockerSchema()ApplianceImageNotFoundKNOWN_EXTANT_IMAGESrequestCheckRegularDocker()requestCheckDockerIo()logProcessContext()
- Submodules
- toil