hotsub

CLI tool to run a batch jobs with ETL/ExTL framework on AWS or other cloud services.

View the Project on GitHub github.com/otiai10/hotsub

Here describes commonly-used examples as showcase.

BWA alignment

This recipe runs BWA alignment on AWS EC2 instances, without configuring instances.

hotsub run \
    --script ./main.sh \
    --image hotsub/bwa-alignment \
    --tasks ./bwa-alignment.csv \
    --aws-ec2-instance-type t2.large \
    --verbose

This command will

All the resources are automatically cleaned up after output files are located back to S3.

View this recipe on GitHub

STAR alignment

This recipe runs STAR RNA-seq alignment on AWS EC2 instances, without configuring instances.

hotsub run \
  --script ./main.sh \
  --image friend1ws/star-alignment \
  --tasks ./star-alignment.csv \
  --aws-ec2-instance-type m4.2xlarge \
  --disk-size 128 \
  --shared REFERENCE=s3://hotsub/resources/reference/GRCh37.STAR-2.5.2a \
  --verbose

This command will

All the resources are automatically cleaned up after output files are located back to S3.

View this recipe on GitHub

GDC Download

This recipe downloads common data from NCI Genomic Data Commons and convert BAM files to Fastq files.

hotsub run \
  --script ./main.sh \
  --image hotsub/gdc-client \
  --tasks ./MCF7-fastq.csv \
  --aws-ec2-instance-type m4.2xlarge \
  --disk-size 64 \
  -v

This command will

All the resources are automatically cleaned up after output files are located back to S3.

View this recipe on GitHub

If you have any working hotsub recipe, please let us know and add it here ;)