4 Matching Annotations
  1. Jul 2025
    1. the advantages of computational pipelines over ad hoc scripts, even for simple tasks, are all more apparent with increasingly complex datasets and the use of parallel processing.

      why pipelines vs ad hoc scripts - track dependencies (statically inferred, DAGs) - rules reused for many files (parallelization) - data tracking (rapid development in subsets of pipeline ~ changing parameters,ie. avoid duplicate work when resuming workflows.)

  2. Mar 2025
    1. the advantages of computational pipelines over ad hoc scripts, even for simple tasks, are all more apparent with increasingly complex datasets and the use of parallel processing.

      why pipelines vs ad hoc scripts - track dependencies (statically inferred, DAGs) - rules reused for many files (parallelization) - data tracking (rapid development in subsets of pipeline ~ changing parameters,ie. avoid duplicate work when resuming workflows.)