687b777912
Merge remote-tracking branch 'origin/workflow_config'
2024-07-23 17:26:03 +02:00
35f8b4c369
Merge branch 'main' of https://forge.chapril.org/GuilloteauQ/study-docker-repro-longevity
2024-07-23 09:38:24 +02:00
557f766a02
Removing global variables, closing #19 .
2024-07-23 09:38:03 +02:00
Quentin Guilloteau
944b1bf6f9
checkpoint in case of reaching walltime
2024-07-21 17:32:04 +02:00
Quentin Guilloteau
ef257e6026
add prefix to config
2024-07-21 16:14:58 +02:00
Quentin Guilloteau
566eb778f2
fix execution on grid5000 from namespace
2024-07-21 16:13:52 +02:00
Quentin Guilloteau
318955865c
typo
2024-07-21 16:12:40 +02:00
Quentin Guilloteau
dd6eb9bcbb
now using tmp dir to decompress the artifact
2024-07-21 16:11:55 +02:00
Quentin Guilloteau
8ed548c435
first try for g5k execution
2024-07-20 15:41:56 +02:00
cc605943a9
Added a comment section for artifacts configuration. Moved hints to the Nickel contract.
2024-07-19 17:05:46 +02:00
Quentin Guilloteau
d7749369b4
move artifacts_nickel and artifacts_json to artifacts/nickel and artifacts/json
2024-07-19 16:38:18 +02:00
Quentin Guilloteau
9769f6b533
Add config for snakemake
2024-07-19 16:33:27 +02:00
9a29c526fb
Small doc fixes.
2024-07-19 16:23:39 +02:00
a7bf9cee94
Written a doc for the output. Removed the "type" attribute from the Nickel contract, closing #17 .
2024-07-19 16:18:49 +02:00
1861f44f1e
Removed the "type" attribute in the misc_package description.
2024-07-19 15:28:58 +02:00
cb98727554
Added artifacts_json to git ignore and deleting, closing #15 . Deleted old YAML files. Added hints on the config template.
2024-07-19 15:23:00 +02:00
57ff34e8b9
Written Nickel + JSON artifact config files from their YAML version. Handling unknown errors during image building.
2024-07-18 17:56:57 +02:00
d791064183
Added Nickel constraint on values.
2024-07-17 14:56:38 +02:00
6897091c0d
Modified README according to changes.
2024-07-16 17:58:13 +02:00
75056727e5
Fixing quotes for formatted strings.
2024-07-16 17:53:34 +02:00
f2cfceffb0
When no cache dir is specified, cache is disabled.
2024-07-16 17:50:55 +02:00
1aaa77cba3
Fixed Nickel contract and check files. Fixed input/output files in Snakefile.
2024-07-16 17:35:09 +02:00
69e447ab0a
Fixed Nickel config files. Added artifact hash log on Snakefile + modified some variables names. Switched from 'tgz' to 'tar' for archive type.
2024-07-16 16:47:17 +02:00
842f9d32a7
Added license.
2024-07-16 16:04:02 +02:00
83a6930593
Merge pull request 'integrate nickel in workflow' ( #11 ) from nickel_workflow into main
...
Reviewed-on: #11
2024-07-16 15:37:13 +02:00
82af4bd521
Written a doc. Changing the name of the variables related to the hash log.
2024-07-16 15:34:15 +02:00
Quentin Guilloteau
4432a93123
integrate nickel in workflow
2024-07-16 13:59:44 +02:00
f068fb91d0
Added artifact hash logging, closing #3 .
2024-07-16 12:04:29 +02:00
0841823096
Switched from YAML to JSON, closing #8 .
2024-07-16 11:01:08 +02:00
5a8d7905c6
Fixed merge.
2024-07-16 10:32:54 +02:00
73b6cd1e70
Merge branch 'main' of https://forge.chapril.org/GuilloteauQ/study-docker-repro-longevity
2024-07-16 10:32:19 +02:00
e06831bd78
Merge branch 'start_workflow'
2024-07-16 10:31:18 +02:00
fd7c11483f
Fixed example YAML to fit 'main'.
2024-07-16 10:29:46 +02:00
127c5c5ef4
Merge pull request 'Add POC for using nickel as a configuration lang' ( #2 ) from nickel into main
...
Reviewed-on: #2
2024-07-16 10:24:20 +02:00
9a01db7faf
Now using f-strings, closing #6 . Made path-related cmd args required, closing #7 . Used variables in check_env for readability, closing #5 .
2024-07-16 10:20:04 +02:00
da114662ba
Added stdout to output. Fixed log. Opening build_status file in append mode. Added verbose flag to run script.
2024-07-15 11:28:38 +02:00
0e6b181310
Continued work on buildstatus_saver. Removed headers for CSV files, closing #4 .
2024-07-15 11:00:08 +02:00
2ad37f46f2
Initial support for Docker errors parsing.
2024-07-12 17:50:19 +02:00
23c4f7ede4
Added support for cache. Added run and clean scripts.
2024-07-12 12:10:03 +02:00
27d0331c06
Added logging to a file.
2024-07-12 11:21:14 +02:00
2f779d0ab0
- Config files now contain 1 Dockerfile only.
...
- Removed support for running ECG with multiple config files at the same time: will be called with a single one only
- Removed storage management of outputs, paths are now given using command line args.
- Started working on cache support.
2024-07-11 17:14:58 +02:00
Quentin Guilloteau
46b7d349c3
Add POC for using nickel as a configuration lang
...
The `check.ncl` file contains the commands what we could use directly from snakemake to validate the configs and generate the json/yaml for ecg
2024-07-11 16:05:08 +02:00
Quentin Guilloteau
28126f8b3e
basic workflow for ecg
2024-07-11 15:17:16 +02:00
Quentin Guilloteau
3737dd29e6
add start of workflow to extract non blacklisted artifacts
2024-07-11 13:37:07 +02:00
Quentin Guilloteau
1d6cec18ff
create folder to store artifacts yamls
2024-07-11 13:36:22 +02:00
Quentin Guilloteau
27e0552bc8
add snakemake and awk to nix flake
2024-07-11 13:35:54 +02:00
30a82efc70
Adding support for multiple config files. Improving structure of output files.
2024-07-10 19:01:46 +02:00
7dfe12ed80
Added support for packages outside of pkgmgr and git.
2024-07-10 17:06:54 +02:00
03c6602db3
Now running the output formatting commands locally, and not on the container.
2024-07-10 15:44:01 +02:00
a18bed49b1
Added support for Git packages.
2024-07-10 14:50:57 +02:00
