Action items:
Notes:
10K NNs: (100 (bootstrapping for stat uncertainty) + 30 systematics) * 100 NNs for averaging due to stochastic nature of detector response
Z+jets measurement: released without binned histograms but with an 24 dimensional ntuple with weights
Some restrictions on memory when applying to more data (e.g., lower pT cut)
Model sizes: not large. 24 dimensional input (dimension of cross section measurements) thus 10-100K parameters. Transformers (for particle inputs) will make them bigger. Next iteration will be much higher dimensionality.
Unfolding is a good data preservation technique. Sharp resonances are not well preserved.
How are the 10K models run? 10K jobs in parallel.
How long do they take to run? Will need to follow up. The time limit on perlmutter is not a problem now but it might be for the next iteration. There are a few tricks to make the training faster.
Is the procedure to train the models automated? Everything is automated.
Computing was challenging but not a bottleneck. The next round R&D is making transformers fast. Transfer learning (pretraining) helps a lot.
Do these methods change the robustness of the models
Did you use any specific tools to automate the procedure? No, we used batch scripts. It would be nice to have a tool to help with bookkeeping.
Preserved data is much more challenging to use than current HEPData.