Changelog

Changelog#

This page keeps track of major and minor changes between versions. Breaking changes are also reported

PocketCoffea 0.9.11#

New features#

  • Weights by subsamples (#294): per-subsample weights, inclusive and by-category, with full variation (up/down) support. Subsample-specific weight variations are correctly isolated per subsample in the histogram variation axis.

  • Shape variations by subsamples (#330): subsample-specific shape (calibrator) variations. Each subsample can carry a different set of shape variations; histogram axes are built per-subsample so the variation labels are correctly isolated.

  • Fix histogram filling for subsample weight and shape variations (#491): several bugs in HistManager were fixed — wrong variation axis used in the fill loop when subsamples are configured, subsample-specific shape variations silently skipped during filling, and subsample nominal weights not applied during shape-variation passes. Comprehensive integration tests added.

  • Variations-aware skimming (#467): the skim step now retains events that pass at least one preselection variation (not just the nominal), ensuring shape-variation passes do not lose events skimmed away by the nominal selection.

  • MET Type-1 correction calibrator (#472): new calibrator for MET Type-1 propagation, including unclustered energy systematics. Extended to Run2.

  • Multi-dimensional variables (#374): HistConf now supports multi-dimensional delayed-evaluation variables.

  • Jet pT regression (#461): jet pT regression support in the jet calibrator, with merge_collections_for_variations and collection_name_alias options to merge regressed and standard jet collections.

  • JetID from correctionlib (#421, #395): JetID delivered via correctionlib, replacing hardcoded values.

  • msoftdrop correction (#420): soft-drop mass correction for AK8 jets.

  • Default JEC via correctionlib (#401, #418): JEC/JER now use correctionlib as the default backend.

  • Read unpublished remote datasets over XRootD (#461): dataset builder can now discover and read files from unpublished XRootD endpoints.

  • merge_columns CLI command (#482): new pocket-coffea merge-columns sub-command to merge columnar outputs.

  • CCsorted function (#456): utility to sort collections by a given field with stable ordering.

Scale factors and corrections#

  • Muon promptMVA scale factors, including ISO SF for 2024 (#470)

  • Electron promptMVA scale factors with promptMVA ID (#474)

  • Updated BTV scale factors for 2024 (#473)

  • Lepton (electron and muon) scale factors for 2024 (#466)

  • Electron trigger scale factor added to common weights for Run3 (#476); sf_ele_trigger renamed for Run2 consistency (#481)

  • Updated electron scale factors and electron SS energy scale/smearing (#444, #414, #419)

  • Muon calibrator update (#432)

  • Rochester muon momentum corrections (#475): new MuonsRochesterCalibrator for Run 2 Ultra-Legacy muon pT corrections using the RoccoR library; correction files included for 2016 pre/post-VFP, 2017, and 2018.

  • Muon SF lower pT bound for Run3 set to 26 GeV; electron to 25 GeV (#476)

  • Jets for NanoAODv12 (#355)

  • JetVetoMap fix (#430)

  • FatJet ID fix and 2024 AK8 support (#429, #440)

  • Updated JEC data tags for 2022/2023 (#489); JEC configuration for 2023 preBPix MC

  • Lepton mvaTTH discriminant added to lepton objects (#455)

  • UParT tagger scores added to jet_taggers_hists() (#460)

  • Updated luminosity values for 2023 and 2024; golden JSON updated for 2022, 2023, 2024 and added for 2025 (#441, #490)

  • Updated correctionlib tags (#415)

Plotter and output improvements#

  • Plotter: correct handling of subsamples and weight/shape variations (#442)

  • Plotter: fix systematic band rendering (#408)

  • Save shape variation arrays to output (#368)

  • Cutflow: major rework with subsample support and various fixes (#369); fix for cutflow with subsamples in plot scripts; fix handling of variations in sum-of-weights

  • Datacard output support (#407)

  • New analysis metadata stored in output (#409)

LAW integration#

  • Updated LAW task parameters and skipbadfiles option (#464)

  • LAW tasks for plotting (#393)

  • Dask scheduler support in runner (#403, #411)

  • Updated dataset-definition parameter handling (#464)

Bug fixes#

  • Fixed priority list not being passed to dataset builder (#485)

  • NanoAOD v15 compatibility for Run 2 UL (#457): rho field access, PU jet ID, and JER now use the correct NanoAOD-version-aware branch paths (Rho.fixedGridRhoFastjetAll for v12+, fixedGridRhoFastjetAll for ≤v9); PUid selection correctly disabled for NanoAOD v15.

  • Fixed jets variations for AK8Puppi collection

  • Fixed jets variations and config when using collection_name_alias

  • Fixed legacy calibrator raw-factor NaN (#428)

  • Fixed filesets configuration exception message with filter details (#448)

  • Fixed histogram filling for only_variations histograms (#392)

  • Fixed calibrators variations handling (#389)

  • Fixed muon SF for Run3 (#397)

  • Fixed dump_ak_array for XRootD mkdir (#448)

  • Improved hadd_skimmed_files, check_jobs, and merge_output scripts

PocketCoffea 0.9.6#

  • Minor release: only fixes and small improvements

    • Some fixes in the plotting script

    • Improvements for the INFN analysis facility executor

PocketCoffea 0.9.5#

  • Generalize handling of common Weights and user-defined Weights

  • Added first integration of LAW tasks to stear an analysis running PocketCoffea workflows

  • Many improvements in the plotting scripts

  • Added Dataset discovery cli to dynamically query for dataset and build the dataset definition file

  • Cleaning up of the default skimming function (Breaking changes! see below)

  • Added CDCI tests of utils and full configuration tests

  • New parameters exploration CLI

  • Added more executors

  • Tested the Swan AF and INFN AF

Breaking changes#

Default skim#

  • Some skimming cuts were included by default in the base workflow and may be unnoticed by the users PR#193. For maximum transparency we have removed those cuts from the base workflow and made the cutting functions available to be used in the configuration. The functions were:

    • nPV_good > 0 selection

    • goldenJson

    • event flags for data and MC.

Users must now include the following cut functions in their skim configuration to keep the same cutflow in their analysis:

1from pocket_coffea.lib.cut_functions import get_nPVgood, goldenJson, eventFlags
2
3cfg = Configurator(skim=[get_nPVgood(1), eventFlags, goldenJson])

This change is enough to preserve the same cuts applied before 0.9.5.

Jet selection function#

The signature of the jet cleaning function used often in the object preselection step of processor has been changes to add explicitely the year argument.

- def jet_selection(events, jet_type, params, leptons_collection=""):
+ def jet_selection(events, jet_type, params, year, leptons_collection=""):

This signature change can be unnoticed by users using the leptons_collection argument. Please cross-check your function usage.

PocketCoffea 0.9.0#

  • New executor plugin setup to support multiple sites and analysis facilities

  • New defaults for Run3 corrections

  • Improve CLI interface and unified scripts under pocket-coffea command

  • Added dataset-discovery-cli to build dataset_definition files

PocketCoffea 0.7#

  • Improved and generalized executors configuration for runner.py

PocketCoffea 1.0rc0#

The main change is the possibility to completely split the analysis configuration files and parameters from the core of the framework.

  • Parameters configuration now in place with OmegaConf

  • Docker and singularity images built with GitLab CD/CI and published on /cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/cms-analysis/general

  • Improved the handling of datasets and samples

  • Added dataset metadata in the output file

Contents