pocket_coffea.scripts package#
Subpackages#
- pocket_coffea.scripts.dataset package
- Submodules
- pocket_coffea.scripts.dataset.append_genweights module
- pocket_coffea.scripts.dataset.append_parents module
- pocket_coffea.scripts.dataset.build_datasets module
- pocket_coffea.scripts.dataset.dataset_query module
DataDiscoveryCLIDataDiscoveryCLI.as_dictDataDiscoveryCLI.do_allowlist_sites()DataDiscoveryCLI.do_blocklist_sites()DataDiscoveryCLI.do_clear()DataDiscoveryCLI.do_list_replicas()DataDiscoveryCLI.do_list_selected()DataDiscoveryCLI.do_login()DataDiscoveryCLI.do_prioritylist_sites()DataDiscoveryCLI.do_query()DataDiscoveryCLI.do_query_results()DataDiscoveryCLI.do_regex_sites()DataDiscoveryCLI.do_replicas()DataDiscoveryCLI.do_save()DataDiscoveryCLI.do_select()DataDiscoveryCLI.do_set_replicas_sorting()DataDiscoveryCLI.do_sites_filters()DataDiscoveryCLI.do_whoami()DataDiscoveryCLI.extract_era_from_dataset_name()DataDiscoveryCLI.extract_xsec_from_dataset_name()DataDiscoveryCLI.extract_year_from_dataset_name()DataDiscoveryCLI.generate_default_metadata()DataDiscoveryCLI.is_mc_dataset()DataDiscoveryCLI.load_dataset_definition()DataDiscoveryCLI.start_cli()
get_indices_query()print_dataset_query()
- pocket_coffea.scripts.dataset.download module
- Module contents
- pocket_coffea.scripts.plot package
Submodules#
pocket_coffea.scripts.build_jec module#
Nice code to build the JEC/JER and JES uncertainties taken from andrzejnovak/boostedhiggs
- pocket_coffea.scripts.build_jec.jet_factory_factory(files)#
pocket_coffea.scripts.check_jobs module#
Simple script that checks the status of the jobs submitted by runner on condor.
The status of the jobs can be checked by looking at the file in the jobs folder.
job_x.idle: The job is waiting to be executed
job_x.running: The job is running
job_x.done: The job has finished
job_x.failed: The job has failed
where x is the job id.
- pocket_coffea.scripts.check_jobs.bump_jobqueue(sub_file, shift=1)#
- pocket_coffea.scripts.check_jobs.check_jobs_logs(jobs_folder)#
- pocket_coffea.scripts.check_jobs.create_layout(with_progress=False)#
Two-column layout. The left column carries the summary table (and the per-group progress table when with_progress is True) and gets twice the width of the log panel on the right, since that’s where the interesting content lives.
- pocket_coffea.scripts.check_jobs.find_other_file(filepath, sitemap, xrootdfaillist=[], blacklist_sites=[], rucio_client=None)#
- pocket_coffea.scripts.check_jobs.get_progress_table(group_counts, label, multi_sample_overlap=False, bar_width=30)#
Build a rich Table showing per-group progress, sorted by % done ascending so straggling groups surface at the top. Includes a stacked coloured progress bar column (done / running / idle / failed).
- pocket_coffea.scripts.check_jobs.get_tables(tot_jobs, idle_jobs, running_jobs, done_jobs, failed_jobs, details=False)#
- pocket_coffea.scripts.check_jobs.update_blacklist(xrootdfaillist, blacklist_threshold)#
pocket_coffea.scripts.config_template module#
pocket_coffea.scripts.hadd_skimmed_files module#
- pocket_coffea.scripts.hadd_skimmed_files.do_hadd(group, overwrite=False)#
pocket_coffea.scripts.make_config module#
- pocket_coffea.scripts.make_config.write_config(processor_class: str, datasets_list: list, samples_list: list, years_list: list, output_path: str)#
pocket_coffea.scripts.merge_columns module#
- pocket_coffea.scripts.merge_columns.find_leaf_dirs(root_input, root_output)#
Return list of (leaf_dir, output_file) pairs.
- pocket_coffea.scripts.merge_columns.merge_leaf_dir(input_dir, output_file, force)#
- pocket_coffea.scripts.merge_columns.worker(task)#
Wrapper for multiprocessing (task is a tuple: (input_dir, output_file, force))
pocket_coffea.scripts.merge_outputs module#
- pocket_coffea.scripts.merge_outputs.append_configs(c1, c2)#
Merges config c2 into c1. Updates dicts, extends lists, and fills None values in c1 with values from c2.
- pocket_coffea.scripts.merge_outputs.merge_group_reduction(output_files, N_reduction=5, cachedir='merge_cache', max_mem_gb=8, verbose=False)#
- pocket_coffea.scripts.merge_outputs.merge_outputs(inputfiles, outputfile, jobs_config=None, force=False, N_reduction=5, max_mem_gb=None, cache_dir=None, verbose=False, skip_check=False, mark_failed=False, configurator=None, skip_initial_events_check_datasets=None)#
Merge coffea output files
- pocket_coffea.scripts.merge_outputs.process_failed(mark_failed, statusfile, job_dir, job_name, message='missing')#
pocket_coffea.scripts.pileup_corrlib module#
pocket_coffea.scripts.print_parameters module#
pocket_coffea.scripts.runner module#
pocket_coffea.scripts.split_output module#
- pocket_coffea.scripts.split_output.split_output(inputfile, outputfile, by, ncategory_per_file, overwrite)#
Split coffea output files