Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
e7b2f37
Add Phase2 OT efficiency client to standalone harvesting sequence
brandiskip Nov 13, 2025
1c9bd03
use DQMGenericClient for Phase2OT track efficiencies; drop manual code
brandiskip Nov 13, 2025
5cdca81
added helper functions for booking histograms
brandiskip Nov 13, 2025
378849f
code using DQMGenericClient for Phase2OT track efficiencies
brandiskip Nov 13, 2025
405d956
reduced many lines of code by use of helper functions
brandiskip Nov 13, 2025
290ca3a
fixed error causing compile issues
brandiskip Nov 14, 2025
18190eb
got rid of unnecessary helper function calls and added histograms del…
brandiskip Nov 18, 2025
ccc774f
got rid of unnecessary helper functions
brandiskip Nov 18, 2025
5571675
added code to calculate stub efficiencies
brandiskip Nov 18, 2025
bf2dc78
removed Phase2OTHarvestStub.cc as efficiency calculations are now don…
brandiskip Nov 18, 2025
0f11897
add helper function for resolution hists
brandiskip Jan 6, 2026
e21bd88
reduced several hundred lines of code by use of helper function
brandiskip Jan 6, 2026
39e2092
add detailed hists to sub-folder and reduce range for deltaPhi hists
brandiskip Jan 21, 2026
f73480f
edit comments
brandiskip Jan 21, 2026
1c8648f
add weights for stub efficiency hists
brandiskip Jan 26, 2026
4cf52e5
add code to make number of stubs per event hist
brandiskip Jan 26, 2026
8901e72
Including EXOnanoAODv1 custom constent
Dec 8, 2025
91ee1bc
Update PhysicsTools/NanoAOD/python/custom_displacedtau_cff.py
kerstinlovisa Feb 19, 2026
9953a84
code-format correction
Feb 20, 2026
98e74f2
fix res_pt and res_eta axis labels
brandiskip Mar 4, 2026
5c58adf
Changed folder naming logic
brandiskip Mar 4, 2026
a8bc657
remove manual efficiency calculation
brandiskip Mar 4, 2026
939b587
remove Phase2OTHarvestStub as file as been removed
brandiskip Mar 4, 2026
68a6558
Merged work_branch from repository brandiskip with cms-merge-topic
brandiskip Mar 9, 2026
e514f2b
Corrected for additional PR comments
Mar 12, 2026
8263cc1
code format
Mar 12, 2026
6391c87
ensure both num and den for efficiency hists have same selections
brandiskip Mar 20, 2026
a07d05b
update geometry and sample
brandiskip Mar 20, 2026
cfc35b0
add missing histograms for efficiency plots
brandiskip Mar 20, 2026
0dda3df
Reduce code duplication in fillDescriptions
brandiskip Mar 23, 2026
f3cba77
updated header comments
brandiskip Mar 23, 2026
ee2022c
updates from running code-format
brandiskip Mar 23, 2026
f3d4e5d
updates from running code-format
brandiskip Mar 23, 2026
0ef0ddf
add comments
brandiskip Mar 23, 2026
c8c6e36
Merge branch 'master' into work_branch
brandiskip Mar 23, 2026
0930a2d
Apply cmsbuild code-format patch
brandiskip Mar 23, 2026
a561bb2
Merge branch 'work_branch' of https://github.com/brandiskip/cmssw int…
brandiskip Mar 23, 2026
b4845fe
Adding pass-through option to MultiTrackSelector (for LST tracking / …
mmasciov Mar 18, 2026
adc0cc6
Add possibility to mask clusters from seeds on fastSIM
mmasciov Mar 23, 2026
abeed7b
Updating DisTauTag.cc test_vector to test_score
Apr 7, 2026
3e58d5f
Merge branch 'master' into EXOnanoAODv1
kerstinlovisa Apr 7, 2026
e890a13
code format fix
Apr 7, 2026
450de32
Merge branch 'EXOnanoAODv1' of https://github.com/kerstinlovisa/cmssw…
Apr 7, 2026
5b2960f
Squashed commit of the following:
fabiocos Mar 22, 2026
a9fdfc5
Add ioread statements for PSimHit backward compatibility in classes_d…
fabiocos Mar 24, 2026
59bf7e6
comment out validation step as per alaperto request
brandiskip Apr 13, 2026
a94bc1b
change number of events to 10
brandiskip Apr 13, 2026
ba046bc
commented out validation step
brandiskip Apr 13, 2026
002e041
Revert "NanoAOD: add "saveTriggerResults" to NanoAODOutputModule"
missirol Apr 20, 2026
05c89f4
Revert "NanoAOD: add "saveTriggerResults" to NanoAODRNTupleOutputModule"
missirol Apr 20, 2026
807263e
Remove TensorFlow dependency from BuildFile.xml
jfernan2 Apr 23, 2026
c81a5af
Add HLT objects monitoring tools depending on TriggerEvent in the NGT…
sakura-ngt Apr 24, 2026
e3dcef3
add infrastructure to have ngt_dqm_sourceclient unit test consume its…
sakura-ngt Apr 24, 2026
ab89451
Add PUGenParticleProducer
francesco-vaselli Apr 24, 2026
06b3c2d
fix code format with scram b code-format
francesco-vaselli Apr 25, 2026
1e375b1
Revert "Add ioread statements for PSimHit backward compatibility in c…
fabiocos Apr 27, 2026
dc6538a
Merge pull request #50811 from sakura-ngt/mm_hltObjectMonitor_ngtClient
cmsbuild Apr 28, 2026
6e8e215
Merge pull request #50793 from jfernan2/patch-29
cmsbuild Apr 28, 2026
a16213b
mpi modules in different paths per group
Apr 28, 2026
f70b784
copy message logger in the mpi configuration splitter
Apr 28, 2026
c16c8cc
comment out unused validation load
brandiskip Apr 28, 2026
a22bd2e
Merge pull request #50821 from Annnnya/splitter_fixes
cmsbuild Apr 28, 2026
5db4994
Merge pull request #50797 from missirol/devel_nanoRemoveParamSaveTrig…
cmsbuild Apr 29, 2026
6ef0900
Merge pull request #50816 from francesco-vaselli/add-PUGenParticlePro…
cmsbuild Apr 29, 2026
4057405
Add Phase2OTEffClient to harvesting sequences to enable central DQM G…
brandiskip Apr 29, 2026
90e02bb
Merge pull request #49969 from fabiocos/fc-psimhit-20260122
cmsbuild Apr 30, 2026
ba086d9
Merge pull request #50467 from SegmentLinking/passThroughMTS
cmsbuild Apr 30, 2026
af65c7b
Merge pull request #50502 from brandiskip/work_branch
cmsbuild Apr 30, 2026
f388a74
Merge pull request #48502 from kerstinlovisa/EXOnanoAODv1
cmsbuild Apr 30, 2026
83dbac1
Merge CMSSW_17_0_X into CMSSW_17_0_EVOLUTION_X.
Dr15Jones Apr 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions Configuration/PyReleaseValidation/python/relval_nano.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,9 @@ def next(self, index: int = None) -> None:
steps['BTVNANO_mc15.0'] = merge([{'-s': 'NANO:@BTV,DQM:@nanoAODDQM', '-n': '1000'},
steps['NANO_mc15.0']])

steps['EXONANO_mc15.0'] = merge([{'-s': 'PAT,NANO:@EXO', '-n': '1000'},
steps['NANO_mc15.0']])

steps['lepTrackInfoNANO_mc15.0'] = merge([{'-s': 'NANO:@LepTrackInfo,DQM:@nanoAODDQM', '-n': '1000'},
steps['NANO_mc15.0']])

Expand All @@ -422,10 +425,14 @@ def next(self, index: int = None) -> None:

# ====== DATA ======
lumis_Run2025C = {392175: [[95, 542]]}
lumis_Run2025C_Muon0 = {392997: [[146, 202]]}

steps['JetMET1_Run2025C_MINIAOD_150X'] = {'INPUT': InputInfo(
location='STD', ls=lumis_Run2025C, dataSet='/JetMET1/Run2025C-PromptReco-v1/MINIAOD')}

steps['Muon0_Run2025C_AOD_150X'] = {'INPUT': InputInfo(
location='STD',ls=lumis_Run2025C_Muon0, dataSet='/Muon0/Run2025C-PromptReco-v1/AOD')}

steps['ScoutingPFRun3_Run2025C_HLTSCOUT_150X'] = {'INPUT': InputInfo(location='STD', ls=lumis_Run2025C,
dataSet='/ScoutingPFRun3/Run2025C-v1/HLTSCOUT')}

Expand Down Expand Up @@ -455,6 +462,9 @@ def next(self, index: int = None) -> None:
steps['BTVNANO_data15.0'] = merge([{'-s': 'NANO:@BTV,DQM:@nanoAODDQM', '-n': '1000'},
steps['NANO_data15.0']])

steps['EXONANO_data15.0'] = merge([{'-s': 'PAT,NANO:@EXO', '-n': '1000'},
steps['NANO_data15.0']])

steps['lepTrackInfoNANO_data15.0'] = merge([{'-s': 'NANO:@LepTrackInfo,DQM:@nanoAODDQM', '-n': '1000'},
steps['NANO_data15.0']])

Expand Down Expand Up @@ -650,6 +660,7 @@ def next(self, index: int = None) -> None:
#workflows[_wfn()] = ['ScoutingNANOmc150X', ['TTbar_13p6_Summer24_MINIAOD_150X', 'scoutingNANO_mc15.0']] COMMENT: reads old format file
#workflows[_wfn()] = ['ScoutingNANOwithPromptmc150X', ['TTbar_13p6_Summer24_MINIAOD_150X', 'scoutingNANO_withPrompt_mc15.0']] COMMENT: reads old format file
#workflows[_wfn()] = ['BPHNANOmc150X', ['TTbar_13p6_Summer24_MINIAOD_150X', 'BPHNANO_mc15.0']] COMMENT: reads old format file
#workflows[_wfn()] = ['EXONANOmc150X', ['TTbar_13p6_Summer24_AOD_140X', 'EXONANO_mc15.0']] COMMENT: reads old format file

# POG/PAG custom NANOs, data
_wfn.subnext()
Expand All @@ -663,6 +674,7 @@ def next(self, index: int = None) -> None:
_wfn() #workflows[_wfn()] = ['ScoutingNANOmonitordata150Xrun3', ['ScoutingPFMonitor_Run2025C_MINIAOD_150X', 'scoutingNANO_monitor_data15.0']] # noqa COMMENT: reads old format file
_wfn() #workflows[_wfn()] = ['ScoutingNANOmonitorWithPromptdata150Xrun3', ['ScoutingPFMonitor_Run2025C_MINIAOD_150X', 'scoutingNANO_monitorWithPrompt_data15.0']] # noqa COMMENT: reads old format file
_wfn() #workflows[_wfn()] = ['BPHNANOdata150Xrun3', ['JetMET1_Run2025C_MINIAOD_150X', 'BPHNANO_data15.0']] COMMENT: reads old format file
_wfn() #workflows[_wfn()] = ['EXONANOdata150Xrun3', ['Muon0_Run2025C_AOD_150X', 'EXONANO_data15.0']] COMMENT: reads old format file
workflows[_wfn()] = ['L1ScoutingNANOdata150Xrun3', ['L1Scouting2025RAW15.0', 'l1ScoutingNANO_data15.0']]
workflows[_wfn()] = ['L1ScoutingSelectionNANOdata150Xrun3', ['L1ScoutingSelection2025RAW15.0', 'l1ScoutingSelectionNANO_data15.0']]

Expand Down
1 change: 0 additions & 1 deletion DQM/DTMonitorClient/BuildFile.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,5 @@
<use name="CalibMuon/DTCalibration"/>
<use name="DQM/DTMonitorModule"/>
<use name="CondFormats/RunInfo"/>
<use name="PhysicsTools/TensorFlow"/>
<use name="boost"/>
<use name="rootgraphics"/>
3 changes: 2 additions & 1 deletion DQM/HLTEvF/plugins/HLTObjectMonitor.cc
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ HLTObjectMonitor::HLTObjectMonitor(const edm::ParameterSet& iConfig)
//now do what ever initialization is needed
debugPrint = false;

topDirectoryName = "HLT/ObjectMonitor";
topDirectoryName = iConfig.getUntrackedParameter<string>("topFolderName");
mainShifterFolder = topDirectoryName + "/MainShifter";
backupFolder = topDirectoryName + "/Backup";

Expand Down Expand Up @@ -866,6 +866,7 @@ double HLTObjectMonitor::get_wall_time() {
void HLTObjectMonitor::fillDescriptions(edm::ConfigurationDescriptions& descriptions) {
edm::ParameterSetDescription desc;
desc.add<std::string>("processName", "HLT");
desc.addUntracked<std::string>("topFolderName", "HLT/ObjectMonitor");

auto addPSet = [](edm::ParameterSetDescription& desc, const std::string& name) {
edm::ParameterSetDescription pset;
Expand Down
11 changes: 10 additions & 1 deletion DQM/Integration/python/clients/ngt_dqm_sourceclient-live_cfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
if unitTest:
process.load("DQM.Integration.config.unitteststreamerinputsource_cfi")
from DQM.Integration.config.unitteststreamerinputsource_cfi import options
process.source.streamLabel = 'streamDQMOnlineScouting' # in test mode use DQMOnlineScouting streamer
process.source.streamLabel = "streamDQMTestDataScouting"
else:
# for live online DQM in P5
process.load("DQM.Integration.config.inputsource_cfi")
Expand Down Expand Up @@ -105,6 +105,13 @@
process.dqmHLTFiltersDQMonitor.folderName = "NGT/Filters"
process.dqmHLTFiltersDQMonitor.lightMonitor = True

# Object Monitoring
process.load("DQM.HLTEvF.FourVectorHLT_cfi")
process.hltResults.topFolderName = cms.untracked.string("NGT/FourVectorHLT")

process.load("DQM.HLTEvF.HLTObjectMonitor_cfi")
process.hltObjectMonitor.topFolderName = cms.untracked.string("NGT/ObjectMonitor")

process.p = cms.Path(process.dqmcommon *
process.hltOnlineBeamSpot *
process.ScoutingTrackMonitorOnline *
Expand All @@ -114,6 +121,8 @@
process.ScoutingDileptonMonitorOnline *
process.ScoutingMuonPropertiesMonitorOnline *
process.ScoutingPi0MonitorOnline *
process.hltResults *
process.hltObjectMonitor *
process.dqmHLTFiltersDQMonitor)

### process customizations included here
Expand Down
4 changes: 2 additions & 2 deletions DQM/SiTrackerPhase2/test/dqmstep_phase2tk_cfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# step3_pre4_inDQM.root - input for the next step in the DQM sequence, harvestingstep_phase2tk_cfg.py

# Source: /local/reps/CMSSW/CMSSW/Configuration/Applications/python/ConfigBuilder.py,v
# with command line options: step3 --conditions auto:phase2_realistic_T21 -s RAW2DIGI,L1Reco,RECO,RECOSIM,VALIDATION:@phase2Validation,DQM:@phase2 --datatier DQMIO -n 10 --geometry Extended2026D98 --era Phase2C11M9 --eventcontent DQM --no_exec
# with command line options: step3 --conditions auto:phase2_realistic_T21 -s RAW2DIGI,L1Reco,RECO,RECOSIM,VALIDATION:@phase2Validation,DQM:@phase2 --datatier DQMIO -n 10 --geometry Extended2026D110 --era Phase2C11M9 --eventcontent DQM --no_exec
import FWCore.ParameterSet.Config as cms

from Configuration.Eras.Era_Phase2C11M9_cff import Phase2C11M9
Expand Down Expand Up @@ -41,7 +41,7 @@

# Input source
process.source = cms.Source("PoolSource",
fileNames = cms.untracked.vstring('/store/relval/CMSSW_14_0_0_pre2/RelValTTbar_14TeV/GEN-SIM-DIGI-RAW/PU_133X_mcRun4_realistic_v1_STD_2026D98_PU200_RV229-v1/2580000/0b2b0b0b-f312-48a8-9d46-ccbadc69bbfd.root'),
fileNames = cms.untracked.vstring('/store/relval/CMSSW_15_0_0/RelValTTbar_14TeV/GEN-SIM-RECO/PU_141X_mcRun4_realistic_v3_STD_Run4D110_PU-v3/2580000/01972f5c-64f5-4fea-890d-ffa3bb070b77.root'),
secondaryFileNames = cms.untracked.vstring()
)

Expand Down
7 changes: 4 additions & 3 deletions DQM/SiTrackerPhase2/test/harvestingstep_phase2tk_cfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root - for local visualization of histograms

# Source: /local/reps/CMSSW/CMSSW/Configuration/Applications/python/ConfigBuilder.py,v
# with command line options: step4 --conditions auto:phase2_realistic_T21 -s HARVESTING:@phase2Validation+@phase2+@miniAODValidation+@miniAODDQM --scenario pp --filetype DQM --geometry ExtendedRun4D98 --era Phase2C11M9 --mc -n -1 --no_exec
# with command line options: step4 --conditions auto:phase2_realistic_T21 -s HARVESTING:@phase2Validation+@phase2+@miniAODValidation+@miniAODDQM --scenario pp --filetype DQM --geometry ExtendedRun4D110 --era Phase2C11M9 --mc -n -1 --no_exec
import FWCore.ParameterSet.Config as cms

from Configuration.Eras.Era_Phase2C11M9_cff import Phase2C11M9
Expand All @@ -27,6 +27,7 @@
process.load('Configuration.StandardSequences.DQMSaverAtRunEnd_cff')
process.load('Configuration.StandardSequences.Harvesting_cff')
process.load('Configuration.StandardSequences.FrontierConditions_GlobalTag_cff')
#process.load('Validation.SiTrackerPhase2V.Phase2OTEffClient_cff')

process.maxEvents = cms.untracked.PSet(
input = cms.untracked.int32(8000),
Expand Down Expand Up @@ -88,5 +89,5 @@
process.dqmsave_step = cms.Path(process.DQMSaver)

# Schedule definition
#process.schedule = cms.Schedule(process.trackerphase2ValidationHarvesting_step,process.dqmsave_step)
process.schedule = cms.Schedule(process.dqmsave_step)
#process.schedule = cms.Schedule(process.trackerphase2ValidationHarvesting_step, process.dqmsave_step)
process.schedule = cms.Schedule(process.dqmsave_step)
Original file line number Diff line number Diff line change
Expand Up @@ -395,10 +395,13 @@ def _copyIfExists(mod, pset, name):
from Configuration.ProcessModifiers.seedingDeepCore_cff import seedingDeepCore
seedingDeepCore.toReplaceWith(TrackSeedMonSequence,_seedingDeepCore_TrackSeedMonSequence)

from Configuration.ProcessModifiers.seedingLST_cff import seedingLST
from Configuration.ProcessModifiers.trackingLST_cff import trackingLST
trackingLST.toModify(locals()["TrackSeedMonhighPtTripletStep"],
SeedProducer = "lstInputProducer"
)
_LST_TrackSeedMonSequence = TrackSeedMonSequence.copyAndExclude([locals()["TrackSeedMoninitialStep"]])
(seedingLST | trackingLST).toReplaceWith(TrackSeedMonSequence, _LST_TrackSeedMonSequence)

TrackingDQMSourceTier0 += TrackSeedMonSequence

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
// system includes
#include <memory>

// framework stuff
#include "FWCore/Framework/interface/Frameworkfwd.h"
#include "FWCore/Framework/interface/global/EDProducer.h"
#include "FWCore/Framework/interface/Event.h"
#include "FWCore/Framework/interface/MakerMacros.h"
#include "FWCore/ParameterSet/interface/ParameterSet.h"
#include "FWCore/Framework/interface/ESHandle.h"

// data formats
#include "DataFormats/Common/interface/ValueMap.h"
#include "DataFormats/TrackerRecHit2D/interface/FastTrackerRecHit.h"
#include "DataFormats/TrackerRecHit2D/interface/FastTrackerRecHitCollection.h"
#include "DataFormats/TrajectorySeed/interface/TrajectorySeedCollection.h"

class FastTrackerRecHitMaskFromSeedProducer : public edm::global::EDProducer<> {
public:
explicit FastTrackerRecHitMaskFromSeedProducer(const edm::ParameterSet&);

~FastTrackerRecHitMaskFromSeedProducer() override {}

void produce(edm::StreamID, edm::Event&, edm::EventSetup const&) const override;

static void fillDescriptions(edm::ConfigurationDescriptions& descriptions);

private:
// an alias
using QualityMaskCollection = std::vector<unsigned char>;

// tokens
edm::EDGetTokenT<TrajectorySeedCollection> trajectories_;

edm::EDGetTokenT<FastTrackerRecHitCollection> recHits_;

edm::EDGetTokenT<std::vector<bool> > oldHitMaskToken_;

edm::EDPutTokenT<std::vector<bool> > collectedHits_;
};

FastTrackerRecHitMaskFromSeedProducer::FastTrackerRecHitMaskFromSeedProducer(const edm::ParameterSet& iConfig)
: trajectories_(consumes(iConfig.getParameter<edm::InputTag>("trajectories"))),
recHits_(consumes(iConfig.getParameter<edm::InputTag>("recHits"))),
collectedHits_(produces()) {
auto const& oldHitRemovalInfo = iConfig.getParameter<edm::InputTag>("oldHitRemovalInfo");
if (!oldHitRemovalInfo.label().empty()) {
oldHitMaskToken_ = consumes<std::vector<bool> >(oldHitRemovalInfo);
}
}

void FastTrackerRecHitMaskFromSeedProducer::produce(edm::StreamID, edm::Event& iEvent, edm::EventSetup const&) const {
// the product

std::vector<bool> collectedHits;

// input

auto const& recHits = iEvent.get(recHits_);

if (!oldHitMaskToken_.isUninitialized()) {
auto const& oldHitMasks = iEvent.get(oldHitMaskToken_);
collectedHits.insert(collectedHits.begin(), oldHitMasks.begin(), oldHitMasks.end());
}
collectedHits.resize(recHits.size(), false);

auto const& seeds = iEvent.get(trajectories_);

// loop over seed tracks and mask hits
for (auto const& seed : seeds) {
for (auto const& hit : seed.recHits()) {
if (!hit.isValid())
continue;
const FastTrackerRecHit& fasthit = static_cast<const FastTrackerRecHit&>(hit);
// note: for matched hits nIds() returns 2, otherwise 1
for (unsigned id_index = 0; id_index < fasthit.nIds(); id_index++) {
collectedHits[unsigned(fasthit.id(id_index))] = true;
}
}
}

iEvent.emplace(collectedHits_, std::move(collectedHits));
}

void FastTrackerRecHitMaskFromSeedProducer::fillDescriptions(edm::ConfigurationDescriptions& descriptions) {
edm::ParameterSetDescription desc;
desc.add<edm::InputTag>("trajectories", edm::InputTag("initialStepSeeds"));
desc.add<edm::InputTag>("recHits", edm::InputTag("fastTrackerRecHits"));
desc.add<edm::InputTag>("oldHitRemovalInfo", edm::InputTag(""));
descriptions.addWithDefaultLabel(desc);
}

DEFINE_FWK_MODULE(FastTrackerRecHitMaskFromSeedProducer);
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import FWCore.ParameterSet.Config as cms

from FastSimulation.Tracking.fastTrackerRecHitMaskFromSeedProducer_cfi import fastTrackerRecHitMaskFromSeedProducer

def maskProducerFromSeedClusterRemover(seedClusterRemover):
maskProducer = fastTrackerRecHitMaskFromSeedProducer.clone(
trajectories = seedClusterRemover.trajectories,
)
if(hasattr(seedClusterRemover,"oldClusterRemovalInfo")):
maskProducer.oldHitRemovalInfo = cms.InputTag(seedClusterRemover.oldClusterRemovalInfo.getModuleLabel().replace("Clusters","Masks"))
return maskProducer
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ def create_remote_process(local_process, modules_to_run, remote_process_name, lo
setattr(remote_process, module, getattr(local_process, module).clone())
for module in local_process.es_producers.keys():
setattr(remote_process, module, getattr(local_process, module).clone())

if hasattr(local_process, "MessageLogger"):
remote_process.MessageLogger = local_process.MessageLogger.clone()

remote_process.load("HeterogeneousCore.MPIServices.MPIService_cfi")
remote_process.MPIService.pmix_server_uri = "file:server.uri"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ def split_remote(local_process, args, cpp_names_of_the_products):
controller_name = add_controller_to_local(local_process, args.remote_process_name)
remote_process = create_remote_process(local_process, modules_to_offload, args.remote_process_name, local_process.name_())

mpi_path_modules_local = [controller_name]
mpi_path_modules_remote = []
mpi_path_modules_local = [[controller_name] for _ in range(len(groups))]
mpi_path_modules_remote = [[] for _ in range(len(groups))]

instance = 1

Expand All @@ -65,7 +65,6 @@ def split_remote(local_process, args, cpp_names_of_the_products):

# send the data needed by offloaded modules from local to remote
remote_filters_by_group = [[] for _ in range(len(groups))]
local_sender_by_group = [[] for _ in range(len(groups))]
for local_dependency, group_indices in producer_to_groups.items():
first_dependency_in_a_group = [groups[i][0] for i in group_indices]
capture_name = f"activityCaptureBefore{local_dependency.title()}"
Expand All @@ -89,15 +88,13 @@ def split_remote(local_process, args, cpp_names_of_the_products):
# create filter for the path state
filter_name = f"activityFilterAfter{local_dependency.title()}"
add_activity_filter(remote_process, local_dependency, filter_name)
setattr(remote_process, local_dependency, receiver)
for group_idx in group_indices:
remote_filters_by_group[group_idx].append(filter_name)
local_sender_by_group[group_idx].append(sender_name)

setattr(remote_process, local_dependency, receiver)
mpi_path_modules_local[group_idx].append(sender_name)
mpi_path_modules_remote[group_idx].append(local_dependency)

instance += 1
mpi_path_modules_local.append(sender_name)
mpi_path_modules_remote.append(local_dependency)

# Handle the case when onle local product is needed by multiple remote groups
# For sending from local process - if data is needed by multiple paths, insert additional path state capture and additional sender per group
Expand Down Expand Up @@ -130,11 +127,10 @@ def split_remote(local_process, args, cpp_names_of_the_products):
filter_name = f"activityFilterBefore{args.remote_process_name.title()}Group{group_idx}"
add_activity_filter(remote_process, receiver_name, filter_name)
remote_filters_by_group[group_idx].append(filter_name)
local_sender_by_group[group_idx].append(sender_name)

instance += 1
mpi_path_modules_local.append(sender_name)
mpi_path_modules_remote.append(receiver_name)
mpi_path_modules_local[group_idx].append(sender_name)
mpi_path_modules_remote[group_idx].append(receiver_name)


per_group_remote_captures = [[] for _ in range(len(groups))]
Expand All @@ -148,18 +144,23 @@ def split_remote(local_process, args, cpp_names_of_the_products):
setattr(remote_process, remote_capture_name, cms.EDProducer("PathStateCapture"))
per_group_remote_captures[group_idx].append(remote_capture_name)

if len(mpi_path_modules_remote[group_idx]) != 0:
sender_upstream = mpi_path_modules_remote[group_idx][-1]
else:
sender_upstream = "source"

sender = create_group_sender(
group=group,
all_products=cpp_names_of_the_products,
instance=instance,
upstream_module=mpi_path_modules_remote[0],
upstream_module=sender_upstream,
path_state_capture=remote_capture_name,
)
sender_name = f"mpiSender{args.remote_process_name.title()}Group{group_idx}"
setattr(remote_process, sender_name, sender)

if len(local_sender_by_group[group_idx]) != 0:
receiver_upstream = local_sender_by_group[group_idx][-1]
if len(mpi_path_modules_local[group_idx]) != 0:
receiver_upstream = mpi_path_modules_local[group_idx][-1]
else:
receiver_upstream = controller_name

Expand All @@ -180,8 +181,8 @@ def split_remote(local_process, args, cpp_names_of_the_products):
add_activity_filter(local_process, receiver_name, filter_name)
insert_modules_before(local_process, getattr(local_process, group[0]), getattr(local_process, filter_name))

mpi_path_modules_remote.append(sender_name)
mpi_path_modules_local.append(receiver_name)
mpi_path_modules_remote[group_idx].append(sender_name)
mpi_path_modules_local[group_idx].append(receiver_name)


for i, offloaded_module in enumerate(group):
Expand All @@ -198,9 +199,9 @@ def split_remote(local_process, args, cpp_names_of_the_products):
delattr(local_process, product)

# add all needed paths to the process and schedule them
make_new_path(local_process, f"Offload{args.remote_process_name.title()}", mpi_path_modules_local)
make_new_path(remote_process, "MPIPath", mpi_path_modules_remote)
for i, group in enumerate(groups):
make_new_path(local_process, f"Offload{args.remote_process_name.title()}Group{i}", mpi_path_modules_local[i])
make_new_path(remote_process, f"MPIPathGroup{i}", mpi_path_modules_remote[i])
make_new_path(remote_process, args.remote_process_name.title()+"RemoteOffloadedSequence"+str(i), remote_filters_by_group[i]+group+per_group_remote_captures[i])

if args.verbose:
Expand Down
Loading