Skip to content

upgrade EvtGen interface for v03.00.00 #51029

Open
kawaho wants to merge 9 commits into
cms-sw:CMSSW_17_0_Xfrom
kawaho:EvtGev3
Open

upgrade EvtGen interface for v03.00.00 #51029
kawaho wants to merge 9 commits into
cms-sw:CMSSW_17_0_Xfrom
kawaho:EvtGev3

Conversation

@kawaho

@kawaho kawaho commented May 25, 2026

Copy link
Copy Markdown

PR description:

Resolves #50935

This commits upgrade EvtGen interface to v03.00.00 in CMSSW, which

  1. is now thread-safe, allowing multithreading in production
  2. adds new final state photon radiation models in Sherpa/Photon++ and Vincia
  3. includes new physics models and decay table implemented by Belle 2

This commit depends on a new cmssw external build cms-sw/cmsdist#10580

PR validation:

Tested by producing a small TTTo2L2Nu Pythia+EvtGen sample locally with multiple threads, using a similar the same fragment as 1 but with PythiaConcurrentHadronizerFilter.

One remaining issue is that the EvtGen interface now calls ThreadHandoff implemented for GEANT4 to ensure a stream started in a certain thread stays in the same thread. Shall we move ThreadHandoff under FWCore/Concurrency instead, so packages with similar concurrency implementation (one instance per thread/use of thread_local) can use the same module? @makortel @Dr15Jones, please advise.

@cmsbuild

cmsbuild commented May 25, 2026

Copy link
Copy Markdown
Contributor

cms-bot internal usage

@cmsbuild

Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51029/49473

ERROR: Build errors found during clang-tidy run.

src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
Suppressed 441 warnings (441 in non-user code).
--
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:35:40: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   35 |   void setSeed(unsigned long int seed) override;
      |                                        ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:338:29: error: no member named 'getSherpaPhotonsModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  338 |     radCorrEngine = genList.getSherpaPhotonsModel(1e-7, 1, 0);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:340:29: error: no member named 'getVinciaQEDModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  340 |     radCorrEngine = genList.getVinciaQEDModel(1.0e-7);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:534:24: error: non-const lvalue reference to type 'EvtDecayTable' cannot bind to a temporary of type 'EvtDecayTable *' [clang-diagnostic-error]
  534 |         EvtDecayTable& evtDecayTable = EvtDecayTable::getInstance();
      |                        ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Suppressed 1178 warnings (1174 in non-user code, 4 NOLINT).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

<use name="FWCore/Concurrency"/>
<use name="FWCore/ParameterSet"/>
<use name="FWCore/Framework"/>
<use name="SimG4Core/Application"/>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(as I commented already in the earlier issue) I think this dependence on SimG4Core should be avoided. I opened #51041 to move the ThreadHandoff to FWCore/Utilities.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, merged

@cmsbuild

Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51029/49482

ERROR: Build errors found during clang-tidy run.

src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
Suppressed 441 warnings (441 in non-user code).
--
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:35:40: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   35 |   void setSeed(unsigned long int seed) override;
      |                                        ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:338:29: error: no member named 'getSherpaPhotonsModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  338 |     radCorrEngine = genList.getSherpaPhotonsModel(1e-7, 1, 0);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:340:29: error: no member named 'getVinciaQEDModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  340 |     radCorrEngine = genList.getVinciaQEDModel(1.0e-7);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:534:24: error: non-const lvalue reference to type 'EvtDecayTable' cannot bind to a temporary of type 'EvtDecayTable *' [clang-diagnostic-error]
  534 |         EvtDecayTable& evtDecayTable = EvtDecayTable::getInstance();
      |                        ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Suppressed 1178 warnings (1174 in non-user code, 4 NOLINT).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

@cmsbuild

Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51029/49483

ERROR: Build errors found during clang-tidy run.

src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
Suppressed 441 warnings (441 in non-user code).
--
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:35:40: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   35 |   void setSeed(unsigned long int seed) override;
      |                                        ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/interface/myEvtRandomEngine.h:37:38: error: only virtual member functions can be marked 'override' [clang-diagnostic-error]
   37 |   unsigned long int lastSeed() const override;
      |                                      ^~~~~~~~
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:338:29: error: no member named 'getSherpaPhotonsModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  338 |     radCorrEngine = genList.getSherpaPhotonsModel(1e-7, 1, 0);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:340:29: error: no member named 'getVinciaQEDModel' in 'EvtExternalGenList' [clang-diagnostic-error]
  340 |     radCorrEngine = genList.getVinciaQEDModel(1.0e-7);
      |                     ~~~~~~~ ^
src/GeneratorInterface/EvtGenInterface/plugins/EvtGen/EvtGenInterface.cc:534:24: error: non-const lvalue reference to type 'EvtDecayTable' cannot bind to a temporary of type 'EvtDecayTable *' [clang-diagnostic-error]
  534 |         EvtDecayTable& evtDecayTable = EvtDecayTable::getInstance();
      |                        ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Suppressed 1178 warnings (1174 in non-user code, 4 NOLINT).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

@cmsbuild

Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51029/49768

  • Found files with invalid states:
    • GeneratorInterface/Pythia8Interface/interface/Pythia8EvtGenPatched.h:

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@lviliani

Copy link
Copy Markdown
Contributor

Hi @kawaho, I was wondering if this new interface would be still compatible with older EvtGen versions. I think we should not completely switch to the new version until it's fully validated, so back-compatibility is important.

If it's not, maybe we should consider having two different interfaces, such as EvtGenInterface and EvtGen3Interface as we did for HepMC.

@kawaho

kawaho commented Jun 22, 2026

Copy link
Copy Markdown
Author

Hi @lviliani , indeed, this interface will not be compatible with older EvtGen versions. In principle, I can move things to EvtGen3Interface but the issue is the new EvtGen also requires a patch to the existing Pythia8 in cmssw (See cms-sw/cmsdist#10580). It might not be feasible to maintain both EvtGen v2 and EvtGen v3. Let me know what you think.

@lviliani

Copy link
Copy Markdown
Contributor

Hi @lviliani , indeed, this interface will not be compatible with older EvtGen versions. In principle, I can move things to EvtGen3Interface but the issue is the new EvtGen also requires a patch to the existing Pythia8 in cmssw (See cms-sw/cmsdist#10580). It might not be feasible to maintain both EvtGen v2 and EvtGen v3. Let me know what you think.

Right, then we have to discuss how to move forward if we can't have both versions in the same release. EvtGen 3 should be carefully validated before it's used in production.

One possibility is to follow the same approach that was recently used for the Pythia8 update, which is including EvtGen 3 in the next CMSSW_17_0_X pre-release (I think it's going to be pre3) and validate it before CMSSW_17_0_0 is ready, which should happen in November 2026 according to the schedule. If we don't manage to do the validation, we revert back to EvtGen 2.

Tagging also @mandrenguyen and @ftenchini to hear their opinion, given that we discussed a similar approach for pythia.

@kawaho

kawaho commented Jun 24, 2026

Copy link
Copy Markdown
Author

Hi @lviliani , indeed, this interface will not be compatible with older EvtGen versions. In principle, I can move things to EvtGen3Interface but the issue is the new EvtGen also requires a patch to the existing Pythia8 in cmssw (See cms-sw/cmsdist#10580). It might not be feasible to maintain both EvtGen v2 and EvtGen v3. Let me know what you think.

Right, then we have to discuss how to move forward if we can't have both versions in the same release. EvtGen 3 should be carefully validated before it's used in production.

One possibility is to follow the same approach that was recently used for the Pythia8 update, which is including EvtGen 3 in the next CMSSW_17_0_X pre-release (I think it's going to be pre3) and validate it before CMSSW_17_0_0 is ready, which should happen in November 2026 according to the schedule. If we don't manage to do the validation, we revert back to EvtGen 2.

Tagging also @mandrenguyen and @ftenchini to hear their opinion, given that we discussed a similar approach for pythia.

Hi @lviliani , is there a standard checklist of things I need to do for the validation? I imagine on the physics side, we should check the differences between v2 and v3 to be expected in maybe a selected list of Rivet routines. Are there other checks needed in addition? I would like to understand the amount of work needed and to allocate my time accordingly. Thanks!

@lviliani

Copy link
Copy Markdown
Contributor

Hi @lviliani , indeed, this interface will not be compatible with older EvtGen versions. In principle, I can move things to EvtGen3Interface but the issue is the new EvtGen also requires a patch to the existing Pythia8 in cmssw (See cms-sw/cmsdist#10580). It might not be feasible to maintain both EvtGen v2 and EvtGen v3. Let me know what you think.

Right, then we have to discuss how to move forward if we can't have both versions in the same release. EvtGen 3 should be carefully validated before it's used in production.
One possibility is to follow the same approach that was recently used for the Pythia8 update, which is including EvtGen 3 in the next CMSSW_17_0_X pre-release (I think it's going to be pre3) and validate it before CMSSW_17_0_0 is ready, which should happen in November 2026 according to the schedule. If we don't manage to do the validation, we revert back to EvtGen 2.
Tagging also @mandrenguyen and @ftenchini to hear their opinion, given that we discussed a similar approach for pythia.

Hi @lviliani , is there a standard checklist of things I need to do for the validation? I imagine on the physics side, we should check the differences between v2 and v3 to be expected in maybe a selected list of Rivet routines. Are there other checks needed in addition? I would like to understand the amount of work needed and to allocate my time accordingly. Thanks!

Hi @kawaho, in addition to the physics validation that you mentioned, we should make sure that workflows with EvtGen v3 can run successfully in central production up to NanoAOD.
And regarding the physics validation, I would recommend to get in touch with BPH, as they are the main customers of EvtGen in CMS.

@kawaho

kawaho commented Jun 26, 2026

Copy link
Copy Markdown
Author

Hi @lviliani , indeed, this interface will not be compatible with older EvtGen versions. In principle, I can move things to EvtGen3Interface but the issue is the new EvtGen also requires a patch to the existing Pythia8 in cmssw (See cms-sw/cmsdist#10580). It might not be feasible to maintain both EvtGen v2 and EvtGen v3. Let me know what you think.

Right, then we have to discuss how to move forward if we can't have both versions in the same release. EvtGen 3 should be carefully validated before it's used in production.
One possibility is to follow the same approach that was recently used for the Pythia8 update, which is including EvtGen 3 in the next CMSSW_17_0_X pre-release (I think it's going to be pre3) and validate it before CMSSW_17_0_0 is ready, which should happen in November 2026 according to the schedule. If we don't manage to do the validation, we revert back to EvtGen 2.
Tagging also @mandrenguyen and @ftenchini to hear their opinion, given that we discussed a similar approach for pythia.

Hi @lviliani , is there a standard checklist of things I need to do for the validation? I imagine on the physics side, we should check the differences between v2 and v3 to be expected in maybe a selected list of Rivet routines. Are there other checks needed in addition? I would like to understand the amount of work needed and to allocate my time accordingly. Thanks!

Hi @kawaho, in addition to the physics validation that you mentioned, we should make sure that workflows with EvtGen v3 can run successfully in central production up to NanoAOD. And regarding the physics validation, I would recommend to get in touch with BPH, as they are the main customers of EvtGen in CMS.

I have started a thread on cms-talk: https://cms-talk.web.cern.ch/t/upgrading-evtgen-in-central-production/146386

@Kiarendil

Copy link
Copy Markdown

Hi

First, I am not sure I understand the current plan for the EvtGen 3 usage in terms of CMSSW releases and MC production campaigns. Are we going to do some backports to older versions, such as 15_0_X at least ? If not, which can be MC conditions and campaigns which target CMSSW_17? I mean, which GT, which Mini / Nano versions, etc?
Only some abstract Run-3 Ultra-Legacy, which is very unknown when and how to happen?

What about CMSSW_20 for Phase-2?

Regarding the physics validation from BPH POV – how can one test some private production with this EvtGen 3? If the way is to pull this PR to a custom CMSSW under lxplus etc, then again – which cmsDriver should be used ?

@Dr15Jones

Copy link
Copy Markdown
Contributor

If useful, it is possible to request a special integration build that contains EvtGen 3 plus all necessary CMSSW changes.

@kawaho

kawaho commented Jun 26, 2026

Copy link
Copy Markdown
Author

Hi @Dr15Jones I think that would be useful!

@Kiarendil

Copy link
Copy Markdown

If useful, it is possible to request a special integration build that contains EvtGen 3 plus all necessary CMSSW changes.

This is indeed a way, which I understand; though my question is more PPD-related about the matching of EvtGen 3 with some production campaign.

Well, I agree these questions can be separated , and one can do some more abstract tests / comparison of EvtGen 2 vs 3 performance under this IB release with cmsDriver for e.g. Summer 24 campaign.
But then the question of how to really use EvtGen 3 in production stands valid, and I think it is important to answer as well before doing IBs, as this determines CMSSW releases and back ports.
Sorry if I understand smth wrong.

@lviliani

Copy link
Copy Markdown
Contributor

@Kiarendil first we need to validate it, then we can discuss if we want to use it already in production or not, and in that case in which release.

@Dr15Jones we discussed that also for Pythia but then decided to not have a special build. But in this case, being a major upgrade, it might be useful indeed.

@Dr15Jones

Copy link
Copy Markdown
Contributor

@smuzaffar would be the one to ask to get an EvtGen3 IB. I'd suggest doing it in CMSSW_20_1.

@smuzaffar

smuzaffar commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@lviliani , we can setup a dedicated IB for EvtGen3 if that helps with the integration/testing.
For externals, its is easy to setup a dedicated branch. For cmssw, is it possible to get EvtGen3 related changes in master branch with some configuration flag and/or ifdef to selected old evtgen and new evtgen3 ? Otherwise we have to maintain a separate cmssw branch for EvtGen3 and deal with merge conflicts ( which for cmssw happens alot).

@Dr15Jones

Copy link
Copy Markdown
Contributor

maintain a separate cmssw branch for EvtGen3 and deal with merge conflicts ( which for cmssw happens alot).

But the merge conflicts would probably only happen if code in master was touching EvtGen related items, which seems unlikely.

@smuzaffar

Copy link
Copy Markdown
Contributor

maintain a separate cmssw branch for EvtGen3 and deal with merge conflicts ( which for cmssw happens alot).

But the merge conflicts would probably only happen if code in master was touching EvtGen related items, which seems unlikely.

yes that is true

@smuzaffar

Copy link
Copy Markdown
Contributor

code-checks with cms.week0.PR_7633f646/100.0-1bcf27b469f48c8773c80cbe8786c1d1

@cmsbuild

cmsbuild commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51029/50049

  • Found files with invalid states:
    • GeneratorInterface/Pythia8Interface/interface/Pythia8EvtGenPatched.h:

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@kawaho

kawaho commented Jul 9, 2026

Copy link
Copy Markdown
Author

Hi @smuzaffar , are there any updates on the special build for EvtGen? Do I need to do anything on my end? Thank you.

@smuzaffar

Copy link
Copy Markdown
Contributor

@kawaho , this should also go in 20.1.X release cycles ... right? If yes then I suggest that we first setup CMSSW_20_1_EVTGEN3_X IBs and test these changes there

@kawaho

kawaho commented Jul 14, 2026

Copy link
Copy Markdown
Author

@kawaho , this should also go in 20.1.X release cycles ... right? If yes then I suggest that we first setup CMSSW_20_1_EVTGEN3_X IBs and test these changes there

Hi @smuzaffar is there a guide on how to port these changes to another release? I have an old 17_0_X IB locally that does not exist anymore. Do I setup a newer 17_0_X IB locally or what should I do?

@Kiarendil

Kiarendil commented Jul 14, 2026

Copy link
Copy Markdown

Could anyone please suggest, If it would be straightforward to produce and validate the private MC in CMSSW_20_X ? Will all cmsDriver steps which work in current CMSSW, be working there ? E.g. SIM step, HLT step etc? My naive feeling that maybe 17_0_X build could be easier to produce private MC and play with it.

@smuzaffar

smuzaffar commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@kawaho , I have setup a CMSSW_20_1_EVTGEN3_X IB and I will include cms-sw/cmsdist#10580 for that IB. Once evtgent3 external is available in the CMSSW_20_1_EVTGEN3_X then you can just use that IB to do the local tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Testing thread-safe EvtGen v3.0.0-beta1 in CMSSW

7 participants