Add an MPISenderPortable and MPIReceiverPortable modules to send/receive arbitrary device collections#50503
Add an MPISenderPortable and MPIReceiverPortable modules to send/receive arbitrary device collections#50503ghyls wants to merge 4 commits into
MPISenderPortable and MPIReceiverPortable modules to send/receive arbitrary device collections#50503Conversation
|
cms-bot internal usage |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/48670
|
|
Milestone for this pull request has been moved to CMSSW_17_0_X. Please open a backport if it should also go in to CMSSW_16_1_X. |
|
please test |
| "On the synchronous CPU backend, only types whose CopyToHost<T>::copyAsync returns T itself are " | ||
| "supported."); |
There was a problem hiding this comment.
This isn't really the condition though at this level of abstraction, right? I think we even have the CopyToHost<T> not being specialized for PortableHostCollection, so strictly speaking there is no copyAsync() that would return itself. How about
| "On the synchronous CPU backend, only types whose CopyToHost<T>::copyAsync returns T itself are " | |
| "supported."); | |
| "On the synchronous CPU backend, the host and device product types must be the same."); |
or more generic
| "On the synchronous CPU backend, only types whose CopyToHost<T>::copyAsync returns T itself are " | |
| "supported."); | |
| "On a backend where host data products are used directly on the device, the host and device product types must be the same."); |
?
There was a problem hiding this comment.
I see. Thanks for the suggestions. Just changed it.
|
+1 Size: This PR adds an extra 16KB to repository Comparison SummarySummary:
|
| } else { | ||
| return nullptr; | ||
| } | ||
| } else { | ||
| return nullptr; | ||
| } |
There was a problem hiding this comment.
Could be simplified as
| } else { | |
| return nullptr; | |
| } | |
| } else { | |
| return nullptr; | |
| } | |
| } | |
| } | |
| return nullptr; |
?
There was a problem hiding this comment.
On second thought, rather than using if constexpr to make the function return nullptr if the conditions are not met, can it be replaced by requires so that the method is simply not available ?
| // The plugin is registered under two keys: | ||
| // | ||
| // 1. mangled typeid name of TYPE_HOST: used to look up the device serialiser | ||
| // for a host type. | ||
| // 2. EDM_STRINGIZE(TYPE_DEVICE): used to look up the device serialiser | ||
| #define DEFINE_TRIVIAL_SERIALISER_PORTABLE_PLUGIN(TYPE_HOST, TYPE_DEVICE) \ | ||
| DEFINE_EDM_PLUGIN(ALPAKA_ACCELERATOR_NAMESPACE::ngt::SerialiserFactoryDevice, \ | ||
| ALPAKA_ACCELERATOR_NAMESPACE::ngt::Serialiser<ALPAKA_ACCELERATOR_NAMESPACE::TYPE_DEVICE>, \ | ||
| typeid(TYPE_HOST).name()); \ | ||
| DEFINE_EDM_PLUGIN2(ALPAKA_ACCELERATOR_NAMESPACE::ngt::SerialiserFactoryDevice, \ | ||
| ALPAKA_ACCELERATOR_NAMESPACE::ngt::Serialiser<ALPAKA_ACCELERATOR_NAMESPACE::TYPE_DEVICE>, \ | ||
| EDM_STRINGIZE(TYPE_DEVICE)) |
There was a problem hiding this comment.
I don't understand why we need two names, and specifically these two names.
Can we not use simply the mangled names for both ?
There was a problem hiding this comment.
I was using EDM_STRINGIZE(TYPE_DEVICE) because just changing that to typeid(ALPAKA_ACCELERATOR_NAMESPACE::TYPE_DEVICE).name() would lead to the "host" version of the type being registered twice when the device is the CPU: Once by DEFINE_EDM_PLUGIN, and another by DEFINE_EDM_PLUGIN2.
Making the TYPE_DEVICE version registered only on the GPU backend solved this issue.
| @@ -3,9 +3,9 @@ | |||
| #include <catch2/catch_all.hpp> | |||
|
|
|||
| #include <Eigen/Dense> | |||
| #include <string> | |||
There was a problem hiding this comment.
Can you move the std headers at the top ?
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/49986 ERROR: Unable to merge PR. See log https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/49986/cms-checkout-topic.log |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/49990
|
|
Pull request #50503 was updated. @Dr15Jones, @Moanwar, @civanch, @cmsbuild, @fwyzard, @jfernan2, @kpedro88, @makortel, @mandrenguyen, @mdhildreth, @smuzaffar, @srimanob can you please check and sign again. |
|
please test |
|
-1 Failed Tests: UnitTests Failed Unit TestsI found 2 errors in the following unit tests: ---> test TestHeterogeneousCoreTrivialSerialisationGenericClonerPortable had ERRORS ---> test testMPISoADeviceTransfer had ERRORS Comparison SummarySummary:
|
|
please test To refresh the tests and get a test area with the latest UCX library. |
|
-1 Failed Tests: UnitTests Failed Unit TestsI found 2 errors in the following unit tests: ---> test TestHeterogeneousCoreTrivialSerialisationGenericClonerPortable had ERRORS ---> test testMPISoADeviceTransfer had ERRORS Comparison SummarySummary:
|
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/50126
Code check has found code style and quality issues which could be resolved by applying following patch(s)
|
…am and limited transformers
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50503/50182
|
|
Pull request #50503 was updated. @Dr15Jones, @Moanwar, @civanch, @cmsbuild, @fwyzard, @jfernan2, @kpedro88, @makortel, @mandrenguyen, @mdhildreth, @smuzaffar, @srimanob can you please check and sign again. |
|
please test |
|
+1 Size: This PR adds an extra 128KB to repository Comparison SummarySummary:
|
|
enable gpu |
|
please test |
|
|
||
| <test name="TestHeterogeneousCoreTrivialSerialisationGenericCloner" command="cmsRun ${LOCALTOP}/src/HeterogeneousCore/TrivialSerialisation/test/testGenericCloner_cfg.py"/> | ||
| <test name="TestHeterogeneousCoreTrivialSerialisationGenericClonerHost" command="cmsRun ${LOCALTOP}/src/HeterogeneousCore/TrivialSerialisation/test/testGenericClonerHost_cfg.py"/> | ||
| <test name="TestHeterogeneousCoreTrivialSerialisationGenericClonerPortable" command="cmsRun ${LOCALTOP}/src/HeterogeneousCore/TrivialSerialisation/test/testGenericClonerPortable_cfg.py"/> |
There was a problem hiding this comment.
This test runs only for the default back-end, it does not for any alpaka backends.
I think to run on all backends you need
| <test name="TestHeterogeneousCoreTrivialSerialisationGenericClonerPortable" command="cmsRun ${LOCALTOP}/src/HeterogeneousCore/TrivialSerialisation/test/testGenericClonerPortable_cfg.py"/> | |
| <test name="TestHeterogeneousCoreTrivialSerialisationGenericClonerPortable" command="cmsRun ${LOCALTOP}/src/HeterogeneousCore/TrivialSerialisation/test/testGenericClonerPortable_cfg.py"> | |
| <!-- dependence and flag only to trigger the unit test for each Alpaka backend --> | |
| <use name="alpaka"/> | |
| <flags ALPAKA_BACKENDS="1"/> | |
| </test> |
| "copyAsync must take (Queue&, edm::WrapperBase const&) and return std::any"); | ||
| static_assert(std::is_same_v<std::invoke_result_t<TTransform, std::any&, std::shared_ptr<EDMetadata>>, | ||
| std::unique_ptr<edm::WrapperBase>>, | ||
| "transform must take (std::any& and std::shared_ptr<EDMetadata>) and return " |
There was a problem hiding this comment.
| "transform must take (std::any& and std::shared_ptr<EDMetadata>) and return " | |
| "transform must take (std::any&, std::shared_ptr<EDMetadata>) and return " |
for consistency with the previous message.
| // product is a device product. Types without this suffix are resolved | ||
| // as host or ROOT types. | ||
| static std::string const kAlpakaNamespacePlaceholder = "ALPAKA_ACCELERATOR_NAMESPACE::"; | ||
| bool const isDeviceType = type.compare(0, kAlpakaNamespacePlaceholder.size(), kAlpakaNamespacePlaceholder) == 0; |
There was a problem hiding this comment.
ALPAKA_ACCELERATOR_NAMESPACE is not necessarily at the beginning, and can be present more than once.
The code should search for that string anywhere in the type, and replace all instances with the actual backend namespace.
| continue; | ||
| } | ||
|
|
||
| // Check 2: Lookup a host serialiser registered in the host |
There was a problem hiding this comment.
What about a portable serialiser for a host product?
| edm::TypeWithDict wrappedTwd = edm::TypeWithDict::byName("edm::Wrapper<" + bareType + ">"); | ||
| LogDebug("MPISenderPortable") << "looking for ROOT serialisation of type \"" << type << "\""; | ||
| if (!twd || !wrappedTwd.getClass()) { | ||
| throw cms::Exception("MPISenderPortable") |
There was a problem hiding this comment.
You also need to check if the type that the type is not transient.
4 participants
PR description:
These PR includes two separate developments, one of which requires the other.
Enable the registration at runtime of DtoH and HtoD product transformations:
Add an
MPISenderPortableandMPIReceiverPortablemodules to send/receive device collectionsMPISenderPortable.ccandMPIReceiverPortable.cc, which can send/receive device runtime-typed device collections for which a device TrivialSerialiser plugin exists, directly to/from device memory.PR validation:
A
GenericClonerPortabletest module is introduced to demonstrate the D to H and H to D transformation registrations at runtime. The module clones a host or a device product and registers the H to D (or D to H) transformation for it. The test is configured viatestGenericClonerDevice.pyA small test is added to
FWCore/Framework/test/stream_producer_catch2.ccto test the non-templatedregisterTransformAsyncoverload added toFramework/interface/stream/implementors.h.The Portable MPI modules (
MPISenderPortable.ccandMPIReceiverPortable.cc) are tested via the two new configurations added toHeterogeneousCore/MPICore/test.Backport
16_0_Xand16_1_Xfor it to be used in the NGT demonstrator during data taking this year.