Opticks : GPU Optical Photon Simulation for Particle Physics with NVIDIA OptiX
Opticks is an open source project that integrates the NVIDIA OptiX GPU ray tracing engine with Geant4 toolkit based simulations. Massive parallelism brings drastic performance improvements with optical photon simulation speedup expected to exceed 1000 times Geant4 when using workstation GPUs. Optical photon simulation time becomes effectively zero compared to the rest of the simulation.
Optical photons from scintillation and Cherenkov processes are allocated, generated and propagated entirely on the GPU, minimizing transfer overheads and allowing CPU memory usage to be restricted to optical photons that hit photomultiplier tubes or other photon detectors. Collecting hits into standard Geant4 hit collections then allows the rest of the simulation chain to proceed unmodified.
Optical physics processes of scattering, absorption, reemission and boundary processes are implemented as CUDA OptiX programs based on the Geant4 implementations. Wavelength dependent material and surface properties as well as inverse cumulative distribution functions for reemission are interleaved into GPU textures providing fast interpolated property lookup or wavelength generation. Geometry is provided to OptiX in the form of CUDA programs that return bounding boxes for each primitive and single ray geometry intersection results. OptiX handles the creation and application of a choice of acceleration structures such as boundary volume heirarchies and the transparent use of multiple GPUs.
A major recent advance is the implementation of GPU ray tracing of complex constructive solid geometry (CSG) shapes. This enables fully automated translation of Geant4 geometries into GPU appropriate forms without approximation, greatly simplifying the adoption of Opticks.
Opticks is validated by comparison with Geant4 simulations. Use of common initial photons and random number sequences allows the two simulations to be run point-by-point aligned. Aligned running has reached near perfect equivalence with test geometries, for larger geometries it provides a powerful debugging tool.
OptiX supports interoperation with OpenGL and CUDA Thrust that has enabled unprecedented visualisations of photon propagations to be developed using OpenGL geometry shaders to provide interactive time scrubbing and CUDA Thrust photon indexing to provide interactive history selection.
Opticks : GPU Optical Photon Simulation for Particle Physics with NVIDIA OptiX
Opticks is an open source project that integrates the NVIDIA OptiX GPU ray tracing engine with Geant4 toolkit based simulations. Massive parallelism brings drastic performance improvements with optical photon simulation speedup expected to exceed 1000 times Geant4 when using workstation GPUs. Optical photon simulation time becomes effectively zero compared to the rest of the simulation.
Optical photons from scintillation and Cherenkov processes are allocated, generated and propagated entirely on the GPU, minimizing transfer overheads and allowing CPU memory usage to be restricted to optical photons that hit photomultiplier tubes or other photon detectors. Collecting hits into standard Geant4 hit collections then allows the rest of the simulation chain to proceed unmodified.
Optical physics processes of scattering, absorption, reemission and boundary processes are implemented as CUDA OptiX programs based on the Geant4 implementations. Wavelength dependent material and surface properties as well as inverse cumulative distribution functions for reemission are interleaved into GPU textures providing fast interpolated property lookup or wavelength generation.
Geometry is provided to OptiX in the form of CUDA programs that return bounding boxes for each primitive and single ray geometry intersection results. Some critical parts of the geometry such as photomultiplier tubes have been implemented analytically with the remainder being tesselated. OptiX handles the creation and application of a choice of acceleration structures such as boundary volume heirarchies and the transparent use of multiple GPUs.
OptiX supports interoperation with OpenGL and CUDA Thrust that has enabled unprecedented visualisations of photon propagations to be developed using OpenGL geometry shaders to provide interactive time scrubbing and CUDA Thrust photon indexing to provide interactive history selection.
Opticks : Optical Photon Simulation for High Energy Physics with OptiX
Opticks is an open source project that brings NVIDIA OptiX ray tracing to existing Geant4 toolkit based simulations. Advantages of separate optical photon simulation and the approaches developed to integrate it with the general Geant4 particle simulation are presented. Approaches to minimize overheads arising from split are shown. Challenges included bringing complex CSG geometries with wavelength dependent material and surface properties to the GPU. Techniques for visualisation of photon propagations with interactive time scrubbing and history selection using OpenGL/OptiX/Thrust interop and geometry shaders are described. Results and demonstrations are shown for the photomultiplier based Daya Bay and JUNO Neutrino detectors.
http://simoncblyth.bitbucket.org/env/presentation/optical_photon_simulation_with_nvidia_optix.html
As a member of the Daya Bay and JUNO collaborations I encountered the bottleneck of Geant4 optical photon processing, taking ~95% of CPU time. Geant4 is a 20 year old project that simulates all particles as they travel through matter, it is used by almost all High Energy Physics detectors.
https://bitbucket.org/simoncblyth/g4dae
To enable the use of GPUs to avoid this bottleneck I developed G4DAE, which exports triangulated detector geometries into COLLADA DAE files, together with material/surface properties as a function of wavelength. I presented G4DAE at the 19th Geant4 Collaboration Meeting in September 2014. The Geant4 Collaboration accepted my proposal to contribute the G4DAE exporter to Geant4 and we plan to include it with the 2015 Geant4 release.
Some aspects of my work to adopt OptiX:
https://bitbucket.org/simoncblyth/env
My developments are currently housed in multiple packages within my bitbucket “env” repository. I plan to create a separate repository named Opticks to house these, to act as a focal point and ease understanding.
I expect my work on Opticks to transition from mainly development to mainly validation against Geant4 over the next months. Fortunately achieving significant performance improvements seems inevitable. The main remaining work is to achieve a match with the Geant4 simulations.