Opticks : GPU Optical Photon Simulation for Particle Physics with NVIDIA OptiX

Meeting the Challenge of JUNO Simulation with Opticks : GPU Optical Photon Acceleration via NVIDIA® OptiX™

https://bitbucket.org/simoncblyth/opticks

Simon C Blyth, IHEP, CAS — DANCE Workshop, October 2019, Houston

Outline

/env/presentation/newtons-opticks.png
 

JUNO_Intro_1

JUNO_Intro_4

JUNO_Intro_2

JUNO_Intro_3

Optical Photon Simulation Problem...








Optical Photon Simulation ≈ Ray Traced Image Rendering

Much in common : geometry, light sources, optical physics

Many Applications of ray tracing :

August 2018 : Major Ray Tracing Advance

SIGGRAPH_2018_Announcing_Worlds_First_Ray_Tracing_GPU








10 Giga Rays/s

Ray-tracing vs Rasterization

/env/presentation/nvidia/nv_rasterization.png /env/presentation/nvidia/nv_raytrace.png

TURING BUILT FOR RTX 2







NVIDIA RTX Metro Exodus










Spatial Index Acceleration Structure












Hardware Traversal of BVH Spatial Index

NVIDIA® OptiX™ Ray Tracing Engine -- http://developer.nvidia.com/optix

OptiX makes GPU ray tracing accessible

NVIDIA expertise:

Opticks provides (Yellow):

[1] Turing RTX GPUs

Geant4OpticksWorkflow

Opticks : translates G4 optical physics to CUDA/OptiX

OptiX : single-ray programming model -> line-by-line translation

CUDA Ports of Geant4 classes
  • G4Cerenkov (only generation loop)
  • G4Scintillation (only generation loop)
  • G4OpAbsorption
  • G4OpRayleigh
  • G4OpBoundaryProcess (only a few surface types)
Modify Cerenkov + Scintillation Processes
  • collect genstep, copy to GPU for generation
  • avoids copying millions of photons to GPU
Scintillator Reemission
  • fraction of bulk absorbed "reborn" within same thread
  • wavelength generated by reemission texture lookup
Opticks (OptiX/Thrust GPU interoperation)
  • OptiX : upload gensteps
  • Thrust : seeding, distribute genstep indices to photons
  • OptiX : launch photon generation and propagation
  • Thrust : pullback photons that hit PMTs
  • Thrust : index photon step sequences (optional)

Opticks : translates G4 geometry to GPU, without approximation

Automated : Geant4 "World" -> Opticks CSG -> CUDA/OptiX
  • intersection functions for ~10 primitives
  • intersection program for arbitrarily complex CSG shapes
Structure
  • repeated geometry instances identified (progeny digests)
  • instance transforms used in OptiX/OpenGL geometry
  • merge CSG trees into global + instance buffers
Material/Surface/Scintillator properties
  • interpolated to standard wavelength domain
  • interleaved into "boundary" texture
  • "reemission" texture for wavelength generation

CUDA/OptiX Intersection functions for ~10 primitives

/env/presentation/tboolean_parade_sep2017.png

Sphere, Cylinder, Disc, Cone, Convex Polyhedron, Hyperboloid, Torus, ...

CUDA/OptiX Intersection Program for Arbitrarily Complex CSG shapes

Complete Binary Tree, pick between pairs of nearest intersects:

UNION tA < tB Enter B Exit B Miss B
Enter A ReturnA LoopA ReturnA
Exit A ReturnA ReturnB ReturnA
Miss A ReturnB ReturnB ReturnMiss
[1] Ray Tracing CSG Objects Using Single Hit Intersections, Andrew Kensler (2006)
with corrections by author of XRT Raytracer http://xrt.wikidot.com/doc:csg
[2] https://bitbucket.org/simoncblyth/opticks/src/tip/optixrap/cu/csg_intersect_boolean.h
Similar to binary expression tree evaluation using postorder traverse.

j1808_top_rtx

j1808_top_ogl

Validation of Opticks Simulation by Comparison with Geant4

Bi-simulations of all JUNO solids, with millions of photons

mis-aligned histories
mostly < 0.25%, < 0.50% for largest solids
deviant photons within matched history
< 0.05% (500/1M)

Primary sources of problems

Primary cause : float vs double

Geant4 uses double everywhere, Opticks only sparingly (observed double costing 10x slowdown with RTX)

Conclude

Performance : Scanning from 1M to 400M photons

Full JUNO Geometry j1808v5

Production Mode : does the minimum

Multi-Event Running : measure interval and launch

interval : avg time between successive launches, including:

NVIDIA Quadro RTX 8000 (48G)

谢谢 NVIDIA China
for loaning the card

scan-pf-1_NHit









scan-pf-1_Interval_over_Launch









scan-pf-1_Opticks_vs_Geant4 2





JUNO Full, 400M photons from center
Geant4 Extrap. 95,600 s (26 hrs)
Opticks RTX ON (i) 58 s

scan-pf-1_Opticks_Speedup 2










JUNO Full, 400M photons from center Speedup
Opticks RTX ON (i) 58s x1660
Opticks RTX OFF (i) 275s x348
Geant4 Extrap. 95,600s (26 hrs)  

scan-pf-1_RTX_Speedup












5x Speedup from RTX with full JUNO geometry

Useful Speedup > 1000x : But Why Not Giga Rays/s ? (1 photon ~10 rays)













OptiX Performance Tools and Tricks, David Hart, NVIDIA https://developer.nvidia.com/siggraph/2019/video/sig915-vid

Where Next for Opticks ?

JUNO+Opticks into Production

Geant4+Opticks Integration : Work with Geant4 Collaboration

Expand Community : Webinars, Conference Tutorials ?

Meeting the Challenge of JUNO Simulation with Opticks

/env/presentation/1px.png

Opticks : state-of-the-art GPU ray tracing applied to optical photon simulation and integrated with Geant4 to eliminate memory and time bottlenecks.

/env/presentation/1px.png
  • Drastic speedup -> better detector understanding -> greater precision
    • any simulation limited by optical photons can benefit
/env/presentation/1px.png
https://bitbucket.org/simoncblyth/opticks code repository
https://simoncblyth.bitbucket.io presentations and videos
https://groups.io/g/opticks forum/mailing list archive
email:opticks+subscribe@groups.io subscribe to mailing list

geocache_360