Opticks : GPU Optical Photon Simulation for Particle Physics with NVIDIA OptiX

Status of Opticks : GPU Optical Photon Simulation for Particle Physics with NVIDIA® OptiX™

Simon C Blyth, IHEP — https://bitbucket.org/simoncblyth/opticks — Jan 2019, SJTU, Shanghai

Contents

/env/presentation/newtons-opticks.png
 

Optical Photon Simulation Problem...

JPMT Before Contact 2

Ray-tracing vs Rasterization

/env/presentation/nvidia/nv_rasterization.png /env/presentation/nvidia/nv_raytrace.png

Rendering + Photon Simulation : Limited by Ray Geometry Intersection

Ray Tracing Tools can Help Optical Photon Simulation

https://nvidianews.nvidia.com/news/nvidia-reveals-the-titan-of-turing-titan-rtx

NVIDIA® OptiX™ Ray Tracing Engine -- http://developer.nvidia.com/optix

OptiX makes GPU ray tracing accessible

NVIDIA expertise:

https://developer.nvidia.com/rtx

User provides (Yellow):

BVH

Opticks : GPU Geometry starts from ray-primitive intersection

/env/presentation/tboolean_parade_sep2017.png

CUDA/OptiX intersection for ~10 primitives -> Exact geometry translation

Ray intersection with general CSG binary trees, on GPU

Pick between pairs of nearest intersects, eg:

UNION tA < tB Enter B Exit B Miss B
Enter A ReturnA LoopA ReturnA
Exit A ReturnA ReturnB ReturnA
Miss A ReturnB ReturnB ReturnMiss
[1] Ray Tracing CSG Objects Using Single Hit Intersections, Andrew Kensler (2006)
with corrections by author of XRT Raytracer http://xrt.wikidot.com/doc:csg
[2] https://bitbucket.org/simoncblyth/opticks/src/tip/optixrap/cu/csg_intersect_boolean.h
Similar to binary expression tree evaluation using postorder traverse.

Torus : much more difficult/expensive than other primitives

3D parametric ray : ray(x,y,z;t) = rayOrigin + t * rayDirection

Solving Quartics

Best Solution : avoid Torus

[1] Depressed quartic + resolvent cubic

https://bitbucket.org/simoncblyth/opticks/src/tip/optixrap/cu/csg_intersect_torus.h

https://bitbucket.org/simoncblyth/opticks/src/tip/optixrap/cu/SolveQuartic.h

Torus : different artifacts as change implementation/params/viewpoint

/env/presentation/torus_cloud_artifact_2017_08_14.png /env/presentation/torus_cuts_artifact_2017_08_08.png /env/presentation/torus_fan_artifact_2017_07_28.png

CSG : (Cylinder - Torus) PMT neck : spurious intersects

/env/presentation/tboolean_12_rasterized.png /env/presentation/tboolean_12_raytrace.png

OptiX Raytrace and OpenGL rasterized wireframe comparing neck models:

  1. Ellipsoid + Hyperboloid + Cylinder
  2. Ellipsoid + (Cylinder - Torus) + Cylinder

Best Solution : use simpler model for optically unimportant PMT neck

CSG : Alternative PMT neck designs

/env/presentation/x018_torus_hyperboloid_plt.png /env/presentation/x018_torus_hyperboloid_plt_zoom.png

Hyperboloid and Cone defined using closest point on ellipse to center of torus circle

https://bitbucket.org/simoncblyth/opticks/src/tip/ana/x018_torus_hyperboloid_plt.py

j1808_top_rtx

j1808_top_ogl

Opticks : Translate Geant4 Geometry to GPU, Without Approximation

Direct Geometry : Geant4 "World" -> Opticks CSG -> GPU
  • much simpler : fully automated geo-management
Material/Surface/Scintillator properties
  • interpolated to standard wavelength domain
  • interleaved into "boundary" texture
  • "reemission" texture for wavelength generation
Structure
  • repeated geometry instances identified (progeny digests)
  • instance transforms used in OptiX/OpenGL geometry
  • merge CSG trees into global + instance buffers
  • export meshes to glTF 2.0 for 3D visualization
Ease of Use
  • easy geometry : just handover "World"
[1] G4 primitives used need corresponding Opticks implementations, contributions for
any unsupported geometry are welcome

Opticks : Translate Geant4 Optical Physics to GPU (OptiX/CUDA)

OptiX : single-ray programming model -> line-by-line translation

CUDA Ports of Geant4 classes
  • G4Cerenkov (only generation loop)
  • G4Scintillation (only generation loop)
  • G4OpAbsorption
  • G4OpRayleigh
  • G4OpBoundaryProcess (only a few surface types)
Modify Cerenkov + Scintillation Processes
  • collect genstep, copy to GPU for generation
  • avoids copying millions of photons to GPU
Scintillator Reemission
  • fraction of bulk absorbed "reborn" within same thread
  • wavelength generated by reemission texture lookup
Opticks (OptiX/Thrust GPU interoperation)
  • OptiX : upload gensteps
  • Thrust : seeding, distribute genstep indices to photons
  • OptiX : launch photon generation and propagation
  • Thrust : pullback photons that hit PMTs
  • Thrust : index photon step sequences (optional)

Random Aligned Validation : Direct comparison of GPU/CPU NumPy arrays

tboolean-box simple geometry test : compare Opticks events


In [11]: pdv = np.where(dv > 0.0001)[0]
In [12]: ab.dumpline(pdv)
      0   1230 : TO BR SC BT BR BT SA
      1   2413 : TO BT BT SC BT BR BR BT SA
      2   9041 : TO BT SC BR BR BR BR BT SA
      3  14510 : TO SC BT BR BR BT SA
      4  14747 : TO BT SC BR BR BR BR BR BR BR
      5  14747 : TO BT SC BR BR BR BR BR BR BR
    ...

In [20]: ab.b.ox[pdv,0]                                 In [21]: ab.a.ox[pdv,0]
Out[20]:                                                Out[21]:
A()sliced                                               A()sliced
A([    [-191.6262, -240.3634,  450.    ,    5.566 ],    A([    [-191.626 , -240.3634,  450.    ,    5.566 ],
       [ 185.7708, -133.8457,  450.    ,    7.3141],           [ 185.7708, -133.8456,  450.    ,    7.3141],
       [-450.    , -104.4142,  311.143 ,    9.0581],           [-450.    , -104.4142,  311.1431,    9.0581],
       [  83.6955,  208.9171, -450.    ,    5.6188],           [  83.6954,  208.9172, -450.    ,    5.6188],
       [  32.8972,  150.    ,   24.9922,    7.6757],           [  32.8973,  150.    ,   24.992 ,    7.6757],
       [  32.8972,  150.    ,   24.9922,    7.6757],           [  32.8973,  150.    ,   24.992 ,    7.6757],
       [ 450.    , -186.7449,  310.6051,    5.0707],           [ 450.    , -186.7451,  310.605 ,    5.0707],
       [ 299.2227,  318.1443, -450.    ,    4.8717],           [ 299.2229,  318.144 , -450.    ,    4.8717],
 ...

http://bitbucket.com/simoncblyth/opticks/src/tip/notes/issues/tboolean_box_perfect_alignment_small_deviations.rst

Opticks : Progress on Validation : JUNO geometry issues

5/40 JUNO solids with CSG translation issues

PMT_20inch_body/pmt_solid

use of "cylinder - torus"

  • causes spurious intersects
  • fix : polycone neck
  • z-cut hyperboloid not supported by Geant4
PMT_20inch_inner1/2_solid

uses depth 4 tree (31 nodes) where 1 primitive sufficient

  • profligate modelling
  • fix : z-cut ellipsoid cathode
sAirTT

box subtract a cylinder with coincident face

  • fix : grow subtracted cylinder to avoid coincidence

Next step : make these geometry changes then proceed to next issue

https://bitbucket.org/simoncblyth/opticks/src/tip/notes/issues/OKX4Test_j1808_x4gen_csg_solids_survey.rst

Opticks : Progress on Ease-of-Use

Geant4 + Opticks History

2014 : 19th Geant4 Collaboration Meeting, Okinawa
proto-Opticks (G4DAE) presented
2017 : 22nd Geant4 Collaboration Meeting, Australia
presented Opticks (CSG on GPU) to plenary session, discussions on how Opticks might be integrated with Geant4, conclude on an advanced example as initial target
2018 : 23rd CHEP Conference, Bulgaria

discussions with Geant4 EM/Optical coordinator reach agreement on high level structure of example

  • primary concern from Geant4 members is that ongoing support for the examples will be provided

Ease-of-use was focus of 2018 developments

  1. Direct Geometry Translation -> automated geometry management
  2. Modern CMake with BCM[1] -> automated configuration
  3. G4Opticks API -> simple embedding of Opticks within G4 apps

[1] Boost CMake 3.5+ modules : configure direct dependencies only https://github.com/BoostCMake/cmake_modules https://github.com/simoncblyth/bcm

Opticks[1] : Aims to Revolutionize JUNO Muon Simulation

State-of-the-art GPU ray tracing[2] applied to optical simulation

Optical photons generated+propagated entirely on GPU

Status : validation iteration ongoing

[1] Open source project http://bitbucket.org/simoncblyth/opticks

[2] NVIDIA OptiX ray tracing engine

[3] using innovative Constructive Solid Geometry implementation on GPU

[4] scattering, boundary, reemission, absorption