Skip to main content
SpringerLink
Log in

The Effect of Data Structuring on the Parallel Efficiency of the HydroBox3D Relativistic Code

  • Conference paper
  • First Online:
Supercomputing (RuSCDays 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14388))

Included in the following conference series:

  • 186 Accesses

Abstract

The hydrodynamic approach to modeling astrophysics problems has several disadvantages in terms of the implementation of a parallel computing code. One of the main drawbacks is the low arithmetic intensity of the methods that implement the computational problem. This peculiarity produces the performance limitation associated with the performance limitations of the DRAM memory of high-performance computing systems. One of the solutions to this problem is data structuring based on the characteristics of processors and memory of a computer system on which supercomputer simulation is to be carried out. In this work, the authors use the specialized Intel SDLT library, which allows you to organize data in a special way that can help the compiler to vectorize a computational code for Intel server processors. The use of this library made it possible to speed up the computational code by fifty times, and for the first time bring the performance of some code functions to the performance limits of server processors on vector FMA instructions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kulikov, I.M., Chernykh, I.G., Snytnikov, A.V., Glinskiy, B.M., Tutukov, A.V.: AstroPhi: a code for complex simulation of dynamics of astrophysical objects using hybrid supercomputers. Comput. Phys. Commun. 186, 71–80 (2015). https://doi.org/10.1016/j.cpc.2014.09.004

    Article  Google Scholar 

  2. Kulikov, I., Vorobyov, E.: Using the PPML approach for constructing a low-dissipation, operator-splitting scheme for numerical simulations of hydrodynamic flows. J. Comput. Phys. 317, 318–346 (2016). https://doi.org/10.1016/j.jcp.2016.04.057

    Article  MathSciNet  Google Scholar 

  3. Kulikov, I., et al.: A new parallel code based on a simple piecewise parabolic method for numerical modeling of colliding flows in relativistic hydrodynamics. Mathematics 10(11), 1865 (2022)

    Article  Google Scholar 

  4. Kulikov, I.M., Chernykh, I.G., Glinskiy, B.M., Protasov, V.A.: An efficient optimization of HLL method for the second generation of Intel Xeon Phi processor. Lobachevskii J. Math. 39, 543–550 (2018). https://doi.org/10.1134/S1995080218040091

    Article  MathSciNet  Google Scholar 

  5. Kulikov, I., Chernykh, I., Tutukov, A.: A new hydrodynamic code with explicit vectorization instructions optimizations that is dedicated to the numerical simulation of astrophysical gas flow. I. numerical method, tests, and model problems. Astrophys. J. Suppl. Ser. 243, 1–15 (2019). https://doi.org/10.3847/1538-4365/ab2237

  6. Kulikov, I.M., Chernykh, I.G., Tutukov, A.V.: A new parallel Intel Xeon Phi hydrodynamics code for massively parallel supercomputers. Lobachevskii J. Math. 39, 1207–1216 (2018). https://doi.org/10.1134/S1995080218090135

    Article  MathSciNet  Google Scholar 

  7. Kulikov, I., et al.: Using adaptive nested mesh code HydroBox3D for numerical simulation of type Ia supernovae: merger of carbon-oxygen white dwarf stars, collapse, and non-central explosion. In: Proceedings of the 2018 Ivannikov ISP RAS Open Conference ISPRAS (2018). https://doi.org/10.1109/ISPRAS.2018.00018

  8. Popov, M., Ustyugov, S.: Piecewise parabolic method on local stencil for gasdynamic simulations. Comput. Math. Math. Phys. 47, 1970–1989 (2007). https://doi.org/10.1134/S0965542507120081

    Article  MathSciNet  Google Scholar 

  9. Popov, M., Ustyugov, S.: Piecewise parabolic method on a local stencil for ideal magnetohydrodynamics. Comput. Math. Math. Phys. 48, 477–499 (2008). https://doi.org/10.1134/S0965542508030111

    Article  MathSciNet  Google Scholar 

  10. Kulikov, I.: A new code for the numerical simulation of relativistic flows on supercomputers by means of a low-dissipation scheme. Comput. Phys. Commun. 257, 107532 (2020). https://doi.org/10.1016/j.cpc.2020.107532

    Article  MathSciNet  Google Scholar 

  11. Tutukov, A.V., Cherepashchuk, A.M.: Evolution of close binary stars: theory and observations. Phys.-Usp. 63, 209 (2020)

    Article  Google Scholar 

  12. Mezcua, M.: Dwarf galaxies might not be the birth sites of supermassive black holes. Nat. Astron. 3, 6–7 (2019). https://doi.org/10.1038/s41550-018-0662-2

    Article  Google Scholar 

  13. Miceli, M., et al.: Collisionless shock heating of heavy ions in SN 1987A. Nat. Astron. 3, 236–241 (2019). https://doi.org/10.1038/s41550-018-0677-8

    Article  Google Scholar 

  14. Mitchell, N., Vorobyov, E., Hensler, G.: Collisionless stellar hydrodynamics as an efficient alternative to N-body methods. Mon. Not. R. Astron. Soc. 428, 2674–2687 (2013). https://doi.org/10.1093/mnras/sts228

    Article  Google Scholar 

  15. Kulikov, I.: GPUPEGAS: a new GPU-accelerated hydrodynamic code for numerical simulations of interacting galaxies. Astrophys. J. Suppl. Ser. 214, 1–12 (2014). https://doi.org/10.1088/0067-0049/214/1/12

    Article  Google Scholar 

  16. Pabst, C., et al.: Disruption of the Orion molecular core 1 by wind from the massive star \(\theta ^1\) Orionis C. Nature 565, 618–621 (2019). https://doi.org/10.1038/s41586-018-0844-1

    Article  Google Scholar 

  17. Forbes, J., Krumholz, M., Goldbaum, N., Dekel, A.: Suppression of star formation in dwarf galaxies by photoelectric grain heating feedback. Nature 535, 523–525 (2016). https://doi.org/10.1038/nature18292

    Article  Google Scholar 

  18. Willcox, D., Townsley, D., Calder, A., Denissenkov, P., Herwig, F.: Type Ia supernova explosions from hybrid carbon-oxygen-neon white dwarf progenitors. Astrophys. J. 832, 13 (2016). https://doi.org/10.3847/0004-637X/832/1/13

    Article  Google Scholar 

  19. Spillane, T., et al.: \(^{12}\)C + \(^{12}\)C fusion reactions near the Gamow energy. Phys. Rev. Lett. 98, 122501 (2007)

    Article  Google Scholar 

  20. Jiang, J.A., et al.: A hybrid type Ia supernova with an early flash triggered by helium-shell detonation. Nature 550, 80–83 (2017). https://doi.org/10.1038/nature23908

    Article  Google Scholar 

  21. Terreran, G., et al.: Hydrogen-rich supernovae beyond the neutrino-driven core-collapse paradigm. Nat. Astron. 1, 713–720 (2017). https://doi.org/10.1038/s41550-017-0228-8

    Article  Google Scholar 

  22. Mendygral, P.J., et al.: WOMBAT: a scalable and high-performance astrophysical magnetohydrodynamics code. Astrophys. J. Suppl. Ser. 228, 23 (2017). https://doi.org/10.3847/1538-4365/aa5b9c

    Article  Google Scholar 

  23. Schneider, E., Robertson, B.: Cholla: a new massively parallel hydrodynamics code for astrophysical simulation. Astrophys. J. Suppl. Ser. 217, 24 (2015). https://doi.org/10.1088/0067-0049/217/2/24

    Article  Google Scholar 

  24. Schneider, E., Robertson, B.: Introducing CGOLS: the cholla galactic outflow simulation suite. Astrophys. J. 860, 135 (2018). https://doi.org/10.3847/1538-4357/aac329

    Article  Google Scholar 

  25. Schneider, E., Robertson, B., Thompson, T.: Production of cool gas in thermally driven outflows. Astrophys. J. 862, 56 (2018). https://doi.org/10.3847/1538-4357/aacce1

    Article  Google Scholar 

  26. Collela, P.: Multidimensional upwind methods for hyperbolic conservation laws. J. Comput. Phys. 87, 171–200 (1990). https://doi.org/10.1016/0021-9991(90)90233-Q

    Article  MathSciNet  Google Scholar 

  27. Gardiner, T., Stone, J.: An unsplit Godunov method for ideal MHD via constrained transport in three dimensions. J. Comput. Phys. 227, 4123–4141 (2008). https://doi.org/10.1016/j.jcp.2004.11.016

    Article  MathSciNet  Google Scholar 

  28. Zhang, U., Schive, H., Chiueh, T.: Magnetohydrodynamics with GAMER. Astrophys. J. Suppl. Ser. 236, 50 (2018). https://doi.org/10.3847/1538-4365/aac49e

    Article  Google Scholar 

  29. Benitez-Llambay, P., Masset, F.: FARGO3D: a new GPU-oriented MHD code. Astrophys. J. Suppl. Ser. 223, 11 (2016). https://doi.org/10.3847/0067-0049/223/1/11

    Article  Google Scholar 

  30. Griffiths, M., Fedun, V., Erdelyi, R.: A fast MHD code for gravitationally stratified media using graphical processing units: SMAUG. J. Astrophys. Astron. 36, 197–223 (2015). https://doi.org/10.1007/s12036-015-9328-y

    Article  Google Scholar 

  31. Pandolfi, M., D’Ambrosio, D.: Numerical instabilities in upwind methods: analysis and cures for the “carbuncle” phenomenon. J. Comput. Phys. 166, 271–301 (2000). https://doi.org/10.1006/jcph.2000.6652

  32. Chauvat, Y., Moschetta, J.-M., Gressier, J.: Shock wave numerical structure and the carbuncle phenomenon. Int. J. Numer. Methods Fluids 47, 903–909 (2005). https://doi.org/10.1002/fld.916

    Article  Google Scholar 

  33. Liou, M.S.: Mass flux schemes and connection to shock instability. J. Comput. Phys. 160, 623–648 (2000). https://doi.org/10.1006/jcph.2000.6478

    Article  Google Scholar 

  34. Xu, K., Li, Z.: Dissipative mechanism in Godunov-type schemes. Int. J. Numer. Methods Fluids 37, 1–22 (2001). https://doi.org/10.1002/fld.160

  35. Kim, S.-S., Kim, C., Rho, O.-H., Hong, S.K.: Cures for the shock instability: development of a shock-stable Roe scheme. J. Comput. Phys. 185, 342–374 (2003). https://doi.org/10.1016/S0021-9991(02)00037-2

    Article  MathSciNet  Google Scholar 

  36. Dumbser, M., Morschetta, J.-M., Gressier, J.: A matrix stability analysis of the carbuncle phenomenon. J. Comput. Phys. 197, 647–670 (2004). https://doi.org/10.1016/j.jcp.2003.12.013

    Article  Google Scholar 

  37. Davis, S.F.: A rotationally biased upwind difference scheme for the Euler equations. J. Comput. Phys. 56, 65–92 (1984). https://doi.org/10.1016/0021-9991(84)90084-6

    Article  Google Scholar 

  38. Levy, D.W., Powell, K.G., Van Leer, B.: Use of a rotated Riemann solver for the two-dimensional Euler equations. J. Comput. Phys. 106, 201–214 (1993). https://doi.org/10.1016/S0021-9991(83)71103-4

    Article  Google Scholar 

  39. Ren, Y.-X.: A robust shock-capturing scheme based on rotated Riemann solvers. Comput. Fluids 32, 1379–1403 (2003). https://doi.org/10.1016/S0045-7930(02)00114-7

    Article  Google Scholar 

  40. Nishikawa, H., Kitamura, K.: Very simple, carbuncle-free, boundary-layer-resolving, rotated-hybrid Riemann solvers. J. Comput. Phys. 227, 2560–2581 (2008). https://doi.org/10.1016/j.jcp.2007.11.003

    Article  MathSciNet  Google Scholar 

  41. Perepelkina, A., Levchenko, V.D.: Functionally arranged data for algorithms with space-time Wavefront. In: Sokolinsky, L., Zymbler, M. (eds.) PCT 2021. CCIS, vol. 1437, pp. 134–148. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-81691-9_10

    Chapter  Google Scholar 

  42. Araudo A., Bosch-Ramon V., Romero G.: Gamma rays from cloud penetration at the base of AGN jets. Astron. Astrophys. 522 (2010)

    Google Scholar 

  43. Begelman, M., Blandford, R., Rees, M.: Theory of extragalactic radio sources. Rev. Mod. Phys. 56, 255–351 (1984)

    Article  Google Scholar 

  44. Laing, R.: The sidedness of jets and depolarization in powerful extragalactic radio sources. Nature 331, 149–151 (1988)

    Article  Google Scholar 

  45. Shakura, N., Sunyaev, R.: Black holes in binary systems. Observational appearance. Astron. Astrophys. 24, 337–355 (1973)

    Google Scholar 

  46. Bisnovatyi-Kogan, G., Blinnikov, S.: A hot corona around a black-hole accretion disk as a model for CYG X-1. Sov. Astron. Lett. 2, 191–193 (1976)

    Google Scholar 

  47. Artemova, Y., Bisnovatyi-Kogan, G., Igumenshchev, I., Novikov, I.: Black hole advective accretion disks with optical depth transition. Astroph. J. 637, 968–977 (2006)

    Article  Google Scholar 

  48. Narayan, R., Yi, I.: Advection-dominated accretion: a self-similar solution. Astroph. J. Let. 428, L13–L16 (1994)

    Article  Google Scholar 

  49. Glushak, A.P.: Microquasar jets in the supernova remnant G11.2-0.3. Astron. Rep. 58, 6–15 (2014)

    Google Scholar 

  50. Barkov, M.V., Bisnovatyi-Kogan, G.S.: Interaction of a cosmological gamma-ray burst with a dense molecular cloud and the formation of jets. Astron. Rep. 49, 24–35 (2005)

    Article  Google Scholar 

  51. Istomin, Ya.N., Komberg, B.V.: Gamma-ray bursts as a result of the interaction of a shock from a supernova and a neutron-star companion. Astron. Reps. 46, 908–917 (2002)

    Google Scholar 

  52. Artyukh, V.S.: Phenomenological model for the evolution of radio galaxies such as Cygnus A. Astron. Rep. 59, 520–524 (2015)

    Article  Google Scholar 

  53. Artyukh, V.S.: Effect of aberration on the estimated parameters of relativistic radio jets. Astron. Rep. 62, 436–439 (2018)

    Article  Google Scholar 

  54. Butuzova, M.S.: Search for differences in the velocities and directions of the kiloparsec-scale jets of quasars with and without X-ray emission. Astron. Rep. 60, 313–321 (2016)

    Article  Google Scholar 

  55. Butuzova, M.S.: The blazar OJ 287 jet from parsec to kiloparsec scales. Astron. Rep. 65, 635–644 (2021)

    Article  Google Scholar 

  56. Sotomayor, P., Romero, G.: Nonthermal radiation from the central region of super-accreting active galactic nuclei. Astron. Astrophys. 664(A178) (2022)

    Google Scholar 

  57. Kulikov, I., Chernykh, I., Tutukov, A.: A new hydrodynamic model for numerical simulation of interacting galaxies on Intel Xeon Phi supercomputers. J. Phys: Conf. Ser. 719, 012006 (2016)

    Google Scholar 

  58. Glinsky, B., et al.: The co-design of astrophysical code for massively parallel supercomputers. In: Carretero, J., et al. (eds.) ICA3PP 2016. LNCS, vol. 10049, pp. 342–353. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49956-7_27

    Chapter  Google Scholar 

  59. Akimova, E.N., Misilov, V.E., Kulikov, I.M., Chernykh, I.G.: OMPEGAS: optimized relativistic code for multicore architecture. Mathematics 10, 2546 (2022). https://doi.org/10.3390/math10142546

    Article  Google Scholar 

  60. relax Intel Corporation. SIMD Data Layout Templates. https://www.intel.com/content/www/us/en/develop/documentation/oneapi-dpcpp-cpp-compiler-dev-guide-and-reference/top/compiler-reference/libraries/introduction-to-the-simd-data-layout-templates.html

  61. Intel Corporation. Intel Advisor User Guide. https://www.intel.com/content/www/us/en/develop/documentation/advisor-user-guide/top.html

  62. NERSC. Introduction to the Roofline model. https://www.nersc.gov/assets/Uploads/Tutorial-ISC2019-Intro-v2.pdf

Download references

Acknowledgements

Computations were performed on the NKS-1P supercomputer at the Siberian Supercomputer Center, Institute of Computational Mathematics and Mathematical Geophysics SB RAS, Novosibirsk, Russia. This work was supported by the Russian Science Foundation (project 23-11-00014) https://rscf.ru/project/23-11-00014/.

Author information

Authors and Affiliations

  1. Institute of Computational Mathematics and Mathematical Geophysics, Siberian Branch of the RAS, Novosibirsk, Russia

    Igor Chernykh & Igor Kulikov

  2. Krasovskii Institute of Mathematics and Mechanics, Ural Branch of the RAS, Ekaterinburg, Russia

    Vladimir Misilov & Elena Akimova

  3. Ural Federal University, Ekaterinburg, Russia

    Vladimir Misilov & Elena Akimova

Authors
  1. Igor Chernykh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Misilov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elena Akimova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Igor Kulikov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Chernykh .

Editor information

Editors and Affiliations

  1. Research Computing Center (RCC), Moscow State University, Moscow, Russia

    Vladimir Voevodin

  2. Research Computing Center (RCC), Moscow State University, Moscow, Russia

    Sergey Sobolev

  3. Russian Academy of Sciences (RAS), Keldysh Institute of Applied Mathematics, Moscow, Russia

    Mikhail Yakobovskiy

  4. Russian Federal Nuclear Center, Sarov, Russia

    Rashit Shagaliev

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chernykh, I., Misilov, V., Akimova, E., Kulikov, I. (2023). The Effect of Data Structuring on the Parallel Efficiency of the HydroBox3D Relativistic Code. In: Voevodin, V., Sobolev, S., Yakobovskiy, M., Shagaliev, R. (eds) Supercomputing. RuSCDays 2023. Lecture Notes in Computer Science, vol 14388. Springer, Cham. https://doi.org/10.1007/978-3-031-49432-1_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49432-1_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49431-4

  • Online ISBN: 978-3-031-49432-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation