Note
Go to the end to download the full example code.
CI test: sink state stays synchronized across MPI ranks through dump/reload#
Creates a tiny SPH setup with a few sinks and gas particles, checks sink sync, evolves, dumps, reloads into a fresh context, and checks again.
SPH setup: generating particles ...
SPH setup: Nstep = 0 ( 0.0e+00 ) Ntotal = 0 ( 0.0e+00 rank min = 0.0e+00 max = 0.0e+00) rate = 0.000000e+00 N.s^-1
SPH setup: Nstep = 70 ( 7.0e+01 ) Ntotal = 70 ( 7.0e+01 rank min = 7.4e+04 max = 7.0e+01) rate = 7.000000e+01 N.s^-1
SPH setup: Nstep = 106 ( 1.1e+02 ) Ntotal = 176 ( 1.8e+02 rank min = 8.5e+04 max = 1.8e+02) rate = 1.760000e+02 N.s^-1
SPH setup: Nstep = 118 ( 1.2e+02 ) Ntotal = 294 ( 2.9e+02 rank min = 1.1e+05 max = 2.9e+02) rate = 2.940000e+02 N.s^-1
SPH setup: Nstep = 109 ( 1.1e+02 ) Ntotal = 403 ( 4.0e+02 rank min = 3.0e+05 max = 4.0e+02) rate = 4.030000e+02 N.s^-1
SPH setup: Nstep = 110 ( 1.1e+02 ) Ntotal = 513 ( 5.1e+02 rank min = 9.3e+04 max = 5.1e+02) rate = 5.130000e+02 N.s^-1
SPH setup: Nstep = 106 ( 1.1e+02 ) Ntotal = 619 ( 6.2e+02 rank min = 3.8e+05 max = 6.2e+02) rate = 6.190000e+02 N.s^-1
SPH setup: Nstep = 118 ( 1.2e+02 ) Ntotal = 737 ( 7.4e+02 rank min = 4.5e+05 max = 7.4e+02) rate = 7.370000e+02 N.s^-1
SPH setup: Nstep = 110 ( 1.1e+02 ) Ntotal = 847 ( 8.5e+02 rank min = 9.5e+04 max = 8.5e+02) rate = 8.470000e+02 N.s^-1
SPH setup: Nstep = 109 ( 1.1e+02 ) Ntotal = 956 ( 9.6e+02 rank min = 4.6e+05 max = 9.6e+02) rate = 9.560000e+02 N.s^-1
SPH setup: Nstep = 115 ( 1.2e+02 ) Ntotal = 1071 ( 1.1e+03 rank min = 3.8e+05 max = 1.1e+03) rate = 1.071000e+03 N.s^-1
SPH setup: Nstep = 121 ( 1.2e+02 ) Ntotal = 1192 ( 1.2e+03 rank min = 4.8e+05 max = 1.2e+03) rate = 1.192000e+03 N.s^-1
SPH setup: Nstep = 118 ( 1.2e+02 ) Ntotal = 1310 ( 1.3e+03 rank min = 1.1e+05 max = 1.3e+03) rate = 1.310000e+03 N.s^-1
SPH setup: Nstep = 119 ( 1.2e+02 ) Ntotal = 1429 ( 1.4e+03 rank min = 2.4e+05 max = 1.4e+03) rate = 1.429000e+03 N.s^-1
SPH setup: Nstep = 121 ( 1.2e+02 ) Ntotal = 1550 ( 1.6e+03 rank min = 5.9e+05 max = 1.6e+03) rate = 1.550000e+03 N.s^-1
SPH setup: Nstep = 121 ( 1.2e+02 ) Ntotal = 1671 ( 1.7e+03 rank min = 6.8e+05 max = 1.7e+03) rate = 1.671000e+03 N.s^-1
SPH setup: Nstep = 118 ( 1.2e+02 ) Ntotal = 1789 ( 1.8e+03 rank min = 5.5e+05 max = 1.8e+03) rate = 1.789000e+03 N.s^-1
SPH setup: Nstep = 119 ( 1.2e+02 ) Ntotal = 1908 ( 1.9e+03 rank min = 5.4e+05 max = 1.9e+03) rate = 1.908000e+03 N.s^-1
SPH setup: Nstep = 122 ( 1.2e+02 ) Ntotal = 2030 ( 2.0e+03 rank min = 7.6e+04 max = 2.0e+03) rate = 2.030000e+03 N.s^-1
SPH setup: Nstep = 120 ( 1.2e+02 ) Ntotal = 2150 ( 2.2e+03 rank min = 5.6e+05 max = 2.2e+03) rate = 2.150000e+03 N.s^-1
SPH setup: Nstep = 118 ( 1.2e+02 ) Ntotal = 2268 ( 2.3e+03 rank min = 4.9e+05 max = 2.3e+03) rate = 2.268000e+03 N.s^-1
SPH setup: Nstep = 63 ( 6.3e+01 ) Ntotal = 2331 ( 2.3e+03 rank min = 2.1e+05 max = 2.3e+03) rate = 2.331000e+03 N.s^-1
SPH setup: the generation step took : 0.019651065000000002 s
SPH setup: final particle count = 2331 beginning injection ...
Info: --------------------------------------------- [DataInserterUtility][rank=0]
Info: Compute load ... [DataInserterUtility][rank=0]
Info: run scheduler step ... [DataInserterUtility][rank=0]
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.50 us (57.2%)
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2000.0 min = 2000.0 factor = 1
- strategy "round robin" : max = 1900.0 min = 1900.0 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2000
max = 2000
avg = 2000
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 1.76 us (0.0%)
patch tree reduce : 1.52 us (0.0%)
gen split merge : 952.00 ns (0.0%)
split / merge op : 1/0
apply split merge : 7.93 ms (92.5%)
LB compute : 603.10 us (7.0%)
LB move op cnt : 0
LB apply : 4.18 us (0.0%)
Info: Compute load ... [DataInserterUtility][rank=0]
Info: run scheduler step ... [DataInserterUtility][rank=0]
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.90 us (54.4%)
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2000.0 min = 2000.0 factor = 1
- strategy "round robin" : max = 1900.0 min = 1900.0 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2000
max = 2000
avg = 2000
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 1.96 us (0.5%)
patch tree reduce : 2.83 us (0.7%)
gen split merge : 761.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 381.00 ns (0.1%)
LB compute : 377.85 us (92.8%)
LB move op cnt : 0
LB apply : 1.80 us (0.4%)
Info: patch count stable after 2 runs npatch = 8 [DataInserterUtility][rank=0]
Info: --------------------------------------------- [DataInserterUtility][rank=0]
SPH setup: injected 2000 / 2331 => 85.8% | ranks with patchs = 1 / 1 -> local loop <-
Info: --------------------------------------------- [DataInserterUtility][rank=0]
Info: Compute load ... [DataInserterUtility][rank=0]
Info: run scheduler step ... [DataInserterUtility][rank=0]
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 7.29 us (39.1%)
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2331.0 min = 2331.0 factor = 1
- strategy "round robin" : max = 2214.4 min = 2214.4 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2331
max = 2331
avg = 2331
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 1.66 us (0.4%)
patch tree reduce : 3.48 us (0.9%)
gen split merge : 841.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.05 us (0.3%)
LB compute : 377.94 us (95.5%)
LB move op cnt : 0
LB apply : 3.92 us (1.0%)
Info: patch count stable after 1 runs npatch = 8 [DataInserterUtility][rank=0]
Info: --------------------------------------------- [DataInserterUtility][rank=0]
SPH setup: injected 2331 / 2331 => 100.0% | ranks with patchs = 1 / 1 <- global loop -> (msg count : 0)
SPH setup: the injection step took : 0.045794613000000005 s
Info: injection perf report: [SPH setup][rank=0]
+======+====================+=======+=============+=============+=============+
| rank | rank get (sum/max) | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+====================+=======+=============+=============+=============+
| 0 | 0.00s / 0.00s | 0.00s | 4.2% 0.0% | 1.24 GB | 5.29 MB |
+------+--------------------+-------+-------------+-------------+-------------+
SPH setup: the setup took : 0.072166034 s
Sinks are in sync !
---------------- t = 0, dt = 0 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2331.0 min = 2331.0 factor = 1
- strategy "round robin" : max = 2214.4 min = 2214.4 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2331
max = 2331
avg = 2331
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.65 us (0.9%)
patch tree reduce : 1.74 us (0.5%)
gen split merge : 651.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 371.00 ns (0.1%)
LB compute : 360.90 us (93.7%)
LB move op cnt : 0
LB apply : 2.30 us (0.6%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.75 us (47.6%)
Info: sink accretion : [sph::Sink][rank=0]
id 0 deltas : mass=0.018999999999999906 r=(1.3877787807814457e-17,-1.5959796454644326e-19,0) v=(0.18645731108930325,-0.0009322865554465126,0) l=(0,0,8.50222023142003e-19)
id 1 deltas : mass=0.02200000000000002 r=(0,0.00020958060681705581,0) v=(0.421455938697318,0.0012643678160919568,0) l=(0,0,-0.0010479030340852771)
id 2 deltas : mass=0.017000000000000015 r=(-0.0005617977528089886,-0.0009368869122459988,-0.0001254809703081436) v=(0.6354307116104868,0,0) l=(0,-0.0003130750209188165,0.0023375328460538015)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 1.730061349693251
patch 1 high interf/patch volume: 1.5783972125435533
patch 2 high interf/patch volume: 1.5133333333333332
patch 3 high interf/patch volume: 1.3568773234200742
patch 4 high interf/patch volume: 1.5966101694915251
patch 5 high interf/patch volume: 1.4597701149425286
patch 6 high interf/patch volume: 1.4240282685512367
patch 7 high interf/patch volume: 1.2936507936507935
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.05500000000000001 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 1.9938650306748458
patch 1 high interf/patch volume: 1.8780487804878045
patch 2 high interf/patch volume: 1.893333333333333
patch 3 high interf/patch volume: 1.7806691449814123
patch 4 high interf/patch volume: 1.8474576271186438
patch 5 high interf/patch volume: 1.7432950191570882
patch 6 high interf/patch volume: 1.7703180212014136
patch 7 high interf/patch volume: 1.6825396825396823
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.06050000000000001 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 1.9938650306748458
patch 1 high interf/patch volume: 1.8780487804878045
patch 2 high interf/patch volume: 1.893333333333333
patch 3 high interf/patch volume: 1.7806691449814123
patch 4 high interf/patch volume: 1.8474576271186438
patch 5 high interf/patch volume: 1.7432950191570882
patch 6 high interf/patch volume: 1.7703180212014136
patch 7 high interf/patch volume: 1.6825396825396823
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.06655000000000001 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 2.128834355828221
patch 1 high interf/patch volume: 2.0313588850174216
patch 2 high interf/patch volume: 2.0733333333333333
patch 3 high interf/patch volume: 1.981412639405204
patch 4 high interf/patch volume: 1.9966101694915255
patch 5 high interf/patch volume: 1.9118773946360152
patch 6 high interf/patch volume: 1.9187279151943462
patch 7 high interf/patch volume: 1.8492063492063493
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.07320500000000002 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 2.4601226993865026
patch 1 high interf/patch volume: 2.35191637630662
patch 2 high interf/patch volume: 2.4400000000000004
patch 3 high interf/patch volume: 2.3457249070631976
patch 4 high interf/patch volume: 2.3491525423728814
patch 5 high interf/patch volume: 2.2643678160919545
patch 6 high interf/patch volume: 2.314487632508834
patch 7 high interf/patch volume: 2.246031746031746
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.08052550000000003 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 3.131901840490797
patch 1 high interf/patch volume: 2.9756097560975623
patch 2 high interf/patch volume: 3.11
patch 3 high interf/patch volume: 2.981412639405205
patch 4 high interf/patch volume: 3.0169491525423733
patch 5 high interf/patch volume: 2.8888888888888884
patch 6 high interf/patch volume: 2.9752650176678443
patch 7 high interf/patch volume: 2.8531746031746024
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.08857805000000003 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 3.4509202453987724
patch 1 high interf/patch volume: 3.337979094076655
patch 2 high interf/patch volume: 3.4599999999999995
patch 3 high interf/patch volume: 3.3717472118959115
patch 4 high interf/patch volume: 3.328813559322034
patch 5 high interf/patch volume: 3.2413793103448274
patch 6 high interf/patch volume: 3.31095406360424
patch 7 high interf/patch volume: 3.2301587301587302
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.09743585500000004 unconverged cnt = 2273
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 4.233128834355829
patch 1 high interf/patch volume: 4.142857142857143
patch 2 high interf/patch volume: 4.113333333333332
patch 3 high interf/patch volume: 4.040892193308551
patch 4 high interf/patch volume: 4.13220338983051
patch 5 high interf/patch volume: 4.057471264367816
patch 6 high interf/patch volume: 4.053003533568904
patch 7 high interf/patch volume: 3.9841269841269833
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.10717944050000006 unconverged cnt = 1086
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 4.469325153374233
patch 1 high interf/patch volume: 4.383275261324042
patch 2 high interf/patch volume: 4.346666666666668
patch 3 high interf/patch volume: 4.301115241635688
patch 4 high interf/patch volume: 4.345762711864408
patch 5 high interf/patch volume: 4.283524904214559
patch 6 high interf/patch volume: 4.300353356890459
patch 7 high interf/patch volume: 4.261904761904761
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.11789738455000007 unconverged cnt = 55
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 5.588957055214724
patch 1 high interf/patch volume: 5.508710801393732
patch 2 high interf/patch volume: 5.673333333333333
patch 3 high interf/patch volume: 5.639405204460967
patch 4 high interf/patch volume: 5.48135593220339
patch 5 high interf/patch volume: 5.421455938697317
patch 6 high interf/patch volume: 5.664310954063605
patch 7 high interf/patch volume: 5.642857142857144
Warning: smoothing length is not converged, rerunning the iterator ... [Smoothinglength][rank=0]
largest h = 0.12968712300500007 unconverged cnt = 2
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.226993865030673
patch 1 high interf/patch volume: 6.177700348432057
patch 2 high interf/patch volume: 6.183333333333333
patch 3 high interf/patch volume: 6.130111524163569
patch 4 high interf/patch volume: 6.159322033898306
patch 5 high interf/patch volume: 6.095785440613028
patch 6 high interf/patch volume: 6.183745583038868
patch 7 high interf/patch volume: 6.11904761904762
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.315,0.035,0)
sum a = (-2.842170943040401e-14,5.684341886080802e-14,0)
sum e = 113.65
sum de = -98.96088366316451
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 1.02e-04 |
| force | 8.26e-06 |
| sink_sink | 6.55e-06 |
+-----------+----------+
Info: cfl dt = 6.551773540016008e-06 cfl multiplier : 0.01 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.3522e+03 | 2273 | 8 | 1.681e+00 | 0.0% | 2.2% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 0 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0, dt = 6.551773540016008e-06 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2273.0 min = 2273.0 factor = 1
- strategy "round robin" : max = 2159.3 min = 2159.3 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2273
max = 2273
avg = 2273
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 6.73 us (1.5%)
patch tree reduce : 3.95 us (0.9%)
gen split merge : 812.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.38 us (0.3%)
LB compute : 424.46 us (93.9%)
LB move op cnt : 0
LB apply : 3.70 us (0.8%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.08 us (49.3%)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.2879999999999985
patch 1 high interf/patch volume: 6.2057761732852
patch 2 high interf/patch volume: 6.247148288973386
patch 3 high interf/patch volume: 6.180887372013654
patch 4 high interf/patch volume: 6.295774647887325
patch 5 high interf/patch volume: 6.1875
patch 6 high interf/patch volume: 6.247311827956988
patch 7 high interf/patch volume: 6.166123778501629
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.314967581535043,0.03499492993291522,1.4489278344931484e-05)
sum a = (1.7053025658242404e-13,1.1368683772161603e-13,-1.4210854715202004e-14)
sum e = 113.64935776925222
sum de = -97.13802261718962
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 3.44e-03 |
| force | 2.81e-04 |
| sink_sink | 2.23e-04 |
+-----------+----------+
Info: cfl dt = 0.00022275667803016137 cfl multiplier : 0.34 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.0316e+04 | 2273 | 8 | 2.203e-01 | 0.0% | 2.2% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 0.10705009283971705 (tsim/hr) [sph::Model][rank=0]
---------------- t = 6.551773540016008e-06, dt = 0.00022275667803016137 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2273.0 min = 2273.0 factor = 1
- strategy "round robin" : max = 2159.3 min = 2159.3 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2273
max = 2273
avg = 2273
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 6.12 us (1.9%)
patch tree reduce : 4.17 us (1.3%)
gen split merge : 772.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.08 us (0.3%)
LB compute : 294.03 us (91.6%)
LB move op cnt : 0
LB apply : 3.51 us (1.1%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.67 us (45.6%)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.2879999999999985
patch 1 high interf/patch volume: 6.2057761732852
patch 2 high interf/patch volume: 6.247148288973386
patch 3 high interf/patch volume: 6.180887372013654
patch 4 high interf/patch volume: 6.394366197183099
patch 5 high interf/patch volume: 6.2875000000000005
patch 6 high interf/patch volume: 6.322580645161289
patch 7 high interf/patch volume: 6.2442996742671015
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.313897269462164,0.03482750060074424,0.0004927289342157148)
sum a = (-2.842170943040401e-14,0,-1.1368683772161603e-13)
sum e = 113.63481505754321
sum de = -44.77941247604406
Warning: the corrector tolerance are broken the step will be re rerunned [BasicGasSPH][rank=0]
eps_v = 0.010106726402497119
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.3137755549084,0.03481355659818742,0.0004961998858912069)
sum a = (-8.526512829121202e-14,-1.1368683772161603e-13,7.105427357601002e-14)
sum e = 113.63355636261585
sum de = -44.82531737945311
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 1.56e-03 |
| force | 2.31e-04 |
| sink_sink | 1.83e-04 |
+-----------+----------+
Info: cfl dt = 0.00018328014262681863 cfl multiplier : 0.28 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 8.1758e+03 | 2273 | 8 | 2.780e-01 | 0.0% | 2.0% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 2.88446405607672 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.0002293084515701774, dt = 0.00018328014262681863 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2273.0 min = 2273.0 factor = 1
- strategy "round robin" : max = 2159.3 min = 2159.3 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2273
max = 2273
avg = 2273
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.71 us (1.7%)
patch tree reduce : 4.03 us (1.2%)
gen split merge : 812.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.14 us (0.3%)
LB compute : 320.08 us (92.5%)
LB move op cnt : 0
LB apply : 3.45 us (1.0%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.93 us (49.2%)
Info: sink accretion : [sph::Sink][rank=0]
id 1 deltas : mass=0.0010000000000000009 r=(-0.0002825733702763178,-2.5992748440797908e-05,-3.7616457445204277e-10) v=(0.01841672772468461,0.0001413780549717747,-3.266139183730546e-06) l=(1.4588956579124865e-08,-1.6035141770323956e-07,0.00012101999018001273)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.2879999999999985
patch 1 high interf/patch volume: 6.2057761732852
patch 2 high interf/patch volume: 6.247148288973386
patch 3 high interf/patch volume: 6.178082191780823
patch 4 high interf/patch volume: 6.394366197183099
patch 5 high interf/patch volume: 6.2875000000000005
patch 6 high interf/patch volume: 6.322580645161289
patch 7 high interf/patch volume: 6.2442996742671015
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.31396950562196,0.0348528560707208,0.0004076315123698396)
sum a = (1.1368683772161603e-13,5.684341886080802e-14,2.842170943040401e-14)
sum e = 113.57799047577421
sum de = -5.303555756092464
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 2.06e-03 |
| force | 4.30e-04 |
| sink_sink | 3.40e-04 |
+-----------+----------+
Info: cfl dt = 0.00034020216026755444 cfl multiplier : 0.52 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.0442e+04 | 2272 | 8 | 2.176e-01 | 0.0% | 2.3% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 3.0325168172974064 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.00041258859419699605, dt = 0.00034020216026755444 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2272.0 min = 2272.0 factor = 1
- strategy "round robin" : max = 2158.4 min = 2158.4 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2272
max = 2272
avg = 2272
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.43 us (1.6%)
patch tree reduce : 3.73 us (1.1%)
gen split merge : 772.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.35 us (0.4%)
LB compute : 318.92 us (91.9%)
LB move op cnt : 0
LB apply : 3.71 us (1.1%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.31 us (45.7%)
Info: sink accretion : [sph::Sink][rank=0]
id 2 deltas : mass=0.0020000000000000018 r=(-0.0008964255531763344,-5.863973251565846e-05,0.000539681636615455) v=(0.06971160516554098,-0.0019102084936308272,-0.0003163294863295363) l=(3.085726827203e-05,0.0013353971433826657,0.00018828011072075535)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.2879999999999985
patch 1 high interf/patch volume: 6.200000000000001
patch 2 high interf/patch volume: 6.247148288973386
patch 3 high interf/patch volume: 6.178082191780823
patch 4 high interf/patch volume: 6.394366197183099
patch 5 high interf/patch volume: 6.2875000000000005
patch 6 high interf/patch volume: 6.322580645161289
patch 7 high interf/patch volume: 6.2442996742671015
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.31283049048958,0.03456458697970308,0.0009308308956632247)
sum a = (2.842170943040401e-14,-5.684341886080802e-14,-9.947598300641403e-14)
sum e = 113.48480531280518
sum de = 81.64648237715255
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 1.74e-03 |
| force | 5.61e-04 |
| sink_sink | 4.46e-04 |
+-----------+----------+
Info: cfl dt = 0.0004464158410420693 cfl multiplier : 0.68 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.0520e+04 | 2270 | 8 | 2.158e-01 | 0.0% | 2.1% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 5.67558099453923 (tsim/hr) [sph::Model][rank=0]
Sinks are in sync !
Info: Dumping state to sink_sync_test.sham [SPH][rank=0]
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 6.04 us (42.5%)
Info: dump to sink_sync_test.sham [Shamrock Dump][rank=0]
- took 757.27 us, bandwidth = 391.30 MB/s
Info: Loading state from dump sink_sync_test.sham [SPH][rank=0]
Info: load dump from sink_sync_test.sham [Shamrock Dump][rank=0]
- took 3.18 ms, bandwidth = 93.08 MB/s
Sinks are in sync !
---------------- t = 0.0007527907544645504, dt = 0.0004464158410420693 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2270.0 min = 2270.0 factor = 1
- strategy "round robin" : max = 2156.5 min = 2156.5 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2270
max = 2270
avg = 2270
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.59 us (1.0%)
patch tree reduce : 1.88 us (0.5%)
gen split merge : 691.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.03 us (0.3%)
LB compute : 356.44 us (94.8%)
LB move op cnt : 0
LB apply : 3.90 us (1.0%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.76 us (50.2%)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.2799999999999985
patch 1 high interf/patch volume: 6.18909090909091
patch 2 high interf/patch volume: 6.247148288973386
patch 3 high interf/patch volume: 6.178082191780823
patch 4 high interf/patch volume: 6.382456140350876
patch 5 high interf/patch volume: 6.275
patch 6 high interf/patch volume: 6.323741007194244
patch 7 high interf/patch volume: 6.2442996742671015
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.312081509820906,0.034398086470381894,0.001239139736493644)
sum a = (-5.684341886080802e-14,-1.1368683772161603e-13,-5.684341886080802e-14)
sum e = 113.5488940143931
sum de = 203.69591646895293
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 1.36e-03 |
| force | 6.48e-04 |
| sink_sink | 5.13e-04 |
+-----------+----------+
Info: cfl dt = 0.000512654960520611 cfl multiplier : 0.7866666666666667 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.0245e+04 | 2270 | 8 | 2.216e-01 | 0.0% | 2.0% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 7.252938349212515 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.0011992065955066197, dt = 0.000512654960520611 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2270.0 min = 2270.0 factor = 1
- strategy "round robin" : max = 2156.5 min = 2156.5 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2270
max = 2270
avg = 2270
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.89 us (1.5%)
patch tree reduce : 2.86 us (0.7%)
gen split merge : 781.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.29 us (0.3%)
LB compute : 357.53 us (93.0%)
LB move op cnt : 0
LB apply : 3.79 us (1.0%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.32 us (50.3%)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.191999999999998
patch 1 high interf/patch volume: 6.105454545454546
patch 2 high interf/patch volume: 6.235741444866922
patch 3 high interf/patch volume: 6.157534246575343
patch 4 high interf/patch volume: 6.407017543859648
patch 5 high interf/patch volume: 6.2906249999999995
patch 6 high interf/patch volume: 6.384892086330936
patch 7 high interf/patch volume: 6.302931596091206
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.311534138478002,0.03426965166609852,0.0014503792275798177)
sum a = (1.1368683772161603e-13,1.1368683772161603e-13,1.1368683772161603e-13)
sum e = 113.68959047010817
sum de = 345.10529771530764
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 1.07e-03 |
| force | 7.04e-04 |
| sink_sink | 5.52e-04 |
+-----------+----------+
Info: cfl dt = 0.0005523343280162558 cfl multiplier : 0.8577777777777778 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 9.9890e+03 | 2270 | 8 | 2.272e-01 | 0.0% | 2.1% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 8.121300044211841 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.0017118615560272306, dt = 0.0005523343280162558 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2270.0 min = 2270.0 factor = 1
- strategy "round robin" : max = 2156.5 min = 2156.5 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2270
max = 2270
avg = 2270
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.61 us (1.5%)
patch tree reduce : 3.09 us (0.8%)
gen split merge : 862.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.20 us (0.3%)
LB compute : 323.66 us (87.8%)
LB move op cnt : 0
LB apply : 22.74 us (6.2%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.08 us (51.0%)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.191999999999998
patch 1 high interf/patch volume: 6.050909090909092
patch 2 high interf/patch volume: 6.2015209125475295
patch 3 high interf/patch volume: 6.0479452054794525
patch 4 high interf/patch volume: 6.5069930069930075
patch 5 high interf/patch volume: 6.340624999999999
patch 6 high interf/patch volume: 6.458483754512637
patch 7 high interf/patch volume: 6.32899022801303
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.31112113508086,0.034152626385041795,0.0015986545577838795)
sum a = (2.842170943040401e-14,0,2.842170943040401e-14)
sum e = 113.92267489801058
sum de = 495.88335143179296
Warning: the corrector tolerance are broken the step will be re rerunned [BasicGasSPH][rank=0]
eps_v = 0.011840076446588807
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.310853913874283,0.034125058930421615,0.0015976128675086188)
sum a = (2.842170943040401e-14,0,1.4210854715202004e-14)
sum e = 113.92097020468077
sum de = 496.19136441410507
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 4.26e-04 |
| force | 3.70e-04 |
| sink_sink | 2.86e-04 |
+-----------+----------+
Info: cfl dt = 0.0002864278912784332 cfl multiplier : 0.4525925925925926 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 8.3059e+03 | 2270 | 8 | 2.733e-01 | 0.0% | 2.2% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 7.275561900518538 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.0022641958840434865, dt = 0.0002864278912784332 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2270.0 min = 2270.0 factor = 1
- strategy "round robin" : max = 2156.5 min = 2156.5 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2270
max = 2270
avg = 2270
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.53 us (1.4%)
patch tree reduce : 3.37 us (0.8%)
gen split merge : 742.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.08 us (0.3%)
LB compute : 374.48 us (93.6%)
LB move op cnt : 0
LB apply : 3.60 us (0.9%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 2.56 us (48.6%)
Info: sink accretion : [sph::Sink][rank=0]
id 2 deltas : mass=0.0010000000000000009 r=(-0.000469680943618494,0.00023359018294974443,-0.000182285749249729) v=(0.036381111686466516,-0.004306088407587616,0.0038620997378118194) l=(1.361207817789772e-05,-0.0003530130357788252,-0.00048744190755523227)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 5.992
patch 1 high interf/patch volume: 5.923636363636364
patch 2 high interf/patch volume: 5.935361216730037
patch 3 high interf/patch volume: 5.900343642611686
patch 4 high interf/patch volume: 6.157342657342658
patch 5 high interf/patch volume: 6.06875
patch 6 high interf/patch volume: 6.12274368231047
patch 7 high interf/patch volume: 6.061889250814333
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.312784944049035,0.034443709106043774,0.000827675429116298)
sum a = (5.684341886080802e-14,5.684341886080802e-14,4.263256414560601e-14)
sum e = 114.00394000559042
sum de = 508.41597651169457
Warning: the corrector tolerance are broken the step will be re rerunned [BasicGasSPH][rank=0]
eps_v = 0.02576228461305771
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.312082770286487,0.03466477252947364,0.0008413994567690319)
sum a = (0,5.684341886080802e-14,-1.4210854715202004e-14)
sum e = 113.99499855347668
sum de = 508.7006719363991
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 2.72e-04 |
| force | 2.59e-04 |
| sink_sink | 1.99e-04 |
+-----------+----------+
Info: cfl dt = 0.00019897300678063906 cfl multiplier : 0.3175308641975309 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 8.8292e+03 | 2269 | 8 | 2.570e-01 | 0.0% | 2.5% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 4.012402919549655 (tsim/hr) [sph::Model][rank=0]
---------------- t = 0.0025506237753219196, dt = 0.00019897300678063906 ----------------
Info: Summary (strategy = round robin): [LoadBalance][rank=0]
- strategy "psweep" : max = 2269.0 min = 2269.0 factor = 1
- strategy "round robin" : max = 2155.5 min = 2155.5 factor = 0.95
Info: Loadbalance stats : [LoadBalance][rank=0]
npatch = 8
min = 2269
max = 2269
avg = 2269
efficiency = 100.00%
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 5.74 us (1.6%)
patch tree reduce : 3.32 us (0.9%)
gen split merge : 851.00 ns (0.2%)
split / merge op : 0/0
apply split merge : 1.08 us (0.3%)
LB compute : 327.88 us (92.5%)
LB move op cnt : 0
LB apply : 3.56 us (1.0%)
Info: Scheduler step timings : [Scheduler][rank=0]
metadata sync : 3.25 us (49.1%)
Info: sink accretion : [sph::Sink][rank=0]
id 0 deltas : mass=0.0029999999999998916 r=(-0.000356549430889877,0.00013789815003367392,-3.8900264204212893e-07) v=(0.036804037266496725,-0.0026049367562814285,-0.0002977115124496166) l=(1.5355132242207976e-05,-3.4042791169362646e-05,-0.0013917210465080327)
Warning: High interface/patch volume ratio. [InterfaceGen][rank=0]
This can lead to high mpi overhead, try to increase the patch split crit
patch 0 high interf/patch volume: 6.016
patch 1 high interf/patch volume: 5.941818181818182
patch 2 high interf/patch volume: 5.984732824427481
patch 3 high interf/patch volume: 5.951557093425607
patch 4 high interf/patch volume: 6.234265734265736
patch 5 high interf/patch volume: 6.153124999999999
patch 6 high interf/patch volume: 6.198555956678701
patch 7 high interf/patch volume: 6.143322475570033
Info: conservation infos : [sph::Model][rank=0]
sum v = (23.313127792253574,0.034803506273773155,0.0005710137311530639)
sum a = (5.684341886080802e-14,5.684341886080802e-14,-5.684341886080802e-14)
sum e = 113.88531834450656
sum de = 578.4716204698631
Info: CFL detail : [sph::Model][rank=0]
+===========+==========+
| key | value |
+===========+==========+
| courant | 4.77e-04 |
| force | 4.43e-04 |
| sink_sink | 3.37e-04 |
+-----------+----------+
Info: cfl dt = 0.0003374001421506465 cfl multiplier : 0.5450205761316873 [sph::Model][rank=0]
Info: Timestep perf report: [sph::Model][rank=0]
+======+============+======+========+===========+======+=============+=============+=============+
| rank | rate (N/s) | Nobj | Npatch | tstep | MPI | alloc d% h% | mem (max) d | mem (max) h |
+======+============+======+========+===========+======+=============+=============+=============+
| 0 | 1.0378e+04 | 2266 | 8 | 2.183e-01 | 0.0% | 2.2% 0.0% | 1.24 GB | 5.29 MB |
+------+------------+------+--------+-----------+------+-------------+-------------+-------------+
Info: estimated rate : 3.2806423145786057 (tsim/hr) [sph::Model][rank=0]
Sinks are in sync !
run_test_sink_synchro: OK
Current sinks:
[{'pos': (0.09877660057755726, -0.0003699820152028824, 0.000292858494405309), 'velocity': (-0.7909594697171893, -0.42835509420057705, 0.21579163220191747), 'sph_acceleration': (-2.788118606064703, 6.194080856496939, -3.386302543973059), 'ext_acceleration': (-371.6829482929875, -190.87247862152324, 85.76563352228862), 'mass': 1.0219999999999998, 'angular_momentum': (1.5355132242207976e-05, -3.4042791169362646e-05, -0.0013917210465080319), 'accretion_radius': 0.15}, {'pos': (-0.19723897477916802, 0.0992597386368148, 5.457513126197257e-05), 'velocity': (1.7840284522444005, -0.646223010107699, 0.03996740374797707), 'sph_acceleration': (40.13267477144062, -16.361891265380407, -1.2376352720798174), 'ext_acceleration': (456.6529453658143, -211.77602552231514, 16.18781052807078), 'mass': 0.523, 'angular_momentum': (1.4588956579124865e-08, -1.6035141770323956e-07, -0.0009268830439052644), 'accretion_radius': 0.15}, {'pos': (0.001919965581750201, -0.1466495904953547, 0.048938432104203365), 'velocity': (2.1365106397204268, 3.022034230508638, -0.9498166179408467), 'sph_acceleration': (4.433172833108257, 16.092200428397565, -6.362794011098259), 'ext_acceleration': (522.3351212189342, 1132.705683330991, -355.99519394799995), 'mass': 0.27, 'angular_momentum': (4.446934644992772e-05, 0.0006693090866850239, 0.0020383710492193246), 'accretion_radius': 0.15}]
check_ref_dataset: OK
Info: collected : 8 patches [PatchScheduler][rank=0]
Current sums: [np.float64(65.79475628688994), np.float64(-15.508903903707498), np.float64(-1.0412208430674998), np.float64(22613.455824802637), np.float64(-5.195701678016274), np.float64(15.008486573476972), np.float64(-19336.88835500463), np.float64(-2117.975619213131), np.float64(5826.038830234722), np.float64(245.85176557074223)]
9 import numpy as np
10
11 import shamrock
12
13 DUMP_NAME = "sink_sync_test.sham"
14
15
16 def check_sinks_are_in_sync(ctx, model):
17 s = str(model.get_sinks())
18 # Collective: every rank must call this
19 hist = shamrock.algs.all_string_histogram([s], delimiter="\n", hash_based=False)
20 if len(hist) != 1:
21 raise RuntimeError(f"sinks not in sync across ranks: {hist}")
22 key, count = next(iter(hist.items()))
23 if count != shamrock.sys.world_size():
24 raise RuntimeError(
25 f"expected count={shamrock.sys.world_size()}, got {count} for key={key!r}"
26 )
27 shamrock.sys.mpi_barrier()
28 if shamrock.sys.world_rank() == 0:
29 print("Sinks are in sync !")
30
31
32 si = shamrock.UnitSystem()
33 sicte = shamrock.Constants(si)
34 codeu = shamrock.UnitSystem(
35 unit_time=sicte.year(),
36 unit_length=sicte.au(),
37 unit_mass=sicte.sol_mass(),
38 )
39 ucte = shamrock.Constants(codeu)
40 G = ucte.G()
41
42
43 def build_model_with_sinks():
44 ctx = shamrock.Context()
45 ctx.pdata_layout_new()
46
47 model = shamrock.get_Model_SPH(context=ctx, vector_type="f64_3", sph_kernel="M4")
48
49 cfg = model.gen_default_config()
50 cfg.set_self_gravity_none()
51 cfg.set_artif_viscosity_Constant(alpha_u=1.0, alpha_AV=1.0, beta_AV=2.0)
52 cfg.set_eos_isothermal(1.0)
53 cfg.set_particle_mass(1e-3)
54 cfg.set_boundary_periodic()
55 cfg.set_show_cfl_detail(True)
56 cfg.set_units(codeu)
57 model.set_solver_config(cfg)
58
59 model.set_cfl_cour(0.1)
60 model.set_cfl_force(0.1)
61 model.set_eta_sink(1.0)
62
63 model.init_scheduler(1000, 1)
64
65 # Very coarse HCP cube -> handful of SPH particles
66 dr = 0.05
67 bmin = (-0.6, -0.6, -0.6)
68 bmax = (0.6, 0.6, 0.6)
69 model.resize_simulation_box(bmin, bmax)
70
71 setup = model.get_setup()
72 gen = setup.make_generator_lattice_hcp(dr, bmin, bmax)
73 setup.apply_setup(gen)
74
75 eng = shamrock.algs.gen_seed(42)
76
77 def vel_func(r):
78 return (10.0, 0.0, 0.0)
79
80 model.set_field_value_lambda_f64_3("vxyz", vel_func)
81
82 # A few sinks (must be added after init_scheduler, on all ranks)
83 model.add_sink(1.0, (0.1, 0.0, 0.0), (0.0, 0.05, 0.0), 0.15)
84 model.add_sink(0.5, (-0.2, 0.1, 0.0), (0.0, -0.03, 0.0), 0.15)
85 model.add_sink(0.25, (0.0, -0.15, 0.05), (0.02, 0.0, 0.0), 0.15)
86
87 return ctx, model
88
89
90 def check_ref_dataset(sinks):
91 if shamrock.sys.world_rank() == 0:
92 print("Current sinks:")
93 print(sinks)
94
95 ref_dataset = [
96 {
97 "pos": (0.09877660057755726, -0.0003699820152028823, 0.00029285849440530886),
98 "velocity": (-0.7909594697171893, -0.42835509420057705, 0.2157916322019174),
99 "sph_acceleration": (-2.7881186060647103, 6.194080856496935, -3.386302543973038),
100 "ext_acceleration": (-371.6829482929875, -190.87247862152324, 85.76563352228864),
101 "mass": 1.0219999999999998,
102 "angular_momentum": (
103 1.5355132242207996e-05,
104 -3.404279116936264e-05,
105 -0.001391721046508032,
106 ),
107 "accretion_radius": 0.15,
108 },
109 {
110 "pos": (-0.19723897477916802, 0.0992597386368148, 5.457513126197254e-05),
111 "velocity": (1.7840284522444, -0.6462230101076988, 0.039967403747977054),
112 "sph_acceleration": (40.132674771440605, -16.361891265380414, -1.2376352720798245),
113 "ext_acceleration": (456.6529453658143, -211.77602552231514, 16.18781052807078),
114 "mass": 0.523,
115 "angular_momentum": (
116 1.4588956579124888e-08,
117 -1.6035141770323982e-07,
118 -0.0009268830439052644,
119 ),
120 "accretion_radius": 0.15,
121 },
122 {
123 "pos": (0.0019199655817502001, -0.1466495904953547, 0.048938432104203365),
124 "velocity": (2.1365106397204268, 3.0220342305086376, -0.9498166179408467),
125 "sph_acceleration": (4.433172833108257, 16.09220042839757, -6.362794011098254),
126 "ext_acceleration": (522.3351212189343, 1132.705683330991, -355.99519394799995),
127 "mass": 0.27,
128 "angular_momentum": (
129 4.4469346449927724e-05,
130 0.0006693090866850239,
131 0.0020383710492193246,
132 ),
133 "accretion_radius": 0.15,
134 },
135 ]
136
137 errors = []
138
139 if len(sinks) != len(ref_dataset):
140 errors.append(f"sink count mismatch: got {len(sinks)}, expected {len(ref_dataset)}")
141 else:
142 for i, (got_sink, ref_sink) in enumerate(zip(sinks, ref_dataset)):
143 for key, ref_val in ref_sink.items():
144 got_val = got_sink[key]
145 got_arr = np.asarray(got_val, dtype=float)
146 ref_arr = np.asarray(ref_val, dtype=float)
147 rtol = 1e-14 if key == "sph_acceleration" else 1e-15
148 if not np.all(np.isclose(got_arr, ref_arr, rtol=rtol, atol=1e-18)):
149 abs_diff = np.abs(got_arr - ref_arr)
150 with np.errstate(divide="ignore", invalid="ignore"):
151 rel_diff = np.where(ref_arr != 0, abs_diff / np.abs(ref_arr), abs_diff)
152 errors.append(
153 f"sink[{i}].{key} mismatch:\n"
154 f" got={got_val}\n"
155 f" ref={ref_val}\n"
156 f" max abs diff={np.max(abs_diff)}\n"
157 f" max rel diff={np.max(rel_diff)}"
158 )
159
160 for err in errors:
161 print(err)
162
163 if errors:
164 raise RuntimeError(f"check_ref_dataset failed with {len(errors)} error(s)")
165
166 if shamrock.sys.world_rank() == 0:
167 print("check_ref_dataset: OK")
168
169
170 def main():
171 ctx, model = build_model_with_sinks()
172
173 check_sinks_are_in_sync(ctx, model)
174
175 for _ in range(5):
176 model.timestep()
177 check_sinks_are_in_sync(ctx, model)
178
179 sinks_before_dump = str(model.get_sinks())
180 model.dump(DUMP_NAME)
181
182 del model
183 del ctx
184
185 ctx2 = shamrock.Context()
186 ctx2.pdata_layout_new()
187 model2 = shamrock.get_Model_SPH(context=ctx2, vector_type="f64_3", sph_kernel="M4")
188 model2.load_from_dump(DUMP_NAME)
189
190 sinks_after_reload = str(model2.get_sinks())
191 if sinks_before_dump != sinks_after_reload:
192 raise RuntimeError(
193 "sink content changed across dump/reload:\n"
194 f" before={sinks_before_dump!r}\n"
195 f" after ={sinks_after_reload!r}"
196 )
197
198 check_sinks_are_in_sync(ctx2, model2)
199
200 for _ in range(5):
201 model2.timestep()
202 check_sinks_are_in_sync(ctx2, model2)
203
204 if shamrock.sys.world_rank() == 0:
205 print("run_test_sink_synchro: OK")
206
207 check_ref_dataset(model2.get_sinks())
208
209 dic = ctx2.collect_data()
210
211 if shamrock.sys.world_rank() > 0:
212 return
213
214 assert 2266 == len(dic["xyz"])
215
216 sum_pos = np.sum(dic["xyz"], axis=0)
217 sum_vel = np.sum(dic["vxyz"], axis=0)
218 sum_acc = np.sum(dic["axyz"], axis=0)
219 sum_hpart = np.sum(dic["hpart"], axis=0)
220
221 dat = np.concatenate([sum_pos, sum_vel, sum_acc, np.atleast_1d(sum_hpart)])
222 print("Current sums: ", [dat[i] for i in range(len(dat))])
223
224 ref_sums = [
225 65.79475628688992,
226 -15.508903903707498,
227 -1.0412208430674998,
228 22613.455824802637,
229 -5.195701678016285,
230 15.008486573476965,
231 -19336.888355004605,
232 -2117.975619213131,
233 5826.038830234707,
234 245.85176557074223,
235 ]
236
237 mismatch = False
238 for i in range(len(dat)):
239 if not np.isclose(dat[i], ref_sums[i], rtol=1e-12, atol=1e-18):
240 abs_diff = np.abs(dat[i] - ref_sums[i])
241 rel_diff = abs_diff / np.abs(ref_sums[i])
242 print(f"sum[{i}] mismatch: got {dat[i]}, expected {ref_sums[i]}")
243 print(f" max abs diff={np.max(abs_diff)}")
244 print(f" max rel diff={np.max(rel_diff)}")
245 mismatch = True
246 if mismatch:
247 raise RuntimeError("sums mismatch")
248
249
250 main()
Total running time of the script: (0 minutes 4.554 seconds)
Estimated memory usage: 161 MB