High bitrate using SRT may cause video data corruption.

[nivhsay]

Observed behavior

We are having trouble with publishing SRT stream with HEVC.
During low motion scenes, the stream breaks almost completely. It seems HEVC cannot reference the previous frames correctly.
We have noticed that during this period, there is almost no Key Frame being sent by the app... and we have mostly P-Frames.
This might be causing the loss of sync from the decoder and the new frames cannot be reconstructed correctly.
If during the same live, the motion gets high again, we start receiving key frames again and the stream recovers.
As soon as motion gets low again, the stream breaks again.

We have also found out that if we disable temporal compression (kVTCompressionPropertyKey_AllowTemporalCompression), the stream always works. This is probably because all frames are being sent as Key Frames.
However, this is not a solution for us, as we need to use temporal compression for better quality at lower bitrates.

Tested with the front camera on iOS 18.1.1
For some reason, it works with the back camera in most of our tests.

Video settings dump is as follows:

VideoCodecSettings(
  videoSize: (1080.0, 1920.0),
  bitRate: 4194304, profileLevel: "HEVC_Main_AutoLevel",
  scalingMode: HaishinKit.VideoCodecSettings.ScalingMode.trim,
  bitRateMode: HaishinKit.VideoCodecSettings.BitRateMode.average,
  maxKeyFrameIntervalDuration: 2,
  allowFrameReordering: Optional(false),
  dataRateLimits: Optional([0.0, 0.0]),
  isHardwareEncoderEnabled: true,
  frameInterval: 0.0,
  format: HaishinKit.VideoCodecSettings.Format.hevc
)

We tried with several SRT servers in listener mode and it does not work with any of them.
The following dump is from running SRT server with FFMPEG.

ffmpeg -loglevel debug -i "srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760" -c copy -f flv rtmp://nms:1935/preview/12345678

ffmpeg -loglevel

node@8fe48fc306ce:/app# ffmpeg -loglevel debug -i "srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760" -c copy -f flv rtmp://nms:1935/preview/12345678
ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --pkg-config-flags=--static --enable-gpl --enable-libx264 --enable-libx265 --enable-libvpx --enable-libfreetype --enable-libopus --enable-libmp3lame --enable-version3 --enable-libsrt --enable-openssl --enable-nonfree
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument 'srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760'.
Reading option '-c' ... matched as option 'c' (select encoder/decoder ('copy' to copy stream without reencoding)) with argument 'copy'.
Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'flv'.
Reading option 'rtmp://nms:1935/preview/12345678' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760.
Successfully parsed a group of options.
Opening an input file: srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760.
[AVFormatContext @ 0xaaab1e89e6f0] Opening 'srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760' for reading
[srt @ 0xaaab1e89ed70] No default whitelist set
[srt @ 0xaaab1e89ed70] accept streamid [12345678], length 8
[mpegts @ 0xaaab1e89e6f0] Format mpegts probed with size=2048 and score=50
[mpegts @ 0xaaab1e89e6f0] stream=0 stream_type=24 pid=100 prog_reg_desc=
[mpegts @ 0xaaab1e89e6f0] stream=1 stream_type=f pid=101 prog_reg_desc=
[mpegts @ 0xaaab1e89e6f0] Before avformat_find_stream_info() pos: 0 bytes read:2068 seeks:0 nb_streams:2
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] Decoding VPS
[hevc @ 0xaaab1e8d2da0] Main profile bitstream
[hevc @ 0xaaab1e8d2da0] Decoding SPS
[hevc @ 0xaaab1e8d2da0] Main profile bitstream
[hevc @ 0xaaab1e8d2da0] Decoding PPS
[hevc @ 0xaaab1e8d2da0] Decoding SEI
[extract_extradata @ 0xaaab1e8cd9e0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[extract_extradata @ 0xaaab1e8cd9e0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[extract_extradata @ 0xaaab1e8cd9e0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[extract_extradata @ 0xaaab1e8cd9e0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[extract_extradata @ 0xaaab1e8cd9e0] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] Decoding VPS
[hevc @ 0xaaab1e8d2da0] Main profile bitstream
[hevc @ 0xaaab1e8d2da0] Decoding SPS
[hevc @ 0xaaab1e8d2da0] Main profile bitstream
[hevc @ 0xaaab1e8d2da0] Decoding PPS
[hevc @ 0xaaab1e8d2da0] Decoding SEI
[hevc @ 0xaaab1e8d2da0] Format yuv420p chosen by get_format().
[hevc @ 0xaaab1e8d2da0] Output frame with POC 0/0.
[hevc @ 0xaaab1e8d2da0] Decoded frame with POC 0/0.
[mpegts @ 0xaaab1e89e6f0] probing stream 1 pp:2500
[mpegts @ 0xaaab1e89e6f0] Probe with size=11, packets=1 detected flac with score=13
[mpegts @ 0xaaab1e89e6f0] probing stream 1 pp:2499
[mpegts @ 0xaaab1e89e6f0] Probe with size=22, packets=2 detected flac with score=13
[mpegts @ 0xaaab1e89e6f0] probing stream 1 pp:2498
[mpegts @ 0xaaab1e89e6f0] Probe with size=33, packets=3 detected aac with score=51
[mpegts @ 0xaaab1e89e6f0] probed stream 1
Transform tree:
    mdct_pfa_3xM_inv_float_c - type: mdct_float, len: 96, factors[2]: [3, any], flags: [unaligned, out_of_place, inv_only]
        fft16_ns_float_neon - type: fft_float, len: 16, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_pfa_15xM_inv_float_c - type: mdct_float, len: 120, factors[2]: [15, any], flags: [unaligned, out_of_place, inv_only]
        fft4_fwd_float_neon - type: fft_float, len: 4, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 128, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft_sr_ns_float_neon - type: fft_float, len: 64, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_pfa_15xM_inv_float_c - type: mdct_float, len: 480, factors[2]: [15, any], flags: [unaligned, out_of_place, inv_only]
        fft16_ns_float_neon - type: fft_float, len: 16, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 512, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft_sr_ns_float_neon - type: fft_float, len: 256, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_pfa_3xM_inv_float_c - type: mdct_float, len: 768, factors[2]: [3, any], flags: [unaligned, out_of_place, inv_only]
        fft_sr_ns_float_neon - type: fft_float, len: 128, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_pfa_15xM_inv_float_c - type: mdct_float, len: 960, factors[2]: [15, any], flags: [unaligned, out_of_place, inv_only]
        fft32_ns_float_neon - type: fft_float, len: 32, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 1024, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft_sr_ns_float_neon - type: fft_float, len: 512, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_fwd_float_c - type: mdct_float, len: 1024, factors[2]: [2, any], flags: [unaligned, out_of_place, fwd_only]
        fft_sr_ns_float_neon - type: fft_float, len: 512, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 64, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft32_ns_float_neon - type: fft_float, len: 32, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 64, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft32_ns_float_neon - type: fft_float, len: 32, factor: 2, flags: [aligned, inplace, out_of_place, preshuf]
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e8d2da0] Decoding SPS
[hevc @ 0xaaab1e8d2da0] Main profile bitstream
[hevc @ 0xaaab1e8d2da0] Decoding PPS
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 54 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 58 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e8d2da0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 32 times
[mpegts @ 0xaaab1e89e6f0] max_analyze_duration 5000000 reached at 5000278 microseconds st:0
[mpegts @ 0xaaab1e89e6f0] rfps: 29.916667 0.014125
    Last message repeated 1 times
[mpegts @ 0xaaab1e89e6f0] rfps: 30.000000 0.000006
[mpegts @ 0xaaab1e89e6f0] rfps: 60.000000 0.000022
[mpegts @ 0xaaab1e89e6f0] rfps: 120.000000 0.000088
[mpegts @ 0xaaab1e89e6f0] rfps: 240.000000 0.000353
[mpegts @ 0xaaab1e89e6f0] rfps: 29.970030 0.001701
    Last message repeated 1 times
[mpegts @ 0xaaab1e89e6f0] rfps: 59.940060 0.006804
    Last message repeated 1 times
[mpegts @ 0xaaab1e89e6f0] After avformat_find_stream_info() pos: 2430276 bytes read:2430276 seeks:0 frames:365
Input #0, mpegts, from 'srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760':
  Duration: N/A, start: 0.099989, bitrate: N/A
  Program 1 
  Stream #0:0[0x100], 147, 1/90000: Video: hevc (Main), 1 reference frame ([36][0][0][0] / 0x0024), yuv420p(tv, left), 1080x1920, 0/1, 30 tbr, 90k tbn
  Stream #0:1[0x101], 218, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 3 kb/s
Successfully opened the file.
Parsing a group of options: output url rtmp://nms:1935/preview/12345678.
Applying option c (select encoder/decoder ('copy' to copy stream without reencoding)) with argument copy.
Applying option f (force container format (auto-detected otherwise)) with argument flv.
Successfully parsed a group of options.
Opening an output file: rtmp://nms:1935/preview/12345678.
[out#0/flv @ 0xaaab1ea63a80] No explicit maps, mapping streams automatically...
[vost#0:0/copy @ 0xaaab1ea4f720] Created video stream from input stream 0:0
[aost#0:1/copy @ 0xaaab1ea42cc0] Created audio stream from input stream 0:1
[rtmp @ 0xaaab1ea57140] No default whitelist set
[tcp @ 0xaaab1ea61740] No default whitelist set
[tcp @ 0xaaab1ea61740] Original list of addresses:
[tcp @ 0xaaab1ea61740] Address 10.150.0.60 port 1935
[tcp @ 0xaaab1ea61740] Interleaved list of addresses:
[tcp @ 0xaaab1ea61740] Address 10.150.0.60 port 1935
[tcp @ 0xaaab1ea61740] Starting connection attempt to 10.150.0.60 port 1935
[tcp @ 0xaaab1ea61740] Successfully connected to 10.150.0.60 port 1935
[rtmp @ 0xaaab1ea57140] Handshaking...
[rtmp @ 0xaaab1ea57140] Type answer 3
[rtmp @ 0xaaab1ea57140] Server version 1.2.3.4
[rtmp @ 0xaaab1ea57140] Proto = rtmp, path = /preview/12345678, app = preview, fname = 12345678
[rtmp @ 0xaaab1ea57140] Window acknowledgement size = 5000000
[rtmp @ 0xaaab1ea57140] Max sent, unacked = 5000000
[rtmp @ 0xaaab1ea57140] New incoming chunk size = 128
[rtmp @ 0xaaab1ea57140] Releasing stream...
[rtmp @ 0xaaab1ea57140] FCPublish stream...
[rtmp @ 0xaaab1ea57140] Creating stream...
[rtmp @ 0xaaab1ea57140] Sending publish command for '12345678'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Output #0, flv, to 'rtmp://nms:1935/preview/12345678':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0, 0, 1/1000: Video: hevc (Main), 1 reference frame (1cvh / 0x68766331), yuv420p(tv, left), 1080x1920, 0/1, q=2-31, 30 tbr, 1k tbn
  Stream #0:1, 0, 1/1000: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, mono, fltp, 3 kb/s
[out#0/flv @ 0xaaab1ea63a80] Starting thread...
[in#0/mpegts @ 0xaaab1e89e490] Starting thread...
Press [q] to stop, [?] for help
Automatically inserted bitstream filter 'aac_adtstoasc'; args=''
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 19 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=11.4x    
    Last message repeated 4 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 8 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=6.25x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=4.48x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=3.62x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=3.09x    
    Last message repeated 6 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 7 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=2.75x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=2.49x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=2.31x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=2.16x    
    Last message repeated 6 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 7 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=2.05x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.95x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.87x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.81x    
    Last message repeated 7 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 6 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.75x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed= 1.7x    
    Last message repeated 15 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.66x    
    Last message repeated 12 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.62x    
    Last message repeated 7 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 6 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.58x    
    Last message repeated 6 times
[hevc @ 0xaaab1e908570] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] Decoding SPS
[hevc @ 0xaaab1e908570] Main profile bitstream
[hevc @ 0xaaab1e908570] Decoding PPS
[hevc @ 0xaaab1e908570] Decoding SEI
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 6 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.55x    
    Last message repeated 15 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.52x    
    Last message repeated 4 times
[hevc @ 0xaaab1e908570] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] Decoding SPS
[hevc @ 0xaaab1e908570] Main profile bitstream
[hevc @ 0xaaab1e908570] Decoding PPS
[hevc @ 0xaaab1e908570] Decoding SEI
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 8 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed= 1.5x    
    Last message repeated 8 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 5 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.48x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.46x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.44x    
    Last message repeated 2 times
[hevc @ 0xaaab1e908570] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] Decoding SPS
[hevc @ 0xaaab1e908570] Main profile bitstream
[hevc @ 0xaaab1e908570] Decoding PPS
[hevc @ 0xaaab1e908570] Decoding SEI
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 10 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.42x    
    Last message repeated 8 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 5 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed= 1.4x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.39x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.37x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.36x    
    Last message repeated 9 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 4 times
[hevc @ 0xaaab1e908570] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] Decoding SPS
[hevc @ 0xaaab1e908570] Main profile bitstream
[hevc @ 0xaaab1e908570] Decoding PPS
[hevc @ 0xaaab1e908570] Decoding SEI
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.35x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.34x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] nal_unit_type: 20(IDR_N_LP), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0xaaab1e908570] Decoding SPS
[hevc @ 0xaaab1e908570] Main profile bitstream
[hevc @ 0xaaab1e908570] Decoding PPS
[hevc @ 0xaaab1e908570] Decoding SEI
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.33x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.32x    
    Last message repeated 9 times
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 0 expected 1 got 0
[mpegts @ 0xaaab1e89e6f0] Continuity check failed for pid 4095 expected 1 got 0
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
    Last message repeated 4 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.31x    
    Last message repeated 13 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed= 1.3x    
    Last message repeated 14 times
[hevc @ 0xaaab1e908570] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 speed=1.29x    
    Last message repeated 5 times
[in#0/mpegts @ 0xaaab1e89e490] Error during demuxing: Input/output error
[in#0/mpegts @ 0xaaab1e89e490] Terminating thread with return code 0 (success)
[out#0/flv @ 0xaaab1ea63a80] All streams finished
[out#0/flv @ 0xaaab1ea63a80] Terminating thread with return code 0 (success)
[flv @ 0xaaab1ea4e0f0] Failed to update header with correct duration.
[flv @ 0xaaab1ea4e0f0] Failed to update header with correct filesize.
[AVIOContext @ 0xaaab1ea445b0] Statistics: 10911799 bytes written, 0 seeks, 1715 writeouts
[rtmp @ 0xaaab1ea57140] UnPublishing stream...
[rtmp @ 0xaaab1ea57140] Deleting stream...
[out#0/flv @ 0xaaab1ea63a80] Output file #0 (rtmp://nms:1935/preview/12345678):
[out#0/flv @ 0xaaab1ea63a80]   Output stream #0:0 (video): 687 packets muxed (10876312 bytes); 
[out#0/flv @ 0xaaab1ea63a80]   Output stream #0:1 (audio): 1011 packets muxed (11121 bytes); 
[out#0/flv @ 0xaaab1ea63a80]   Total: 1698 packets (10887433 bytes) muxed
[out#0/flv @ 0xaaab1ea63a80] video:10621KiB audio:11KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.223799%
frame=  687 fps= 38 q=-1.0 Lsize=   10656KiB time=00:00:23.47 bitrate=3718.6kbits/s speed=1.29x    
[in#0/mpegts @ 0xaaab1e89e490] Input file #0 (srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760):
[in#0/mpegts @ 0xaaab1e89e490]   Input stream #0:0 (video): 687 packets read (10876312 bytes); 
[in#0/mpegts @ 0xaaab1e89e490]   Input stream #0:1 (audio): 1011 packets read (11121 bytes); 
[in#0/mpegts @ 0xaaab1e89e490]   Total: 1698 packets (10887433 bytes) demuxed
[AVIOContext @ 0xaaab1e8cdde0] Statistics: 11389040 bytes read, 0 seeks

Please note that everything works fine using a different publisher.
e.g.

• OBS, works fine with HEVC and SRT.
• Larix Broadcaster, works fine with HEVC and SRT.

Any help is much appreciated! Thanks!

Expected behavior

SRT stream with HEVC to work without breaking on the receiver side.

To Reproduce

• Use video settings as in bug report
• Try publishing to an SRT receiver
• Observe scene with little or no motion (largely static ones)

Version

HaishinKit.swift 2.0.3

Smartphone info.

• iPhone 14
• iOS 18.1.1

Additional context

No response

Screenshots

No response

Relevant log output

[nikolaiaas]

Hi @shogo4405
I work with nivhsay, and want to say that we are very happy with the work that you are doing.
I have send a sponsorship to you today.
I hope you can help us with this issue, as it is the last bit we are missing after a rewrite from 1.9.x to 2.x

[shogo4405]

Hello. Thank you for your support.

I have also confirmed the reproducibility on my end and am prioritizing this issue.
I am currently debugging it, but it looks like it will take some time.

[shogo4405]

I am currently testing, but this fix is expected to resolve the issue.

• fixed Large PES packets could be dropped. #1678

Potentially, it also occurs in the 1.9.x series, so I plan to apply the fix to 1.9.x as well.
https://github.com/shogo4405/HaishinKit.swift/blob/c4abd22c48cfe32590a94c9493306ed0d8a856ce/Sources/ISO/PacketizedElementaryStream.swift#L243-L248

[shogo4405]

@nivhsay @nikolaiaas
The issue has been fixed. Please take a look. Thank u.
You can test at main branch. f2ab619

[nivhsay]

@shogo4405 We are still having corrupt packets

2025-02-05T11:10:03.908406192Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6381335).
2025-02-05T11:10:03.908428549Z fd:: corrupt input packet in stream 0
2025-02-05T11:10:03.908455158Z [mpegts @ 0x556d10629f80] PES packet size mismatch
2025-02-05T11:10:03.908476508Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6393336).
2025-02-05T11:10:03.908501549Z fd:: corrupt input packet in stream 0
2025-02-05T11:10:03.908530856Z [mpegts @ 0x556d10629f80] PES packet size mismatch
2025-02-05T11:10:03.908554648Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6399336).
2025-02-05T11:10:03.908576918Z fd:: corrupt input packet in stream 0
2025-02-05T11:10:03.908599918Z [mpegts @ 0x556d10629f80] PES packet size mismatch
2025-02-05T11:10:03.908622662Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6402337).
2025-02-05T11:10:03.908650644Z fd:: corrupt input packet in stream 0
2025-02-05T11:10:03.908674635Z [mpegts @ 0x556d10629f80] PES packet size mismatch
2025-02-05T11:10:03.908700473Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6408336).
2025-02-05T11:10:03.908726193Z fd:: corrupt input packet in stream 0
2025-02-05T11:10:03.908754575Z [mpegts @ 0x556d10629f80] PES packet size mismatch
2025-02-05T11:10:03.908776832Z [mpegts @ 0x556d10629f80] Packet corrupt (stream = 0, dts = 6435338).
2025-02-05T11:10:03.908802296Z fd:: corrupt input packet in stream 0

[shogo4405]

Specifying latency=1500 reproduced the issue. Since there were no problems with a single stream, could you let me know the version you are using 1.x.x series.

Reproduable

$ ffmpeg -loglevel debug -i "srt://0.0.0.0:2935?streamid=12345678&latency=1500&mode=listener&recv_buffer_size=10485760" -c copy -f flv rtmp://localhost/live/live

Not Reproduable

$ ffmpeg -loglevel debug -i "srt://0.0.0.0:2935?streamid=12345678mode=listener&recv_buffer_size=10485760" -c copy -f flv  rtmp://localhost/live/live

[nivhsay]

SRT version?
We are running 1.5.3

I have noticed that the issue corrupt packet issue does not happen when using WiFi... but if I switch the iPhone to LTE, then there is a lot of them.