问题1:Why do most RTP sessions use separate streams for audio and video? How can a receiving client synchronize these streams?
回答:Sending audio and video in separate RTP streams provides a great deal of flexibility. For example, this makes it possible for a player to receive only the audio stream, but not video (or vice-versa). It would even be possible to have one computer receive and play audio, and a separate computer receive and play video.

These audio and video streams are synchronized using RTCP “Sender Report” (SR) packets – which map each stream’s RTP timestamp to ‘wall clock’ (NTP) time. For more information, see the IETF’s RTP/RTCP specification.

Receivers can then use this mapping to synchronize the incoming RTP streams. The LIVE555 Streaming Media code does this automatically: For subclasses of “RTPSource”, the “presentationTime” parameter that’s passed to the ‘afterGettingFunc’ of “getNextFrame()” (see “liveMedia/include/FramedSource.hh”) will be an accurate, time-synchronized time. (For this to work, you need to have also created a “RTCPInstance” for each RTP source.)

For example, if you use “openRTSP” to receive RTSP/RTP streams, then the contents of each RTP stream (audio and video) are written into separate files. This is done using the “FileSink” class. If you look at the “FileSink::afterGettingFrame()” member function, you’ll notice that there’s a “presentationTime” parameter for each incoming frame. Some other receiver could use the “presentationTime” parameter to synchronize audio and video.

问题2:But I notice that there’s an abrupt change in a stream’s presentation times after the first RTCP “SR” packet has been received. Is this a bug?
回答:No, this is normal, and expected; there’s no bug here. This happens because the first few presentation times – before RTCP synchronization occurs – are just ‘guesses’ made by the receiving code (based on the receiver’s ‘wall clock’ and the RTP timestamp). However, once RTCP synchronization occurs, all subsequent presentation times will be accurate.

This means is that a receiver should be prepared for the fact that the first few presentation times (until RTCP synchronization starts) will not be accurate. The code, however, can check this by calling “RTPSource:: hasBeenSynchronizedUsingRTCP()”. If this returns False, then the presentation times are not accurate, and should not be used for synchronization. However, once the call to returns True, then the presentation times (from then on) will be accurate.

1. Live555中关于写文件时,需要先同步音视频流的例子见Live555的源码QuickTimeFileSink.cpp:

void QuickTimeFileSink
::afterGettingFrame(void* clientData, unsigned packetDataSize,
            unsigned numTruncatedBytes,
            struct timeval presentationTime,
            unsigned /*durationInMicroseconds*/) {
  SubsessionIOState* ioState = (SubsessionIOState*)clientData;
  if (!ioState->syncOK(presentationTime)) {
    // Ignore this data:音视频还未同步,忽略这些数据包,syncOK中调用了hasBeenSynchronizedUsingRTCP()
  ioState->afterGettingFrame(packetDataSize, presentationTime);

2.1 通过Live555源码搜索,查找RTPSource同步标记(fCurPacketHasBeenSynchronizedUsingRTCP)设置的逻辑:
—- hasBeenSynchronizedUsingRTCP Matches (17 in 9 files) —-
MediaSession.cpp (livemedia): if (!rtpSource()->hasBeenSynchronizedUsingRTCP()) {
MultiFramedRTPSource.cpp (livemedia): fPresentationTime, fCurPacketHasBeenSynchronizedUsingRTCP,
RTPSource.cpp (livemedia): fCurPacketHasBeenSynchronizedUsingRTCP(False), fLastReceivedSSRC(0),
RTPSource.hh (livemedia\include): virtual Boolean hasBeenSynchronizedUsingRTCP();
RTPSource.hh (livemedia\include): Boolean fCurPacketHasBeenSynchronizedUsingRTCP;
testRTSPClient.cpp: if (fSubsession.rtpSource() != NULL && !fSubsession.rtpSource()->hasBeenSynchronizedUsingRTCP()) {
—- fHasBeenSyncedUsingRTCP Matches (3 in 2 files) —-
MultiFramedRTPSource.cpp (livemedia): fHasBeenSyncedUsingRTCP = hasBeenSyncedUsingRTCP;
MultiFramedRTPSource.cpp (livemedia): hasBeenSyncedUsingRTCP = fHasBeenSyncedUsingRTCP;
MultiFramedRTPSource.hh (livemedia\include): Boolean fHasBeenSyncedUsingRTCP;
—- assignMiscParams Matches (3 in 2 files) —-
MultiFramedRTPSource.cpp (livemedia): bPacket->assignMiscParams(rtpSeqNo, rtpTimestamp, presentationTime,
—- hasBeenSyncedUsingRTCP Matches (18 in 4 files) —-
MultiFramedRTPSource.cpp (livemedia): fHasBeenSyncedUsingRTCP = hasBeenSyncedUsingRTCP;
RTPSource.cpp (livemedia): resultHasBeenSyncedUsingRTCP = fHasBeenSynchronized;

2.2 真正进行RTCP时间同步标记判断和保存的地方,最终会通过bPacket->assignMiscParams函数,赋值给RTPSource::fCurPacketHasBeenSynchronizedUsingRTCP变量:

void MultiFramedRTPSource::networkReadHandler1()
    struct timeval presentationTime; // computed by:
    Boolean hasBeenSyncedUsingRTCP; // computed by:
      .noteIncomingPacket(rtpSSRC, rtpSeqNo, rtpTimestamp,
              usableInJitterCalculation, presentationTime,
              hasBeenSyncedUsingRTCP, bPacket->dataSize());

    // Fill in the rest of the packet descriptor, and store it:
    struct timeval timeNow;
    gettimeofday(&timeNow, NULL);
    bPacket->assignMiscParams(rtpSeqNo, rtpTimestamp, presentationTime,
                  hasBeenSyncedUsingRTCP, rtpMarkerBit,


—- curPacketRTPTimestamp( Matches (3 in 2 files) —-
MediaSession.cpp (livemedia): u_int32_t timestampOffset = rtpSource()->curPacketRTPTimestamp() – rtpInfo.timestamp;
MediaSession.cpp (livemedia): u_int32_t timestampOffset = rtpSource()->curPacketRTPTimestamp() – rtpInfo.timestamp;
RTPSource.hh (livemedia\include): u_int32_t curPacketRTPTimestamp() const { return fCurPacketRTPTimestamp; }
—- getNormalPlayTime Matches (4 in 3 files) —-
MediaSession.cpp (livemedia):double MediaSubsession::getNormalPlayTime(struct timeval const& presentationTime) {
MediaSession.hh (livemedia\include): double getNormalPlayTime(struct timeval const& presentationTime);
MediaSession.hh (livemedia\include): double fNPT_PTS_Offset; // set by “getNormalPlayTime()”; add this to a PTS to get NPT
testRTSPClient.cpp: envir() << “\tNPT: ” << fSubsession.getNormalPlayTime(presentationTime);

共 1 个评论(Comment)

  1. Uranus Zhou says:

    你好,我在用live555做MPEG-2 PS视频回放时发现正常播放VLC的进度条是好的,
    live555 debug: seek start: 39.621000 stop:79.147000
    main debug: Stream buffering done (26918 ms in 13 ms)
    main warning: picture is too late to be displayed (missing 26121 ms)
    live555 debug: tk->rtpSource->hasBeenSynchronizedUsingRTCP()
    main error: ES_OUT_RESET_PCR called



你的邮件地址不会被公开(Your email address will not be published.) Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>