First release

node_modules/@videojs/http-streaming/docs/lhls/current-flow.plantuml

@@ -0,0 +1,13 @@
+@startuml
+
+state "Download Segment" as DL
+state "Prepare for Append" as PfA
+
+[*] -> DL
+DL -> PfA
+PfA : transmux (if needed)
+PfA -> Append
+Append : MSE source buffer
+Append -> [*]
+
+@enduml

node_modules/@videojs/http-streaming/docs/lhls/current-transmux-and-append-flow.plantuml

@@ -0,0 +1,57 @@
+@startuml
+
+participant SegmentLoader order 1
+participant "media-segment-request" order 2
+participant "videojs-contrib-media-sources" order 3
+participant mux.js order 4
+participant "Native Source Buffer" order 5
+
+SegmentLoader -> "media-segment-request" : mediaSegmentRequest(...)
+
+group Request
+  "media-segment-request" -> SegmentLoader : doneFn(...)
+  note left
+    At end of all requests
+    (key/segment/init segment)
+  end note
+  SegmentLoader -> SegmentLoader : handleSegment(...)
+  note left
+    "Probe" (parse) segment for
+    timing and track information
+  end note
+  SegmentLoader -> "videojs-contrib-media-sources" : append to "fake" source buffer
+  note left
+    Source buffer here is a
+    wrapper around native buffers
+  end note
+  group Transmux
+    "videojs-contrib-media-sources" -> mux.js : postMessage(...setAudioAppendStart...)
+    note left
+      Used for checking for overlap when
+      prefixing audio with silence.
+    end note
+    "videojs-contrib-media-sources" -> mux.js : postMessage(...alignGopsWith...)
+    note left
+      Used for aligning gops when overlapping
+      content (switching renditions) to fix
+      some browser glitching.
+    end note
+
+    "videojs-contrib-media-sources" -> mux.js : postMessage(...push...)
+    note left
+      Pushes bytes into the transmuxer pipeline.
+    end note
+    "videojs-contrib-media-sources" -> mux.js : postMessage(...flush...)
+    "mux.js" -> "videojs-contrib-media-sources" : postMessage(...data...)
+    "videojs-contrib-media-sources" -> "Native Source Buffer" : append
+    "Native Source Buffer" -> "videojs-contrib-media-sources" : //updateend//
+    "videojs-contrib-media-sources" -> SegmentLoader : handleUpdateEnd(...)
+  end
+end
+SegmentLoader -> SegmentLoader : handleUpdateEnd_()
+note left
+  Saves segment timing info
+  and starts next request.
+end note
+
+@enduml

node_modules/@videojs/http-streaming/docs/lhls/expected-flow.plantuml

@@ -0,0 +1,29 @@
+@startuml
+
+state "Request Segment" as RS
+state "Partial Response (1)" as PR1
+state "..." as DDD
+state "Partial Response (n)" as PRN
+state "Prepare for Append (1)" as PfA1
+state "Prepare for Append (n)" as PfAN
+state "Append (1)" as A1
+state "Append (n)" as AN
+
+[*] -> RS
+
+RS --> PR1
+PR1 --> DDD
+DDD --> PRN
+
+PR1 -> PfA1
+PfA1 : transmux (if needed)
+PfA1 -> A1
+A1 : MSE source buffer
+PRN -> PfAN
+PfAN : transmux (if needed)
+PfAN -> AN
+AN : MSE source buffer
+
+AN --> [*]
+
+@enduml

node_modules/@videojs/http-streaming/docs/lhls/index.md

@@ -0,0 +1,109 @@
+# LHLS
+
+### Table of Contents
+
+* [Background](#background)
+* [Current Support for LHLS in VHS](#current-support-for-lhls-in-vhs)
+* [Request a Segment in Pieces](#request-a-segment-in-pieces)
+* [Transmux and Append Segment Pieces](#transmux-and-append-segment-pieces)
+  * [videojs-contrib-media-sources background](#videojs-contrib-media-sources-background)
+  * [Transmux Before Append](#transmux-before-append)
+  * [Transmux Within media-segment-request](#transmux-within-media-segment-request)
+  * [mux.js](#muxjs)
+  * [The New Flow](#the-new-flow)
+* [Resources](#resources)
+
+### Background
+
+LHLS stands for Low-Latency HLS (see [Periscope's post](https://medium.com/@periscopecode/introducing-lhls-media-streaming-eb6212948bef)). It's meant to be used for ultra low latency live streaming, where a server can send pieces of a segment before the segment is done being written to, and the player can append those pieces to the browser, allowing sub segment duration latency from true live.
+
+In order to support LHLS, a few components are required:
+
+* A server that supports [chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding).
+* A client that can:
+  * request segment pieces
+  * transmux segment pieces (for browsers that don't natively support the media type)
+  * append segment pieces
+
+### Current Support for LHLS in VHS
+
+At the moment, VHS doesn't support any of the client requirements. It waits until a request is completed and the transmuxer expects full segments.
+
+Current flow:
+
+![current flow](./current-flow.plantuml.png)
+
+Expected flow:
+
+![expected flow](./expected-flow.plantuml.png)
+
+### Request Segment Pieces
+
+The first change was to request pieces of a segment. There are a few approaches to accomplish this:
+
+* [Range Requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests)
+  * requires server support
+  * more round trips
+* [Fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)
+  * limited browser support
+  * doesn't support aborts
+* [Plain text MIME type](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Sending_and_Receiving_Binary_Data)
+  * slightly non-standard
+  * incurs a cost of converting from string to bytes
+
+*Plain text MIME type* was chosen because of its wide support. It provides a mechanism to access progressive bytes downloaded on [XMLHttpRequest progress events](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequestEventTarget/onprogress).
+
+This change was made in [media-segment-request](https://github.com/videojs/http-streaming/blob/master/src/media-segment-request.js).
+
+### Transmux and Append Segment Pieces
+
+Getting the progress bytes is easy. Supporting partial transmuxing and appending is harder.
+
+Current flow:
+
+![current transmux and append flow](./current-transmux-and-append-flow.plantuml.png)
+
+In order to support partial transmuxing and appending in the current flow, videojs-contrib-media-sources would have to get more complicated.
+
+##### videojs-contrib-media-sources background
+
+Browsers, via MSE source buffers, only support a limited set of media types. For most browsers, this means MP4/fragmented MP4. HLS uses TS segments (it also supports fragmented MP4, but that case is less common). This is why transmuxing is necessary.
+
+Just like Video.js is a wrapper around the browser video element, bridging compatibility and adding support to extend features, videojs-contrib-media-sources provides support for more media types across different browsers by building in a transmuxer.
+
+Not only did videojs-contrib-media-sources allow us to transmux TS to FMP4, but it also allowed us to transmux TS to FLV for flash support.
+
+Over time, the complexity of logic grew in videojs-contrib-media-sources, and it coupled tightly with videojs-contrib-hls and videojs-http-streaming, firing events to communicate between the two.
+
+Once flash support was moved to a distinct flash module, [via flashls](https://github.com/brightcove/videojs-flashls-source-handler), it was decided to move the videojs-contrib-media-sources logic into VHS, and to remove coupled logic by using only the native source buffers (instead of the wrapper) and transmuxing somewhere within VHS before appending.
+
+##### Transmux Before Append
+
+As the LHLS work started, and videojs-contrib-media-sources needed more logic, the native media source [abstraction leaked](https://en.wikipedia.org/wiki/Leaky_abstraction), adding non-standard functions to work around limitations. In addition, the logic in videojs-contrib-media-sources required more conditional paths, leading to more confusing code.
+
+It was decided that it would be easier to do the transmux before append work in the process of adding support for LHLS. This was widely considered a *good decision*, and provided a means of reducing tech debt while adding in a new feature.
+
+##### Transmux Within media-segment-request
+
+Work started by moving transmuxing into segment-loader, however, we quickly realized that media-segment-request provided a better home.
+
+media-segment-request already handled decrypting segments. If it handled transmuxing as well, then segment-loader could stick with only deciding which segment to request, getting bytes as FMP4, and appending them.
+
+The transmuxing logic moved to a new module called segment-transmuxer, which wrapped around the [WebWorker](https://developer.mozilla.org/en-US/docs/Web/API/Worker/Worker) that wrapped around mux.js (the transmuxer itself).
+
+##### mux.js
+
+While most of the [mux.js pipeline](https://github.com/videojs/mux.js/blob/master/docs/diagram.png) supports pushing pieces of data (and should support LHLS by default), its "flushes" to send transmuxed data back to the caller expected full segments.
+
+Much of the pipeline was reused, however, the top level audio and video segment streams, as well as the entry point, were rewritten so that instead of providing a full segment on flushes, each frame of video was provided individually (audio frames still flush as a group). The new concept of partial flushes was added into the pipeline to handle this case.
+
+##### The New Flow
+
+One benefit to transmuxing before appending is the possibility of extracting track and timing information from the segments. Previously, this required a separate parsing step to happen on the full segment. Now, it is included in the transmuxing pipeline, and comes back to us on separate callbacks.
+
+![new segment loader sequence](./new-segment-loader-sequence.plantuml.png)
+
+### Resources
+
+* https://medium.com/@periscopecode/introducing-lhls-media-streaming-eb6212948bef
+* https://github.com/jordicenzano/webserver-chunked-growingfiles

node_modules/@videojs/http-streaming/docs/lhls/new-segment-loader-sequence.plantuml

@@ -0,0 +1,118 @@
+@startuml
+
+participant SegmentLoader order 1
+participant "media-segment-request" order 2
+participant XMLHttpRequest order 3
+participant "segment-transmuxer" order 4
+participant mux.js order 5
+
+SegmentLoader -> "media-segment-request" : mediaSegmentRequest(...)
+"media-segment-request" -> XMLHttpRequest : request for segment/key/init segment
+
+group Request
+  XMLHttpRequest -> "media-segment-request" : //segment progress//
+  note over "media-segment-request" #moccasin
+   If handling partial data,
+   tries to transmux new
+   segment bytes.
+  end note
+  "media-segment-request" -> SegmentLoader : progressFn(...)
+  note left
+    Forwards "progress" events from
+    the XML HTTP Request.
+  end note
+  group Transmux
+    "media-segment-request" -> "segment-transmuxer" : transmux(...)
+
+    "segment-transmuxer" -> mux.js : postMessage(...setAudioAppendStart...)
+    note left
+      Used for checking for overlap when
+      prefixing audio with silence.
+    end note
+    "segment-transmuxer" -> mux.js : postMessage(...alignGopsWith...)
+    note left
+      Used for aligning gops when overlapping
+      content (switching renditions) to fix
+      some browser glitching.
+    end note
+
+    "segment-transmuxer" -> mux.js : postMessage(...push...)
+    note left
+      Pushes bytes into the transmuxer pipeline.
+    end note
+
+    "segment-transmuxer" -> mux.js : postMessage(...partialFlush...)
+    note left #moccasin
+      Collates any complete frame data
+      from partial segment and
+      caches remainder.
+    end note
+    "segment-transmuxer" -> mux.js : postMessage(...flush...)
+    note left
+      Collates any complete frame data
+      from segment, caches only data
+      required between segments.
+    end note
+
+    "mux.js" -> "segment-transmuxer" : postMessage(...trackinfo...)
+    "segment-transmuxer" -> "media-segment-request" : onTrackInfo(...)
+    "media-segment-request" -> SegmentLoader : trackInfoFn(...)
+    note left
+      Gets whether the segment
+      has audio and/or video.
+    end note
+    "mux.js" -> "segment-transmuxer" : postMessage(...audioTimingInfo...)
+    "segment-transmuxer" -> "media-segment-request" : onAudioTimingInfo(...)
+    "mux.js" -> "segment-transmuxer" : postMessage(...videoTimingInfo...)
+    "segment-transmuxer" -> "media-segment-request" : onVideoTimingInfo(...)
+    "media-segment-request" -> SegmentLoader : timingInfoFn(...)
+    note left
+      Gets the audio/video
+      start/end times.
+    end note
+
+    "mux.js" -> "segment-transmuxer" : postMessage(...caption...)
+    "segment-transmuxer" -> "media-segment-request" : onCaptions(...)
+    "media-segment-request" -> SegmentLoader : captionsFn(...)
+    note left
+      Gets captions from transmux.
+    end note
+
+    "mux.js" -> "segment-transmuxer" : postMessage(...id3Frame...)
+    "segment-transmuxer" -> "media-segment-request" : onId3(...)
+    "media-segment-request" -> SegmentLoader : id3Fn(...)
+    note left
+      Gets metadata from transmux.
+    end note
+
+    "mux.js" -> "segment-transmuxer" : postMessage(...data...)
+    "segment-transmuxer" -> "media-segment-request" : onData(...)
+    "media-segment-request" -> SegmentLoader : dataFn(...)
+    note left
+      Gets an fmp4 segment
+      ready to be appended.
+    end note
+
+    "mux.js" -> "segment-transmuxer" : postMessage(...done...)
+    note left
+      Gathers GOP info, and calls
+      done callback.
+    end note
+    "segment-transmuxer" -> "media-segment-request" : onDone(...)
+    "media-segment-request" -> SegmentLoader : doneFn(...)
+    note left
+      Queues callbacks on source
+      buffer queue to wait for
+      appends to complete.
+    end note
+  end
+  XMLHttpRequest -> "media-segment-request" : //segment request finished//
+end
+
+SegmentLoader -> SegmentLoader : handleAppendsDone_()
+note left
+  Saves segment timing info
+  and starts next request.
+end note
+
+@enduml

node_modules/@videojs/http-streaming/docs/lhls/transmux-before-append-changes.md

@@ -0,0 +1,36 @@
+# Transmux Before Append Changes
+
+## Overview
+
+In moving our transmuxing stage from after append (to a virtual source buffer from videojs-contrib-media-sources) to before appending (to a native source buffer), some changes were required, and others made the logic simpler. What follows are some details into some of the changes made, why they were made, and what impact they will have.
+
+### Source Buffer Creation
+
+In a pre-TBA (transmux before append) world, videojs-contrib-media-source's source buffers provided an abstraction around the native source buffers. They also required a bit more information than the native buffers. For instance, they used the full mime types instead of simply relying on the codec information, when creating the source buffers. This provided the container types, which let the virtual source buffer know whether the media needed to be transmuxed or not. In a post-TBA world, the container type is no longer required, therefore only the codec strings are passed along.
+
+In terms of when the source buffers are created, in the post-TBA world, the creation of source buffers is delayed until we are sure we have all of the information we need. This means that we don't create the native source buffers until the PMT is parsed from the main media. Even if the content is demuxed, we only need to parse the main media, since, for now, we don't rely on codec information from the segment itself, and instead use the manifest-provided codec info, or default codecs. While we could create the source buffers earlier if the codec information is provided in the manifest, delaying provides a simpler, single, code path, and more opportunity for us to be flexible with how much codec info is provided by the attribute. While the HLS specification requires this information, other formats may not, and we have seen content that plays fine but does not adhere to the strict rules of providing all necessary codec information.
+
+### Appending Init Segments
+
+Previously, init segments were handled by videojs-contrib-media-sources for TS segments and segment-loader for FMP4 segments.
+
+videojs-contrib-media-sources and TS:
+* video segments
+  * append the video init segment returned from the transmuxer with every segment
+* audio segments
+  * append the audio init segment returned from the transmuxer only in the following cases:
+    * first append
+    * after timestampOffset is set
+    * audio track events: change/addtrack/removetrack
+    * 'mediachange' event
+
+segment-loader and FMP4:
+* if segment.map is set:
+  * save (cache) the init segment after the request finished
+  * append the init segment directly to the source buffer if the segment loader's activeInitSegmentId doesn't match the segment.map generated init segment ID
+
+With the transmux before append and LHLS changes, we only append video init segments on changes as well. This is more important with LHLS, as prepending an init segment before every frame of video would be wasteful.
+
+### Test Changes
+
+Some tests were removed because they were no longer relevant after the change to creating source buffers later. For instance, `waits for both main and audio loaders to finish before calling endOfStream if main loader starting media is unknown` no longer can be tested by waiting for an audio loader response and checking for end of stream, as the test will time out since MasterPlaylistController will wait for track info from the main loader before the source buffers are created. That condition is checked elsewhere.