Tech Tips: Apache, Nginx, JWPlayer, Streaming, and the Case of the Missing Bytes
By Jordan Goldman on August 23rd, 2012
Share on Google+Share on LinkedInShare on RedditTweet about this on TwitterShare on Facebook

We ran into an interesting problem this morning with streaming on iDevices (verifiable on the iPhone and iPad). The problem was that iDevices wouldn’t play/stream videos when the request was sent to Nginx, but it would begin working when sent directly to Apache. Using tcpdump/WireShark, we were able to troubleshoot what was happening: these devices specify a Range header in the HTTP request that they send. RFC permits a response code of 206 or 200. 206 is “partial content” and 200 is “OK”. Nginx serves a 200 response with the content of the entire file, whereas Apache was responding with a 206. After the 206 request was received, the iDevice would issue another request asking for the entire file.

Setup: Nginx cache, Apache webserver, JW Player to stream/support various clients

Here’s the difference in requests:

GET video.mp4 HTTP/1.1
Host: localhost:82
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPhone; U; CPU OS 5_1_1 like Mac OS X; en_us)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: E75B70F5-1EC5-4BE9-84F8-D9D2B44A77DF
Cookie: PHPSESSID=aiq0kouocsjlhdd01ua627enj5; __utma=163342885.1648847639.1345699273.1345699273.1345702806.2; __utmb=163342885.9.10.1345702806; __utmc=163342885; __utmz=163342885.1345699273.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); cookie_version=2.2.2; user=06_ORt1j3d5cLuLxZD4-2U2flvsC5FvKLouw4bOvr3Yn8tggq_pei4kXX8zIqpkuLSezD2gpeNZztL1FOBeifg
Connection: keep-alive

HTTP/1.1 206 Partial Content
Date: Thu, 23 Aug 2012 06:31:06 GMT
Server: Apache/2.2.15 (CentOS)
X-Powered-By: PHP/5.3.16
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: private
Pragma: no-cache
Accept-Ranges: 0-21558241
Content-Range: bytes 0-1/21558241
Content-Length: 2
Keep-Alive: timeout=15
Connection: Keep-Alive
Content-Type: video/mp4

… and then:

GET video.mp4 HTTP/1.1
Host: localhost:82
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPhone; U; CPU OS 5_1_1 like Mac OS X; en_us)
Accept: */*
Range: bytes=0-21558240
Accept-Encoding: identity
X-Playback-Session-Id: E75B70F5-1EC5-4BE9-84F8-D9D2B44A77DF
Cookie: PHPSESSID=aiq0kouocsjlhdd01ua627enj5; __utma=163342885.1648847639.1345699273.1345699273.1345702806.2; __utmb=163342885.9.10.1345702806; __utmc=163342885; __utmz=163342885.1345699273.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); cookie_version=2.2.2; user=06_ORt1j3d5cLuLxZD4-2U2flvsC5FvKLouw4bOvr3Yn8tggq_pei4kXX8zIqpkuLSezD2gpeNZztL1FOBeifg
Connection: keep-alive
HTTP/1.1 206 Partial Content
Date: Thu, 23 Aug 2012 06:31:07 GMT
Server: Apache/2.2.15 (CentOS)
X-Powered-By: PHP/5.3.16
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: private
Pragma: no-cache
Accept-Ranges: 0-21558241
Content-Range: bytes 0-21558240/21558241
Content-Length: 21558241
Keep-Alive: timeout=15
Connection: Keep-Alive
Content-Type: video/mp4

Whereas this is what the Nginx stream looked like:

GET video.mp4 HTTP/1.1
Host: localhost:8882
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPhone; U; CPU OS 5_1_1 like Mac OS X; en_us)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: E8F97D61-E0D5-476D-8DC9-E4F76FCC3FBB
Cookie: PHPSESSID=aiq0kouocsjlhdd01ua627enj5; __utma=163342885.1648847639.1345699273.1345699273.1345702806.2; __utmb=163342885.8.10.1345702806; __utmc=163342885; __utmz=163342885.1345699273.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); cookie_version=2.2.2; user=06_ORt1j3d5cLuLxZD4-2U2flvsC5FvKLouw4bOvr3Yn8tggq_pei4kXX8zIqpkuLSezD2gpeNZztL1FOBeifg
Connection: keep-alive
HTTP/1.1 200 OK
Server: nginx/1.0.15
Date: Thu, 23 Aug 2012 06:25:19 GMT
Content-Type: video/mp4
Connection: keep-alive
X-Powered-By: PHP/5.3.16
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: private
Pragma: no-cache
Content-Length: 21558241
X-Cached: EXPIRED

The solution in this case was to have HTTP requests with the Range header go straight to Apache, rather than having Nginx cache/handle them.

proxy_set_header Range $http_range;
proxy_set_header If-Range $http_if_range;
proxy_cache_bypass $http_range $http_if_range;