Unmasking the secrets of video on the Web.
Have you ever wondered how it all works? How your videos get from your device to the folks who watch those videos on their devices anywhere in the world? If you have, you’re not alone. As the 2008 founder of iPlayerHD, I am often asked how it all works. In this blog post, I am going to unmask the secrets and reveal to you how it’s done, at least how it is generally done by companies like iPlayerHD.
Of course I’m not going to reveal the most important stuff anymore than a magician is going to tell you how to levitate a beautiful woman, though I suspect that secret must be out there on the web somewhere. In fact, that secret is out and if you read on, I’ll reveal it to you.
Years ago, when companies like iPlayerHD began delivering video and competing with the likes of YouTube, there was a reluctance to speak about the technologies used to deliver video from the farm to the table. In 2017, companies like Amazon Web Services (AWS), which wisely leveraged the technologies they use to deliver a whopping 43% of all online retail sales into valuable services for sale to anyone – services like servers, content delivery networks (CDNs), database utilities, transcoding services, storage, and so many more – it’s really no longer a mystery how much of the internet works. So there’s little I can tell you here you could not find digging through the web. Just the same, I’ll save you the time and hand it off to you in one easy-to-digest blog post.
In the beginning there were three.
Getting video on the web in the early years was nothing more than buying or renting a server, connecting it to the internet, and sending video files to anyone who clicked play on a Windows Media, QuickTime or Flash player. Delivery to those players was no more complicated then sending web pages. It was just data, lots of data. No features. No social sharing. Just video. You may recall seeing websites with three players in the first decade of this century (Is it really 2017?). Today we have (mostly) HTML5 which is a player-less player built into every browser that never needs to be managed or updated. Just about all of the video you watch today on the Web is HTML5. The one notable holdout is Facebook which still delivers via Flash. I’ve no reasonable answer to the why on that.
Up to the Cloud.
The first step to getting your video on the web, after you procure it of course, is to upload it to a video hosting service that you have subscribed to. You’ve entered the Cloud. You pay them a monthly fee to ingest, store, manage, publish and accumulate important data about the folks who watch your videos. Once your video is uploaded, that file is moved very quickly to a server with one purpose – to transcode your file into several files, all of which are optimized for the web and all the devices with access to the web. In the early years, there was a lot of heavy lifting involved in just this first step. Today, technology is relatively flat and much of the heavy lifting of transcoding is gone, especially with readily available services like Amazon’s Elastic Transcoder or Encoding.com.
Once your video has been transcoded, the next step is to move the newborn files to the crib to be stored and where they will be retrieved by the CDN that will deliver them to your audience. Amazon calls that crib a “bucket” and the bucket lives in a place they call Amazon Simple Storage Service, more commonly known as S3. Google’s answer to S3 is Google Cloud Storage. There are many other companies in the same space.
Until you delete the file, it will rest comfortably in storage while it waits to be called into action by your content delivery network (CDN – last time I promise). There are many CDN’s to choose from with some of the most popular being Akamai, EdgeCast, Limelight, and of course Amazon’s Cloudfront. Each of these providers deliver your videos anywhere in the world you choose to make your content available.
First class delivery is just a tap or click away.
Let’s suppose you upload a video to your provider and the associated files reside in Amazon’s East database center in Virginia. We call that the “origin”. Let’s also suppose someone from Ottumwa, Iowa (yes M*A*S*H fans, there really is such a place) sees your video on your web page and clicks or taps play. Finally, let’s assume this is the first time anyone from that part of the United States has played the video.
Your CDN will grab the file from your origin and faster than you can read this sentence, moves the entire file to a server that’s probably as close as 200 miles from the viewing machine in Ottumwa. That server’s location is called the “edge” and it exists because it is believed the file will play faster the closer it is to the location that is viewing it. While that is generally true, I’ve done extensive testing and have found that the latency (the time it takes from hitting play to the time the video begins playing) is often not noticeable when played from thousands of miles away. The pipes that feed the web are fat and getting fatter. Having said that, any video host provider worth working with, iPlayerHD included, uses a CDN with a robust network of global edge locations, many not more than 200 miles at most from the machine playing the content.
It takes a good CDN to deliver content, particularly if your goal is to eliminate any chance of buffering, at least from the server side. Fortunately, here also the technology is flat and buffering is generally not a problem, at least not from the server side. And these providers are very reliable with outages becoming very rare. Most buffering is a result of the connection between your router and your machine playing the video. If the bandwidth – that’s the amount of data allowed at any given time by your ISP (Internet Service Provider) – is too low and the bitrate of the video is too high, buffering can occur. This also is becoming more rare. In the early years, iPlayerHD’s number one support call was to explain buffering. I can’t remember the last time we had this type of support request. Those pipes are fat also.
By the way, your bandwidth is controlled by your router which is controlled by your ISP. If your ISP increases your bandwidth – we typically call it “speed” – their computer simply sends a message to your router and the change is made.
If you have a truly global audience, it’s entirely possibly your video files are stored on many edge locations. So long as the content is popular in a particular edge location area, it will remain there. If it goes untouched for a certain period of time in the area near a particular edge location, it will be removed by the CDN. However, no matter how popular your content is or not, it will remain on the origin until you delete it. Of course you are paying for that storage. You are also paying for delivery which is generally charged by the GB. So is storage. It’s really all very simple.
What about all of them features?
Hosting providers like iPlayerHD offer their users all sort of features. Typically, we add the features that make the most sense to the most people. Our coders, that is, our development team, writes the code that puts together all of the features and assembles them so that they work smoothly on all of the many pieces of the puzzle presented above. And there are many other services and technologies that make up that puzzle but presenting them here would be like singing you a bedtime lullaby. So let’s not do that. Suffice it to say, there are millions of lines of code supporting the videos you deliver on your website. And with minor exception, it all works flawlessly. Until it doesn’t. Just kidding of course.
So now you know the secrets behind all those videos you are watching. It’s really no secret. The true secret to being an outstanding video host provider, like any other business, is delivering outstanding service. That’s where the magic lives.
Oh, and the levitation thing? The secret is out. Actually, I am disappointed. I’ve always believed she was actually levitating. And that David Copperfield really did make the Statue of Liberty disappear. Apparently not.
Have a great day!