Latest Developments in TV Monitoring Software and Technology on SnapStream's TV Searcher Blog

Archive for April, 2010

NAB Afterglow

Thursday, April 29th, 2010

The team jetted back from Vegas at the end of week before last, in a total whirlwind. We had a real successful showing at the
National Association of Broadcasters conference. At our booth, we had the 50-channel DVR on display and ramped up for active demos and got to meet tons of interesting people in the entertainment industry.

We were in South Upper Hall, a lively, active spot. Passerby came over to enter our DVR contest, which is still going on until the end of this week (April 30), if you haven’t entered. But man, did we get some funny names and reactions at the booth!

The Broadcast Education Association had a co-located conference that Thursday (April 15) where our CEO Rakesh Agrawal participated in a panel along with other news assignment editors and journalism professors. The panel explored adapting news video for the Web.

For those of you who couldn’t make it and requested the rebroadcast, we’ve clipped out Rakesh’s portion for you to watch. (It’s on our YouTube channel.)

All in all, definitely a great show. This was SnapStream‘s fourth time, my third, and I know we’ll be back to kick it again next year.

TV search 101: searching traditional TV with SnapStream

Wednesday, April 14th, 2010

Several of us from the SnapStream team are in Las Vegas right now at the National Association of Broadcaster’s event (we’re in the south upper hall #SU2707– if you’re here, stop by and say hi). One of the questions we get a lot of is, simply, “What is SnapStream?”

It’s probably fair to assume that many of you here on the blog share this question… so let me answer that question here.

So, what is SnapStream?

You probably search the web every day with search engines like Google, Bing and Yahoo. SnapStream delivers a similar search experience for traditional TV shows — you know, the stuff you get over-the-air, cable, or satellite. That’s to say, SnapStream makes it possible to search TV shows from ABC, CBS, NBC, CNN, ESPN, Comedy Central, CSPAN, or anything on network television.

Here’s an example of the kind of search results that we deliver for traditional television:

How does it work?

SnapStream’s TV search technology uses closed-captioning (that’s included in almost all TV content in the U.S. and Canada, here in the U.S. it’s mandated by the FCC) and program guide data to power it’s TV search engine.

Our customers purchase a turn-key TV search appliance (a piece of hardware) from us. It’s delivered pre-configured, making it easy to setup and install. This SnapStream TV search appliance can record up to 10 channels at a time and then our software, running on this appliance, indexes those recordings and makes them searchable.

Here’s a video demo’ing our TV search interface:

So there are the basics. Have any other questions? Post them in the comments and I’ll answer them!

Name the world’s largest DVR. Enter to win a free Apple iPad!

Saturday, April 10th, 2010

Hear ye, Hear ye. The contest is officially open. Let the name game commence! After you’ve read up on all of the technical specs of the 50-channel, world’s largest DVR–or maybe just gotten the gist of it–rack your brain for what we could name this crazy thingamajig. Engadget is calling it a “Monster DVR.” But can you come up with something more awesome? The best entry will win a free Apple iPad + case.  Click here to enter.

Introducing… the world’s largest DVR (& TV search engine)

Thursday, April 8th, 2010

(First time visitors: Welcome to SnapStream! We make TV search software used by organizations like the U.S. Senate and The Soup (on E!) to search inside TV shows. And it’s not too expensive, starting at $2,000. What’s TV search, you ask? Watch this YouTube video or visit our website to learn more.)

50 Screens With DVR Server

If you’ve read any of our previous articles on our monster DVR systems, you know we have a knack for throwing together skunkworks DVRs that can record a lot of TV. In the past, these have been done in the wee hours of the morning with parts we’ve scavenged from other systems.

50 Channel Recording Guide

Yup! This is what it looks like when you record 50 channels at once… (click to see it up close)

But… what would happen if we set out to make the World’s Largest DVR? And not from scrap parts, but with shiny new hardware and a real (gasp) budget?  And while we’re at it, let’s not just go from 12 tuners to 13. Let’s blow the doors off of television recording. Putting on my infomercial voice here, we’re not doing 10, not 20, not 35, but 50…. Yes, FIFTY (50) TV channels at once. :-) And while we’re at it, let’s make a storage system to sit behind it that lets you keep all of those shows. Forever. Well, forever as long as you’re willing to buy the drives to plug into the system. In total, the storage capacity is equivalent to 326 TiVo Premier boxes. But that might take up a little more space than our rack has, though.

Well, for the past little while we’ve been working on adding clustering capabilities to our TV recording and search software… technology that would enable us to go beyond the 10 TV tuners that we could put on a single PC motherboard.  After planting the seeds for this new architecture last year, we’ve been working hard on getting this new version of SnapStream completed.

Well, spring has sprung and that seed sprouted into a 25U half rack packed to the gills with the most monster of DVRs ever created.

And we can’t wait to introduce it at the National Association of Broadcasters (NAB) Show in Las Vegas next week (April 12-15).  If you’ll be there, come check it out in person! (we’ll be in the South Upper Hall in booth #2707)

Open Door 50 Channel TV Search Engine

The finished product… our 50 channel DVR + TV search engine. The largest in the world.

The juicy details:

5-Server Cluster

World’s largest DVR in the rack

TV Source: You plug it in with one simple coax cable feed from your local cable company.

In that regard, it’s pretty normal. Of course, do you know of other DVRs that need two 15amp power plugs and a 10GB Ethernet (or up to 8 bonded 1GB ports) uplink? :-)

Tuners and Storage: It has a grand total of 50 analog TV tuners recording to 125+TB of storage. (10 tuners and 18TB of storage per 3U chassis or 10 tuners and 30TB storage of 4U chassis).  As a demo, this system shows both of our 3U and 4U chassis sizes. Depending on the storage requirements, one could do all 4U or all 3U boxes.

Note: Due to the higher bitrates, if you were to do this with digital recording sources like QAM or ATSC, each server would only do 5 shows per box. You’d just need more boxes to get to 50 channels.

Clustering: The magic behind building a TV search engine like this one is our new clustering technology.  With this setup, one SnapStream node is designated as the master and other SnapStream nodes which can then be designated to performed a narrow or broad set of functions (see diagram at the left).  Then the “master” doles out work to be done to the various nodes it has at its disposal and balancing resources appropriately.  For example, when using the SnapStream Link software to playback shows, each Link Client will request (under the hood, of course) the video in question directly from the server that houses the video file to prevent any duplicate network traffic.

Networking: Because this is a “distributed” DVR, the networking between the nodes is important — this network meshes the nodes together, allowing them to talk to one another, use each others storage, etc.  The interconnect on this SnapStream Cluster is via an HP Procurve 2910 switch, with two 1GB copper Ethernet ports bonded via LACP per server.


World’s largest DVR from behind

Storage allocation: Using drive pooling, all of the storage shows up as one giant video folder… all 102TB of it!  As a result of this, each system can record to either itself or other machines on the network as space allows.  Our software handles all the magic of distributing recordings across the various logical storage volumes.

TV Search: And, of course, this giant SnapStream DVR cluster can not only record shows, but it makes them all searchable as well, using our powerful TV search technology.  This is the same TV search technology that shows like The Soup (on E!) use to find their TV clips. With our clustering technology, there’s one unified search interface all the recordings. With the storage capacity on this SnapStream, this means the ability to search about 115,200 hours of recorded TV… or about 13 years worth of TV recordings!

And then it gets even cooler….

Distributed Transcoding: For the uninitiated, ShowSqueeze is our built-in transcoding feature — the feature that allows you to take recordings that are natively made in MPEG-2 and convert them automatically to H.264 or Windows Media.  In the older architecture, each server could only ShowSqueeze up to the limit of the local CPU.

Anyhow, being limited to the CPU on one computer won’t do for such a massive system, so in addition to scalable recordings and storage across nodes, we added scalable ShowSqueeze across nodes as well.

The base unit is a 1U dual-processor quad-core Nehalem Xeon server (8 physical, 8 logical cores) with a ton of ram.  On the price to performance curve in a 1U space, this box is theperfect choice.  It can do more than one ShowSqueeze from HD to HD h.264 in real time or can take eight analog shows at once and squeeze to an archival quality file.  Under the hood, we have all of the software knobs to turn and buttons to push so you can optimize the performance of the box to the task at hand.

Assuming one box isn’t enough, one could just add more SnapStream ShowSqueeze nodes.  As additional ShowSqueeze boxes come online, the master SnapStream node will dole out any available tasks to them.  And if a ShowSqueeze node is taken back offline and the master node automatically adjusts by moving ShowSqueeze tasks to the remaining nodes!

HD-SDI playout: To finish out our rack, we also threw in one of our HD-SDI playout nodes.  (Once again, one can add as many as needed for your workflow.)  For our traditional television customers like NBC, The Soup (E!), Current TV and MTV, being able to get SnapStream recorded shows to their HD-SDI video router is important.

Stats Quick View (Maximum Configuration):

Xeon Processors 8
2tb Hard Drives 68
Raw Space 136TB
Raid Storage 102TB
Total Storage 24×7 Recording* 115,200 hours = ~13 Years
Storage Per Tuner 96 Days

*Assuming a recording quality of 2mbit/s.

Help us name the World’s Largest DVR… and win an iPad:

We need a really cool name for this thing and we want your help!  And the winning submission will get an Apple iPad – 16GB + Apple’s iPad case/cover. Enter the contest here from April 9 to April 30.

And, as mentioned above, you can come see this thing live at the National Association of Broadcaster’s (NAB 2010) show in Las Vegas.  We’ll be showcasing it in the South Upper Hall, #SU2707.

And, finally, some more pics:

(see them all on Flickr here)

50 Channel DVR-search engine
Internal Hardware
Internal Hardware
Inserting tuner cards
2 Servers in the Making
Gearing up the DVR with memory and tuners
Lifting the Nodes in Place
Building the DVR Rack
5-Server Cluster
50 Channel TV Search Engine

How journalism schools and TV shows use SnapStream (by Columbia Journalism Review)

Wednesday, April 7th, 2010

Alexandra Fenwick at the Columbia Journalism Review interviewed me on a recent trip to New York City. That interview resulted in this piece about SnapStream and how folks are using it in academia and in entertainment. Check it out!

Is Apple’s iPad the supernova of tech-launch publicity?

Saturday, April 3rd, 2010

We took it to our TV Trends analytics to investigate the uptick of buzz surrounding the iPad, which hits the market tomorrow. The forecast calls for an explosive spike in the media frenzy as users get their hands on experimenting with the highly anticipated Apple tablet. This Wednesday’s episode of Modern Family indicates that it’s going to be a full-on mob scene. Brilliant stroke of product placement, I might add.

Let’s conduct a side-by-side comparison next to other major tech product launches from recent history to see where the iPad stands. If I’ve missed any big debuts, you can plug in keywords yourself on the TV Trends database.

Tech Product Launches in Review

April 2, 2010 – Day before the iPad launches. Media index currently at 146.

January 27, 2010 – Coming of the iPad is announced. Media index shoots to 289.

January 5, 2010 – Google’s Nexus One Phone premieres. Peak of 180.

October 22, 2009 – Launch party for Windows 7. Interest caps at 155.

November 11, 2008 – Google launches Gmail video chat. Reaching 147.

October 22, 2008 – Google’s first Android phone, HTC Dream is released. Index is 77.

Based on our data history, the iPad is positioned to be the overwhelming winner in garnering extensive interest from broadcast media. We will continue to monitor how the world reacts to the iPad post-release.

Click the graph to manipulate the search and view specific channel sources.

Before we started tracking TV trends in October 2008, there are some prime dates from recent memory that would be cool to look at as a frame of reference. These hallmark technology events generate contagious excitement in the media.

July 11, 2008 – Nationwide launch of Apple’s iPhone 3G.

November 19, 2007 – Amazon Kindle launches.

June 29, 2007 – iPhone is introduced into the United States.

January 9, 2007 – Apple’s first iPhone model is announced.