How to capture audio/video streams

[Fred]

With quite a few concerts on the 2013 tour either being streamed or likely to be streamed, I thought we could gather some useful information on how exactly to capture an audio/video stream without having to resort to "primitive" methods, like screen recording or recording audio via your sound card's Stereo Mix input. So, anyone who has experience in this field want to kick off proceedings? This rookie could use a lesson.

(I realise this thread is in the completely wrong place, but it's three days to Lolla and my assessment is that it'd be a good idea to kick off this thread somewhere where people can actually see it and might be looking for it, just so we don't run into a "shit, I hope someone's recording this"-situation again.)

[Erik]

I cannot say that this still works, but back in April, here was the method a lot of folks used for Coachella.

http://stefansundin.com/blog/452

[ninjaw]

I cannot say that this still works, but back in April, here was the method a lot of folks used for Coachella.

This produced the glitchy one ? the only working one was made with HiDownload Platinum, you can still read this info here. There was only one good rip.

The most known live lossless capture tools are TubeMaster++ and Replay Media Catcher.
The most known non-live youtube capture tool would be JDownloader
I'm still love using the french tool captvty it got a very powerful and updated misc link downloader (this is a tv tool) this tool already know lot's of video links, and it helped me one time when TubeMaster++ was fooled refusing to download revealed links.

After capturing you need to change format, mux, cut, tag (but never convert)
mp3directcut is the most know tool it can cut MP2, MP3, and MP4 (require plugins) without loss, you can find mp2 in all mpeg2 videos, and MP4 in all youtube vids, a mp4 is like a video but with only an AAC file inside. This tool is also known to cut mp3+fucking CUE files
To lossless cut full hd videos, i've been amazed by SmartCutter, even better than Tmpgenc MPEG Smart Renderer 4
MkvToolNix's MKVmerge is very powerful to (de)mux MKV
YAMB can help you (de)mux MP4

[DigitalChaos]

paging @TheBang and probably a shitload of others that I can't think of.

Thanks for getting this thread started!

[DigitalChaos]

It's probably best if there is a diversity of tools used across people when capturing. Tool used, geographic location, etc all matter.
We will want to mention the use of TunnelBear and other proxy tools when it comes to dealing with region locked streams.

[Leviathant]

rtmpdump is my friend, and could be yours too, depending on how you wield your google-fu.

[cynicmuse]

This produced the glitchy one ? the only working one was made with HiDownload Platinum, you can still read this info here. There was only one good rip.

That was the second method that I tried after I had some glitches in the live rip. That method didn't work at all... there were visible and audible glitches at every join of the files. Both the audio and the video were variable frame rate. If you had a continuous capture, it wasn't an issue. Otherwise, no dice.
Edit: I'm a video novice (and I was doing all of the editing on a linux machine, which means extra fun). At this point, I don't remember if I saved the files as .ts or as .mp4. I know that weird things happened when I tried to convert from one file format to another.

I used VLC to save the glitchy stream; I think that the glitches happened because I was running two copies of the 1080p stream and I ran into buffering issues. Next time I'll run the monitoring stream on a different machine. I hadn't had any glitches in the earlier recordings when I hadn't been running two copies. I'll give rtmpdump a shot and see what happens.

[ninjaw]

That was the second method that I tried after I had some glitches in the live rip. That method didn't work at all... there were visible and audible glitches at every join of the files. Both the audio and the video were variable frame rate. If you had a continuous capture, it wasn't an issue. Otherwise, no dice.
Edit: I'm a video novice (and I was doing all of the editing on a linux machine, which means extra fun). At this point, I don't remember if I saved the files as .ts or as .mp4. I know that weird things happened when I tried to convert from one file format to another.

I used VLC to save the glitchy stream; I think that the glitches happened because I was running two copies of the 1080p stream and I ran into buffering issues. Next time I'll run the monitoring stream on a different machine. I hadn't had any glitches in the earlier recordings when I hadn't been running two copies. I'll give rtmpdump a shot and see what happens.

I read some details on the hidownload thing, i even tried myself on the stream, it worked ! the thing was saving numberous files and i guess ppl were having buffer issues past 1h, the guy who succeded was given a list of revealed urls from another tool.
Please remember that stream was using an unknown new youtube protection at the time, since then, i saw same thing on regular vids, and some classsic tools already updated.

[cynicmuse]

I read some details on the hidownload thing, i even tried myself on the stream, it worked ! the thing was saving numberous files and i guess ppl were having buffer issues past 1h, the guy who succeded was given a list of revealed urls from another tool.
Please remember that stream was using an unknown new youtube protection at the time, since then, i saw same thing on regular vids, and some classsic tools already updated.

Thanks for the info on what hidownload saved. I think that I goofed when I saved the files with vlc. I should have just used mplayer with the dumpstream option; the dumpstream option allows mplayer to just dump the stream with no re-encoding or re-encapsulation while vlc requires re-encapsulation when saving a stream.

As far as the protection on the stream goes, it was relatively easy to determine the url of the playlist (m3u8 file). I used the web console of my browser to track network requests as I switched the quality of the stream and got it that way.

[TheBang]

There are two types of Flash streaming: RTMP and HTTP Live Streaming

RTMP is the traditional Flash streaming protocol which was developed by Adobe. This is what you'd use rtmpdump or Replay Media Catch (which uses rtmpdump) to capture. Frankly, for YouTube, I haven't seen any RTMP streams since last year. YouTube appears to have transitioned entirely to HTTP Live Streaming. If you're watching a Flash stream from someone besides YouTube then it might still be RTMP.

HTTP Live Streaming (HLS) is a new(er) streaming protocol developed by Apple, initially for iOS streaming, but ultimately replacing the traditional QuickTime RTSP streaming (Darwin Streaming Server, etc.). This is the one that uses the .m3u8 files, and chunks the video stream as discreet HTTP downloads. According to the specification, the encapsulation is MPEG-2 Transport Stream, so .ts is probably the correct encapsulation to choose if you're using VLC to capture the stream.

I'm actually not sure of the best way to capture HTTP Live Streaming streams. I failed on both the HTDA and Fuij Rock streams (which were both HLS). The nice thing about HLS is that if you can get the correct m3u8, I think you can kind of start and stop anywhere you want in the stream, since it's just chunked HTTP requests. This isn't really possible with live RTMP and RTSP streams.

I'll have to check out HiDownload Platinum. ffmpeg and VLC are also supposed to be able to do it. I had a hard time getting ffmpeg to work on the HDTA stream though. If anyone has any tips, please contribute.

Fortunately, there's plenty of time to practice on the streams before NIN takes the stage tomorrow!

[TheBang]

The Lollapalooza stream is live. Start practicing your captures!

[ninjaw]

I confirm Hidownload working : launch youtube, start sniffer, go to option, add .ts to the download automatically list, here you go ! it will record a long list of 5sec chunks.
After that ? rename them in TS, i use total commander for that, join them together, i use MKVmerge it will produce a final MKV file

[[parasite]]

any mac recommendations?

[ecirb]

If you download all the files to a folder with nothing else in it, you can open notepad and copy and paste the following into it:

copy /b * merged.ts

Save that as a .bat file in the folder you are downloading to. Go to the folder and double click the .bat file. It will take all the pieces and merge them into one file named merged.ts. Obviously you can replace merged with whatever you want to name the file.

[cynicmuse]

The stream only works live, unlike the Coachella stream where you could scan through it. The m3u8 file only has a couple of chunks in it instead of the whole stream so the Coachella method won't work (unless you'd like to download about a hundered m3u8 files and then a few hundred more ts files). I've gotten chunks with vlc somewhat successfully because it will regenerate the m3u8 file; I'll have to try the vlc method on the faster machine to see if it's a viable method. Otherwise, I'll boot up the windows partition and use hidownload.

[ninjaw]

Hidowloading keeps bugging about memory as reported before, however, Replay Media Catcher is working and is far easier: it does record automaticaly chunks and even the whole video at the same time, however he's naming them mp3 and mpeg, they are obviously TS files, I recommand removing the annoying capture sound, and the youtube renaming, so you can keep those numerical type files, maybe we can even exchange missing one ?

[Skeptikk]

So I have no idea what I've done but I think Ive got Replay Media Catcher recording so I'm not going to touch the laptop until NIN's over then cut out all this Killers stuff hahaha.

My screen has 20 odd thumbnails some failed and complete and then one is counting upwards of 300,000% downloading. No I have no idea what the other ones are but this one with the mega % is actually recording and playing me back video.

[finckscl]

Errr. hi
I downloaded ReplayMediaCatcher from torrent, followed all instructions, and no luck.
So, as Skeptikk I won't touch anything until NIN's over.

PS: FUCK The Killers.

[TheBang]

For Replay Media Catcher 5, I think what you have to do is go to Internet > HTTP > Content Type / File Extensions, and Delete the first 5 or so entries you see for File Extension "mp3". This will keep it from initiating all those extraneous MP3 downloads, and after that you should only see a single mpg download, which should hopefully be the stream.

For HiDownload, you want to go to System > Options > Sniffer, and add ".ts" to end of the long list of file extensions. You probably also want to go to General and check "Disable balloon notifications" or else it will get annoying. You should then see a bunch of numbered files being downloaded in sequence. For a 1080p stream, each chunk would be about 2-3 MB. At the end of it, you'll have to stitch all the chunks together. That's the part I'm not sure about yet.

[gstead]

I'm on my Mac using ffmpeg on the command line.

export URL720p="<get_url_from_firebug>"
ffmpeg -re -i $URL720p -codec copy -y "lollapalooza.m3u8"

This creates the 5-second .ts files mentioned above. I have a script to 'cat' those together and voila! Super nice video.

Caveat: I've never done this before but it seems to work.

[cynicmuse]

I'm on my Mac using ffmpeg on the command line.

This creates the 5-second .ts files mentioned above. I have a script to 'cat' those together and voila! Super nice video.

Caveat: I've never done this before but it seems to work.

Good to know. I got annoyed enough to boot up the windows partition and have Replay Media Catcher working there. I hadn't thought about using firebug to grab the m3u8 files.

[finckscl]

I downloaded ReplayMediaCatcher 5 (the previous was the 4), followed yr instructions. Made some tests, went good and freed 10 GB on HDD.
Let's see what happens!

[ItsJustDave]

I was trying to capture at 1080p but I've dropped to 720p in the hopes that will work out better. I'm using hidownload.

Here we go!

[finckscl]

I went straight for the 1080p stream. Some glitches at youtube, I hope they won't reflect on the final output.

PS: It's just me or this Sanctified is way more evil than the original?

[ItsJustDave]

I was trying to capture at 1080p but I've dropped to 720p in the hopes that will work out better. I'm using hidownload.

Here we go!

hidownload out of memory. Continuing to queue, but stopped downloading. Any tips?

[finckscl]

Close any window/tab/program unneeded at this time.
Hint: Using fb reduces free memory.

[cynicmuse]

I'm seeing a few glitches on the live YouTube feed as well with the 1080 stream. I'm waiting until the stream is done and then I'll watch it in full screen and see what it looks like.

[TheBang]

hidownload out of memory. Continuing to queue, but stopped downloading. Any tips?

I ended up quitting and relaunching it. Got a gap in my files. I'll have to check next time it runs out of memory and see if it's a system memory problem? I highly doubt it, as i have 16GB of RAM. So I suspect it's just a HiDownload issue.

[ItsJustDave]

Close any window/tab/program unneeded at this time.
Hint: Using fb reduces free memory.

I understand how memory consumption works, hidownload indicates out of memory when I'm only using 31% of memory. Just wondering if I can keep things in queue for later download (after I close and reopen)?

Screen Shot 2013-08-02 at 11.37.58 PM.jpg

[TheBang]

I'm missing the following 1080p chunks from tonight's broadcast. If anyone has them, please PM me:

8541-8570
8858-8862
9156-9160
9463-9488