Transcoding MTS/M2TS AVCHD Video Into AVI Files with Free Software
Linux January 3rd, 2008
This holiday season, I was an extremely lucky recipient of a Sony HDR-SR5 video camera with a whopping 40GB hard drive inside.
When I opened it, the first question in my head was not atypical of a Linux users’ train of thought:
“Is it compatible with Linux?”
The answer was a tiny bit more complicated than I imagined, and immersed me into the otherwise unknown world of video transcoding, which is something millions of people do every day but probably never think about it. The best example is on YouTube, where it happens behind the scenes in the Google computing cloud, and everything “just works.”
I’ve read a few things about the format the camera uses, called AVCHD. Even in Windows (or OSX for that matter) many people have headaches with the AVCHD format, so I figure that Linux users are probably going to be having a helluva time trying to get it to work.
The files the camera generates are compressed at an incredible 15:1 ratio. I’ve seen it mentioned that a Standard Definition compressed clip of is actually larger in file size than an HD clip in AVCHD format.
The compression is where a majority of the AVCHD format complaints come originate from, since there is not much out there in terms of being able to edit video stored in the format without uncompressing it first. It takes some rather respectable computing horsepower to actually convert the files into an uncompressed format that can be played with in a video editing software package. Using the method detailed here, it takes about 5-6 minutes to uncompress and transcode a minute of video, on an Intel Core2Duo running at 3.2GHz with 4GB of RAM and a 10,000RPM hard drive - OUCH!
When plugging in the Sony HDR-SR5, it functions as a USB Mass Storage Device. Copying files from the camera to the computer is a simple drag n’ drop affair. This would be a familiar process to anyone who has used a digital camera and imports pictures to their computer.
It would certainly be faster to playback the videos in real-time and capture them using the component outputs from the camera… but that wouldn’t be a challenge. I’ve never actually installed the software that came with the Sony Handycam, so I’m not sure what I’m missing out on, but it doesn’t seem like all that much, to be honest. I wouldn’t be surprised if it was the typical Sony software suite that is about as useless as the media it is distributed on.
Let’s get down to the meaty bits, shall we?
The first thing I searched for was “AVCHD Linux” and found a post over at AVS Forum. On a side note, my Google-Fu is really spot on lately.
That post is a gold mine. It literally contains everything we need to transcode the video into formats of our choosing that are usable in non-linear video editing programs such as Avidemux or Cinelerra.
m2tstoavi is the best information I have found (so far!) to uncompress the AVCHD video files on my camcorder and convert them into a .AVI file in Linux.
Installation is very simple, all it requires in Ubuntu is a quick apt-get install:
sudo apt-get install mplayer ffmpeg x264 faad2 faac a52dec mencoder faad libfaad2-0 libfaad2-dev subversion csh
Then edit the install script as noted in the README to grab the newest version of ldecod. Everything else is completely automated - nice job Axel!
I did need to modify the settings a tiny bit to actually get audio synchronized as well as fix a possible issue with the script itself. I had problems until I commented out the echo ffmpeg yadda yadda part. When modified, it runs the following commands:
xporthdmv -hn $file 1 1 1 && mv bits0001.mpa $audiofile
ldecod -i bits0001.mpv -o $videofifo &
ffmpeg -r 29.97 -s 1440x1080 -i $videofifo -i $audiofile -vcodec mpeg4 -sameq -acodec copy -aspect 16:9 -b 15000k
I’m using the m2tstoavi.fifo script that uses named pipes (denoted above with dollar signs) to stream the output of one command into another. It is efficient with disk space and does not fill up the drive as the regular m2tstoavi script does, not sure why the fifo version is not the default. Ah well, nothing is perfect.
Once installed, I just run this command to convert every MTS video in the current directory to .AVI files:
./m2tstoavi.fifo *
It turns this proprietary format mess:
Into precious memories:
All in all, I’m glad I don’t need proprietary software for this conversion. I really wish it were faster - but here’s the point of this exercise:
I never used Windows software, so I don’t know what I’m missing. I’m happy with the solution I have, and if 20 people comment on this post and say they can edit HD video in realtime, good for them.
If they do, then I would have a better reason to upgrade to a quad-core CPU.
Here’s a (VERY unpolished!) video talking about the process in a little bit more detail, and showing the actual encoding process and the finished result. I’m not responsible for anybody tripping out at the end on my extremely crazily bad camera skills in the rendered video.
Oh yeah, and for those of you who were wondering, I also received quite possibly one of the best books I’ve read in a long time… it’s all about how cool Eric S. Raymond is ninjas are.
Tags: AVCHD, FFMPEG, HDR, HDR-SR5, LDCOD, Linux, M2TS, m2tstoavi, MTS, Sony, SR1, SR2, SR3, SR4, XPORTHDMV
DO WANT.
It’s a sexy HD camera if I do say so myself.
DO WANT (the book, not really interested in cameras
)
If you want Ninjas, then study Ninjutsu:
http://www.winjutsu.com/
I find it much more natural in movement, less “stiff” than generic Karate, yet with a much greater range of skills and style. Check out Human Weapon #14 for a glimpse.
On video, wow! 1 to 5 speed ratio on that hardware? I’m not going to be getting a Sony, that’s fore sure.
I was very happy with my Sony mini-DV camera until it broke, it interfaced with Linux seamlessly. Linux runs fine on my Sony laptop, so it’s nothing against Sony.
Pinnacle has a product for Windows that transcodes and renders the timeline in real-time.
So yeah, I am missing out. *shrug*
-Wayne
FYI, please read the comments too:
http://eugenia.gnomefiles.org/2008/01/04/canon-avchd-24p-pulldown-removal/
If you can test for me a few things, I can add Linux in the tutorial. Please let me know if you are available to help out.
looks like you made it to the front page of digg again, via wired…
congratulations!
this is my favorite linux blog
[...] Transcoding MTS/M2TS AVCHD Video Files with Free Software | fsckin w/ linux Tags: none January 6, 2008, at 4:30 — links — BY-NC-SA [...]
Hey thanks for another great post! I am curious if this process could be sped up with the addition of a graphics card that would handle h.264 encode/decode. For instance the 8000 series nvidia cards have pureHD which off loads most of the decode/encode labor.
If you continue to test different methods please update us with your progress
Thanks a bunch! Finally got a couple vids off my new camera thanks to you (and Mr. Olmos & a host of others) with the mighty Gutsy Gibbon of Ubuntu.
cf: That’s great!
Daniel: I’m pretty sure that it is theoretically possible, but I’m not sure that the decoding ability of the video card has been harnessed yet… it would certainly be much faster with a dedicated device.
How happy are you with the video quality? Is it worth getting one of these cameras? Is the video de/re compressed or is it only the container that’s modified?
Jason: Low-light video is noisy. Otherwise I would say it is the most compelling product in its price range next to the Canon models.
It has me seriously considering a 24 or 27″ monitor to be able view the output natively on my computer. It looks terrific on my HDTV.
I think there is no support in linux drivers for decoding h264 video (and of course encoding) with videocard’s hardware, just for decoding mpeg2.
[...] General info [...]
Yeah only mpeg2 hardware decoding in Linux with Nvidia at the moment. The cards can do h.264 in Windows, so it’s a matter of the driver.
Linky: http://forums.nvidia.com/index.php?showtopic=35698
and now to see if we can get them on to a video ipod
http://ubuntuforums.org/showthread.php?t=114946
https://help.ubuntu.com/community/Medibuntu
Hey guys I have a Question.
I am following the directions above as I am trying to convert an MTS file to an avi from my Sony HDR7 on Ubuntu.
I am entering the following commands :
xporthdmv -hn $file 1 1 1 && mv bits0001.mpa $audiofile
ldecod -i bits0001.mpv -o $videofifo &
ffmpeg -r 29.97 -s 1440×1080 -i $videofifo -i $audiofile -vcodec mpeg4 -sameq -acodec copy -aspect 16:9 -b 15000k
But I have NO luck getting the video synchronized with the audio. The audio keeps lagging behind. Does anyone have any idea on what I am doing wrong?
Also, anyone know if there is any tool to edit HD video directly? Is there any work being done on that?
Thank you!
Yuri
The same as Yuri above, and the author of this article, I am having audio/video sync problems with the resulting .avi file. The tutorial mentions commenting out a “echo ffmpeg” line, but where exactly is this? And I want to mak sure this commenting out is the solution to the sync problem. Thanks.
Yuri / Adam - if you have a PAL camera, this may fix it:
ffmpeg -r 25 -s 1440×1080 -i $videofifo -i $audiofile -vcodec mpeg4 -sameq -acodec copy -aspect 16:9 -b 15000k
The video is being encoded at 25 vs 29.97 frames a second
Thanks for the input, but no that didn’t solve the problem
I don’t think I have a PAL camera, but I’ll try your suggestion in a few minutes.
I’m still having the trouble with the scripts, but I *did* get it working perfectly when entering the commands manually on the command line in some fashion. I did this late, late last night, when half asleep, so I hope I wasn’t dreaming it.
I’ll now try to reproduce it, and detail the steps in my next post in an hour.
An update: I was wrong.
But thankfully wrong in a good way: it works fine with both manual and the fifo script. I have run it many times, changing many options around to make sure I really do have a good configuration.
To note: I installed the latest CVS ffmpeg with the following ./configure flags:
–enable-gpl –enable-pp –enable-pthreads –enable-nonfree –enable-libvorbis –enable-libtheora –enable-liba52 –enable-libgsm –enable-libdc1394 –disable-debug –enable-shared –enable-libxvid –enable-libfaac –enable-libfaad –enable-libfaadbin –enable-libmp3lame –enable-libx264 –enable-x11grab –enable-swscaler –enable-nonfree –prefix=/usr/local
And here are some other stats for ffmpeg’s installation:
libavutil version: 49.6.0
libavcodec version: 51.49.0
libavformat version: 52.5.0
libavdevice version: 52.0.0
built on Jan 28 2008 01:23:48, gcc: 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
And here is the middle section of my m2tstoavi.fifo script (middle because I’ve commented out and changed many parts)
if ( ! -f $audiofile ) then
# echo xporthdmv -hn $file 1 1 1
xporthdmv -hn $file 1 1 1 && mv bits0001.mpa $audiofile
else
echo $audiofile already exists, not creating it.
endif
mkfifo $videofifo
# echo ldecod -i bits0001.mpv -o $videofifo
ldecod -i bits0001.mpv -o $videofifo &
if ( ! -f $outputfile ) then
# echo ffmpeg -r 29.97 -s 1440×1080 -i $videofifo -i $audiofile \
# -vcodec mpeg4 -sameq -b 15000k \
# -acodec libmp3lame -ac 2 -ab 256000 \
# $outputfile
# First pass for statistics
ffmpeg -r 29.97 -s 1440×1080 -i $videofifo -i $audiofile \
-vcodec mpeg4 -aspect 16:9 -sameq -b 15000k \
-acodec libmp3lame -ac 2 -ab 256000 \
-pass 1 -passlogfile $filebase $outputfile || exit 1
# Second pass for better quality… I hope
ffmpeg -r 29.97 -s 1440×1080 -i $videofifo -i $audiofile \
-vcodec mpeg4 -aspect 16:9 -sameq -b 15000k \
-acodec libmp3lame -ac 2 -ab 256000 \
-pass 2 -passlogfile $filebase -y $outputfile || exit 1
else
echo $outputfile exists, not creating it.
endif
I hope this helps someone, and since I am by no means an ffmpeg expert, if someone sees me doing something wrong in any of this, please let me know!
Well, a change: I spoke too soon when I said my above script works. It stalled at the 2nd pass. I’ve changed it by removing the 2nd block of ffmpeg and the “-pass” references. It now works perfectly, verified after half a dozen videos in a row.
Sweet! How long is it taking to encode the videos?
I have a bunch of little videos, each only 1 to 3 minutes long, encoded in 7 mb/s bitrate (half the full High Def bitrate), resulting in MTS files around 35-80 MBs in size. It is taking about 7-14 minutes to process each one, I *estimate*, but note I am doing this on a relatively slow laptop (1GB RAM, Pentium Dual Core 1.8 GHz). I have a server I’m in the process of setting up, and I hope that might be a little better (that’ll be a dual processor Xeon 64 bit with 1GB RAM).
Oh, and the resulting mpeg4 avi file is about 75% of the size of the original MTS file. Is this *reduction* in size normal? I thought the AVCHD formt was highly compressed, and that any re-encoded video would be significantly larger. Am I unwittingly reducing the quality somewhere along this process?
Oh, and another thing (sorry to fill up your feedback forum here), but I had trouble with the ac3 audio encoding. No errors were reported, but neither VLC nor Totem played audio when I would play back the AVI (video was fine). I switched to the libmp3lame (mp3), and it works perfectly.
For me, 3.5GB of HD video in .mts format converts to 14.6GB. So roughly 4x larger after they’re converted. You might be loosing quality.
I’m not much of an ffmpeg expert by any stretch of the imagination, perhaps at this point starting a thread on doom9 forums or even the ffmpeg users mailing list would be highly beneficial for all of us.
Thanks a lot adam! your solution solved my problem completely!
Yuri, glad it worked. Question: what are your file sizes looking like? As I mention above, I’m having an odd result, but I realized this might be because my videos are all “bluescreen”, where most of the background is a stationary solid color, and therefore mpeg4 might be compressing that, resulting in the small file size.
Hey Guys,
The files are working out great, except I noticed that Cinelerra will not recognize any of the AVI files that I create. I have tried multiple codecs such as XVID and VORBIS, but no luck. The LIVES editing program works fine. Any ideas why cinelerra won’t work with these files?
Thanks,
Yuri
I used the steps outlined in the thread and was able to convert MTS to AVI - but I see audio video sync problems.
Audio seems to be out by a few seconds.
I tried to use the “-acodec libmp3lame” but it does not work inspite of compiling ffmepg with ‘libmp3lame’.
Any ideas?
Thnaks,
Ramana
% /usr/local/bin/ffmpeg -formats | grep mp3
FFmpeg version SVN-r11872, Copyright (c) 2000-2008 Fabrice Bellard, et al.
configuration: –enable-libmp3lame –enable-liba52 –enable-gpl
libavutil version: 49.6.0
libavcodec version: 51.50.0
libavformat version: 52.7.0
libavdevice version: 52.0.0
built on Feb 5 2008 21:29:17, gcc: 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
DE mp3 MPEG audio layer 3
EA libmp3lame
D A mp3
D A mp3adu
D A mp3on4
text2movsub remove_extra noise mov2textsub mp3decomp mp3comp mjpegadump imxdump h264_mp4toannexb dump_extra
[...] Transcoding MTS/M2TS AVCHD Video Into AVI Files with Free Software [...]
The current top-of-truck version of mplayer/mencoder from SVN can properly decode and transcode .mts files from a Canon HG10 into other formats.
For example, after compiling a new mencoder, I’m using the following single commandline to scale the .mts files down to something that will decode in realtime on my PC:
./mencoder /tmp/00126.mts -o /tmp/00126.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=5000 -fps 60 -vf scale=800:450
There’s no A/V sync issues either!
I’m trying to get this to work in Ubuntu 7.10, what am I doing wrong?
travis@travis-ubuntu:~/Desktop/m2ts$ ./m2tstoavi.fifo samplevideo.m2ts
bash: ./m2tstoavi.fifo: /bin/csh: bad interpreter: No such file or directory
you need to install the c shell interpreter:
sudo apt-get install csh
@Wayne - that makes sense, thanks!
Yeah sorry about that, I updated the article apt-get line so it’ll install it.
Whoops!
Awesome! Works like a charm now. What sources do you have in your /etc/apt/sources.list? For some reason, mine failed to get faad2.
You don’t happen to know how to deinterlace the video too do you? The interlacing in the video from my Canon HG10 is very obvious.
Thanks again for this guide and your help!
Cool, I just needed to modify the ffmpeg command to add -deinterlace and -aspect 16:9
I noticed faad2 doesn’t get too. Not sure why I had it in the list. Unfortunately, my Sony cam uses adaptive interlacing - if a scene has lots of movement it interlaces, if its a still shot, none. I’m waiting for the next crop of 1080p non-interlacing, true “progressive” cameras to come out. Several were shown at CES this year by cannon and samsung and others.
Hi,
i use “MEncoder dev-SVN-r26268-4.1.2″ (compiled on a P4/2GHz Xubuntu edgy) with this options
mencoder 00001.mts -o 1.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=10000 -fps 60 -vf scale=1140:1080
and it works fine & fast. My Cam is a Panasonic HDC-SD5.
Install Mencoder (MPlayer):
get codecs from http://www.mplayerhq.hu/design7/dload.html and install
svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer
./configure
make
make install
(maybe ln -s /usr/local/bin/mencoder /usr/bin/mencoder)
Thats all.
yup latest svn of mplayer demuxes and decodes MTS .. you can even play them straight, sort of, without reencoding
straight remuxing without reencoding to mp4 also works, but then player needs to have decoder that understands interlaced and highprofile h264 stream.. very few of them do.
I have a .mts file that is the first of 2 that were broken in 2 during a 1 hour video. There are 2-2 gig files and one less than 2. With that being said, the first of the 2 gig files I cannot open In my Sony Vegas 7 edit software. Ive been using this camera for a year and never had this issue before. How or can I get this file repaired. I’ve said enough…sorry for being so long winded! Please help if you can, very important video lost!
Rick
[...] Det kan gøres med Super (se på Softwaresiden under videolykke), DGAVCdec og et par andre programmer/metoder. Det tager også tid og er svært, men er måske den bedste løsning for [...]
Hey,
I am running the following ffmpeg line for the conversion :
ffmpeg -deinterlace -r 29.97 -s 1440×1080 -i $videofifo -i $audiofile -vcodec mpeg4 -sameq -acodec aac -ac 2 -ab 256k -aspect 16:9 -b 15000k $outputfile
But the resulting video has really annoying slow lines running down the screen. Especially when there is motion in the video.
Any ideas…?
Thanks
rename the .mts to .mpg and do what you want with it
@geoffrey, if only it were that easy, haha
@Yuri it sounds like your video isn’t getting de-interlaced properly
it is that easy try it and see
I just did and neither VirtualDub nor Windows Movie Maker could open it.
windows media player works with it
add this codec Haali Matroska Splitter to your system
It’s not working with ubuntu 8.04…help-me!
Here is how you do it in windows xp..
http://eugenia.gnomefiles.org/2008/05/04/avchd-transcoding-using-free-tools/
Can convert to any format that super can handle. Converted 2 min m2ts video to DVD vob (the website explains converting to mp4, but you can select any other output format in super), took about 20 min.
hope it helps,
peddu
[...] http://www.fsckin.com/2008/01/03/transcoding-mtsm2ts-avchd-video-files-with-free-software/ [...]
I have actually read this book cover to cover and Robert Hamburger is a genius. I am glad to see that others think so too. Thanks for the software… I would donate money but I don’t see a link… add one!
Britt
What about converting it onto something that a BluRay player will read? Which software for Win XP?
What if you simply wanted to put it on DVD-R in HD format for non-BluRay players & on a BDR for play in a BluRay player.
Thanks
When I try to run the apt-get line, I am having trouble. It appears that ffmpeg depends: libmp3lame0 and mplayer depends: liblame0, but libmp3lame0 conflicts: liblame0. Am I doing something wrong?
It’s not working with ubuntu 8.04 ..anyone..?
I use this guide to convert avchd pal to mpeg4 matroska:
http://linux-tipps.blogspot.com/2008/08/hd-video-encoding-in-sync-with-ffmpeg.html
It works quite well and fast under Ubuntu 8.04.
Thanks Dennis! Works fine on my Kubuntu 8.04 (2.6.24-19 AMD64)
What software other than iMovie can you use to pay MTS video files created by my Sony SR12?
Thanks
Doug
[...] http://www.fsckin.com/2008/01/03/transcoding-mtsm2ts-avchd-video-files-with-free-software/ [...]
So once you’ve transcoded the video, what is your final video editing solution? I use Vegas 8 (under XP) and find it edits AVCHD ok in realtime (on a dual core), but it certainly taxes the machine. However, the Aiptek cameras which record MOV files in H.264 not only require exporting from QuickTime to MP4 (due to a gimped audio format… its the audio which requires the export not the picture data), but can NOT be edited smoothly in realtime. I’ve got a quad core Dell XPS 430 and multi camera editing of 2 Aiptek 720p videos results in 1-frame-per-second updating in my editor. Crazy slow.
My point being even if you get a top-of-the-line PC, the cameras seem to be employing video compression which requires more and more power to decompress on the fly. It may be transcoding for responsive editing (like you have done) becomes the norm.
(But once again in case it was lost in this rant, what do you use to edit?)
Doug, VLC media player for both Linux and Windows should play your files. And pretty much anything else too!
It has many CODECs built in so other changes to CODECs on your system will not affect your ability to play video files.
Rob
Doug (and Robbie), my experience with VLC player is for some reason it does not play footage captured on my SONY HDR-SR1, often closing with an error. If anyone (Robbie?) has used it to play AVCHD footage captured with this camera and thinks the problem is me, please lemmie know!
Sorry to disappoint Gordon, the files I’ve played with VLC are not from this source so can’t confirm experience with that data. Good luck in your search for a solution.
[...] media players available in Ubuntu seems to work at the time of writing. For editing, there are workarounds, but I end up fiddling too long with conversions and not enough with the actual editing. Video [...]