Memcache Necessities

While helping a fellow facebook developer get memcache setup, I ran across a few problems getting memcache running. Luckily I found the solution at http://louzell.wordpress.com/2007/10/11/how-to-install-memcached/

First install libevent:

#curl -O http://monkey.org/~provos/libevent-1.3e.tar.gz
#tar xzvf libevent-1.3e.tar.gz
#cd libevent-1.3e
#./configure --prefix=/usr/local
#make
#make install

And then finally memcache:

#curl -O http://www.danga.com/memcached/dist/memcached-1.2.2.tar.gz
#tar xzvf memcached-1.2.2.tar.gz
#cd memcached-1.2.2.tar.gz
#LDFLAGS='-Wl,--rpath /usr/local/lib' ./configure --prefix=/usr/local
#make
#make install

Update: For the latest 1.2.6 i used

#LDFLAGS='-Wl,--rpath /usr/local/lib' ./configure --prefix=/usr/local --enable-64bit --enable-threads

MacBook Pro separation anxiety

It would seem that not having a laptop would cause more withdrawal for me than it has at this point. I dropped off my MBP at the local Apple Retail Store this past Sunday so that they could take repair/replace the loose hinge that causes the lid to fall closed under it's own weight. They called me Sunday night saying that they needed to order a part for it since the hinge had to be replaced instead of adjusted.

What really confused me is that I brought the MBP in Friday night for them to look at and they said they needed to order a part for it, so they let me take my portable home with me until they got the part in. They then called me Saturday saying they had the part in, thus I brought it back in on Sunday.Now I could have gone back to the Apple Store and picked it up until they got the new part in again, but I figured that if I did that, it would take an "additional" 7-10 days for them to have it fixed once I brought the MBP back in again and I'm not in the mood to play games with them on this. I figure I'll give them their 7-10 days from Sunday and bitch and complain loudly if it ain't fixed by then.

Blah. enough of my rant. They really have me pissed off. I want my portable back and I want it now. I paid $2500 for a laptop and I expect nothing less than perfection.

mencoder goodness for TivoDecode Manager

As a follow up to yesterday's entry, I have finally managed to compile a custom Mac Intel Core 2 Duo build of mencoder that provides drop in functionality to TivoDecode Manager (TDM) while cutting encoding times in half. 7 hours of sleepless night-time later I present to you the new & improved Mac Intel-optimized mencoder binary (7.5MB).

Here are the benchmarks I have so far:
TDM's provided mencoder binary

$ time /Applications/TiVoDecode\ Manager.app/Contents/Resources/mencoder -af volume=13:1 -of lavf -lavfopts i_certify_that_my_video_stream_does_not_use_b_frames -demuxer lavf -lavfdopts probesize=128 -oac lavc -ovc lavc -lavcopts keyint=15:aglobal=1:vglobal=1:coder=1:vcodec=mpeg4:acodec=aac:vbitrate=1800:abitrate=128 -vf pp=lb,scale=640:480,harddup -o ~/Desktop/clip.tdm.mp4 ~/Desktop/clip.mpg

real 1m9.518s
user 1m7.768s
sys 0m0.847s


Mac Intel-optimized mencoder binary

$ time ./mencoder -af volume=13:1 -of lavf -lavfopts i_certify_that_my_video_stream_does_not_use_b_frames -demuxer lavf -lavfdopts probesize=128 -oac lavc -ovc lavc -lavcopts keyint=15:aglobal=1:vglobal=1:coder=1:vcodec=mpeg4:acodec=aac:vbitrate=1800:abitrate=128 -vf pp=lb,scale=640:480,harddup -o ~/Desktop/clip.custom.mp4 ~/Desktop/clip.mpg

real 0m29.061s
user 0m27.210s
sys 0m0.743s


and here are the file sizes

$ ls -ltr clip.*
-rw-r--r-- 1 xxx xxx 12654193 Mar 22 01:59 clip.mpg
-rw-r--r-- 1 xxx xxx 20839638 Mar 23 01:50 clip.tdm.mp4
-rw-r--r-- 1 xxx xxx 20859165 Mar 23 01:51 clip.custom.mp4


and just for fun, I lowered the bitrate down to 600kbs so that it actually does compression and here are the times and filesizes.

TDM:

real 1m7.573s
user 0m59.340s
sys 0m0.904s

Intel-optimized

real 0m31.287s
user 0m25.143s
sys 0m0.787s


$ ls -ltr clip*
-rw-r--r-- 1 xxx xxx 12654193 Mar 22 01:59 clip.mpg
-rw-r--r-- 1 xxx xxx 8135459 Mar 23 02:26 clip.custom.mp4
-rw-r--r-- 1 xxx xxx 8131077 Mar 23 02:27 clip.tdm.mp4


And well if you've managed to wade through all that I guess I'll leave you with some installation instructions for the mencoder binary.
1. Download
2. Backup old mencoder binary

mv /Applications/TiVoDecode\ Manager.app/Contents/Resources/mencoder /Applications/TiVoDecode\ Manager.app/Contents/Resources/mencoder.old

3. Copy in new mencoder binary

cp mencoder /Applications/TiVoDecode\ Manager.app/Contents/Resources/mencoder

4. Enjoy (Did I mention this probably only works for the Core2Duo??)
5. Digg this

Simple performance boost for TivoDecode Manager

With the release of AppleTV I started playing around with the TivoDecode Manager again tonight and started converting content from my Tivo. While playing around with the latest version of TDM (2.1), I noticed that the version of mencoder bundled with TDM was completely unoptimized. Here is the output from their build of mencoder

$ /Applications/TiVoDecode\ Manager.app/Contents/Resources/mencoder
MEncoder dev-SVN-r21629-4.0.1 (C) 2000-2006 MPlayer Team
CPU: Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz (Family: 6, Model: 15, Stepping: 6)
MMX supported but disabled
MMX2 supported but disabled
SSE supported but disabled
SSE2 supported but disabled
CPUflags: Type: 6 MMX: 0 MMX2: 0 3DNow: 0 3DNow2: 0 SSE: 0 SSE2: 0
Compiled for x86 CPU with extensions:


Now compare this output to the version recommended by ffmpegX from the sourceforge build

$ /Applications/ffmpegX.app/Contents/Resources/mencoder
MEncoder dev-CVS-060307-04:23-4.0.1 (C) 2000-2006 MPlayer Team
Mac OSX static build for ffmpegX
CPU: Intel (Family: 6, Stepping: 6)
FIXME:Hardenabling SSE and SSE2 without detection
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled for x86 CPU with extensions: MMX MMX2 SSE SSE2


Notice anything missing? Yeah. TDM is completely ignoring the MMX and SSE registers!

So a quick timing on a 87 second video clip yielded the following with TDM's mencoder

real 1m10.966s
user 1m8.981s
sys 0m0.565s


The same timing with identical options on ffmpeg's mencoder yielded a result of

real 0m24.834s
user 0m23.379s
sys 0m0.443s


That's about a 50% reduction in encoding time!

Of course the -lavfdopts probesize=128 -lavcopts aglobal:1:acodec:aac options did not work and had to be stripped or changed to compatible options in both, but the data is somewhat clear that TDM needs to recompile their mencoder with SSE and MMX extensions enabled.

So last night I decided to undertake the task of doing all that and BLEH. It's harder than it sounds. Apparently I'm missing the faac library among other things and compiling the libfaac on the Mac isn't the most straight-forward task. It looks like I'll be pushing this task to another day, but the findings so far are interesting to say the least.

The Apple TV debate continues

Well, if you haven't read Walt Mossberg's review of AppleTV yet, this is a good place to start. Of course many people will argue that the AppleTV is a niche product, or has limited content, or doesn't have DVR functionality or this or that. Personally, I've already come up with a great use for the AppleTV that fits my needs and possibly the needs of millions of people.

Tivo. But wait a second, AppleTV isn't a DVR. That's not my point. My focus is the subset of people out there who have a Mac, already have Tivo service, and have two or more TVs, but have only one Tivo. Now Tivo is great, but is made better by TivoDecode Manager(TDM) for the Mac. For most TV owners, the TV content is stuck on your Tivo and one TV unless of course you don't mind watching your TV shows on your computer. But that is exactly what AppleTV is here to solve. Moving content away from your computer and to your TV.

With TDM I can batch queue all the new shows on my Tivo to have them converted to an iTunes compatible TV show or movie while I am asleep or at work, ready to be watched in a different room at my leisure. Now I don't have to fight over what's on the Tivo and be limited to watching it on the one TV that it is hooked up to. Of course I could always take the plunge and purchase another Tivo and record even more shows, but in reality, there aren't enough TV shows to keep a Dual-Tuner Tivo busy. Plus, by the time I purchase another Tivo and then the monthly service, I've already sunk another $350 into Tivo.

Of course there are plenty of other ways to use the new AppleTV, and everyone will find a way to make it work for them, but as always, this is just my $0.02.

Prototype Array fun

I must say, I love prototype and I love cliches. Prototype is the best thing since sliced bread. Okay, maybe not the absolute best thing, but it's close. For example, I was pretty mad that javascript doesn't have a function to check if an array key exists like PHP's array_key_exists function, but with prototype, I can come pretty close.

Array.prototype.keyExists = function(key) {
 for (var i in this) {
   if (i==key) return true;
 }
 return false;
};

Hopefully this is self explanatory as I'm being lazy and I'm not in the mood to write out detailed examples of how to use it.

Apple Leaks Novatel V740 ExpressCard?

If anyone has been careful enough to scrutinize the latest Apple Software Update for WWAN devices, you may have noticed a mysterious Verizon Novatel V740 ExpressCard EVDO Rev A. Now as it seems now, Sprint just started shipping there Rev A devices today according to the folks over at EVDO Info, but there has been no real word about a Rev A ExpressCard solution from Verizon. Hopefully, there is more news about this new device in the future.

Compressed JS and mod_rewrite

Thanks to Joseph Scott's Compressed Javascript article and comments, I've managed to make some headway on making a solution that degrades nicely for browsers that don't support gzipped JS well at this point. Basically, by sticking to

<script type="text/javascript" src="js/prototype.js"></script>

and then letting apache serve up the gzipped file works best for me so far. By adding the following to my .htaccess file in my js directory

<FilesMatch "\\.js.gz$">
  ForceType text/javascript
  Header set Content-Encoding: gzip
</FilesMatch>
<FilesMatch "\\.js$">
  RewriteEngine On
  RewriteCond %{HTTP_USER_AGENT} !".*Safari.*"
  RewriteCond %{HTTP:Accept-Encoding} gzip
  RewriteCond %{REQUEST_FILENAME}.gz -f
  RewriteRule (.*)\.js$ $1\.js.gz [L]
  ForceType text/javascript
</FilesMatch>

I am able to push the correct headers to the browser as well as determine whether to serve up the js or js.gz version.

This also relies on the fact that a static precompressed gzipped version of the js file is created so there is little overhead required to gzip the file on every request. If the gzipped version of the file doesn't exist, it simply serves up the uncompressed file.

Of course there are a couple of caveats to this as if you change the JS file, you should update the gz file too or you will wonder why your changes aren't showing up.

Have fun!

New stomping grounds

I've decided to start a new blog for my misadventures and ideas. Hopefully something here will inspire you, if not leave you in awestruck. A few warning for those of you who may read this.

I'm not the most politically correct person.
I will offend SOMEBODY at SOME POINT.
I suck with grammar, so don't expect half of my posts to make sense. That's why this is a rambling.

Okay, back to work. Until then...