This is more of a note to self.
To compile MPlayer svn with x264 git first compile x264 with
./configure --enable-shared --prefix=/usr/local/x264
And then compile MPlayer with
./configure --extra-ldflags=\"-L/usr/local/x264/lib -Wl,-rpath,/usr/local/x264/lib\" --extra-cflags=\"-I/usr/local/x264/include\"
EDIT: Though now I use Arch Linux with mplayer2 from the AUR, so I don't have to bother with what's above.