Snow Leopard and a 64-bit MAMP

5 September 2009

Update: I've written an updated version of this article that's far simpler to implement.

As a Mac user, the recent release of Snow Leopard meant an excuse to do a clean install of the OS. After I booted into the new OS it was time to set up my development environment all over again. Which got me thinking that how my system is set up and how I got it that way could be useful for others.

My Goals

I've set up my system to make setting up new projects and maintaining older ones as simple and quick as possible. That said, I also don't want to use third-party setup tools like MAMP.app or Headdress because a lot of my work is done in Django meaning my setup needs python integration. I also want to use OS X's built in versions of Apache, PHP, Python, etc. for two reasons:

  1. I don't have to maintain the software when new releases or security patches become available
  2. I am forced to make my code work with these newer versions

Installations and Setup

A couple of notes about managing all of these downloads. For anything that I compile from source (typically Python libraries), I put the source code in ~/Code/ so I can easily update the source and recompile as new versions are available. For simplicity, I use Git to maintain these repositories, even if they are originally in svn or cvs (see http://www.viget.com/extend /effectively-using-git-with-subversion/ and http://kerneltrap.org/mailarchive/git/2008/6/13/2118414 respectively). For package installers, I keep the most recent copy on a drive attached to my network so I don't need to re-download if a reinstall is needed.

TextMate is my editor of choice, and I use it's command line mate command throughout the article; you, of course, can replace that with the editor of your choosing.

zsh and the Command Line in General

I used to fear this application called Terminal (/Applications/Utilities/Terminal). Now I embrace it and find myself using it for a lot of my daily tasks. For the rest of this article, when I refer to the command line or say "run" followed by a command, I am talking about running a command in Terminal.

The Z shell (zsh) is my shell of choice (thanks to Lang Martin). It's already installed on OS X and using it is as simple as typing zsh at the command line. If you want to make zsh your default shell, run chsh `which zsh` on the command line and follow the prompts.

To start, you'll want to make a bin folder in your home directory. Run the following commands to get that setup.

cd ~
mkdir bin

Now you need to add your new "bin" folder to your path (a list of directories the shell looks at for programs it can execute).

    
mate ~/.zprofile

At the bottom of your .zprofile file, add

PATH=$PATH:~/bin; export PATH

SSH Shortcuts

You can customize a lot of things about zsh by modifying the file ".zprofile" in your home directory. (Run mate ~/.zprofile on the command line to edit it.) One trick Lang and I came up with is a way to have each tab in Terminal show the name of the server you are ssh'ed into. To do this, run the following steps for each server you want to add a shortcut for, replacing "nickname" with the nickname you want to give that server.

Now you should be able to just run nickname and be connected to your server with the tab showing nickname as it's title.

Case-insensitive Tab Completion

Tab completion on the command line saves a lot of time and mistyping errors. By default zsh will tab complete only on exact matches. However, getting case- insensitive completion is easy. Run mate ~/.zshrc then add zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}' 'r:|[._-]=* r:|=*' 'l:|=* r:|=*' to that file, save it and close it.

Git

I can't say enough good things about Git. It's my favorite version control system (VCS). If you're not familiar with version control, basically VCS's let you record different states of your code as you work so you can later go back and see a full history of all of your patches (what each revision is referred to as). In a group development environment, VCS's also allow multiple people to work on the same files at the same time and then merge their changes together into new code. Git really excels at the merge stage, being very smart about how it handles conflicts and refusing to allow a user to push (sending files to the central repository) conflicts. If Git sees that there are modifications to a file or files that you have been editing, it will make you download the newest revision first, let you handle any conflicts it can't figure out, then you can push the clean code up to the central repo.

Installation on OS X is very simple. Go to http://code.google.com/p/git-osx- installer/, look in the Featured Downloads box on the right, and get the latest version. (The Leopard versions run fine on Snow Leopard.) Run the installer. You now have Git installed. You'll want to do a few configuration steps next (several of these are from Tim Dysinger's article on installing Git).

First, run these commands:

    
git config --global user.name "Your Name" 
git config --global user.email "youremail@domain.com"

Then there are some optional, but handy, options:

    
git config --global color.diff auto
git config --global color.status auto
git config --global color.branch auto
git config --global color.interactive auto
git config --global alias.st status
git config --global alias.ci commit
git config --global alias.co checkout
git config --global alias.br branch

The first three tell git to colorize it's output to make the information it gives you more distinguishable. The last four set up some shortcuts to save you typing. If you've run all of these, you can now just run git st, etc. instead of the longer git status. Additionally, I add alias g="git status" to my .zprofile so I just need to run g to get the status of my current repository.

MAMP

MAMP stands for Mac + Apache + MySQL + PHP. OS X ships with all but MySQL pre- installed, although PHP isn't turned on by default. A lot of people do local development by adding entries to their "/etc/hosts" file for each project, but I've gone the route of a smarter MAMP using local DNS to automagically find projects in my Sites folder and serve them up without extra configuration. I've mostly followed the article above with a few changes:

  1. I use VirtualDocumentRoot "/Users/dryan/Sites/%-2+/html" for my VirtualDocumentRoot setting (replace dryan with your home folder name)
  2. Instead of putting my content in "projectname/public" I put it in "projectname/html" to match the layout of my host MediaTemple
  3. My project folders live directly inside "Sites", not in "Sites/dev/", etc.

Enabling PHP

To turn on PHP, run mate /etc/apache2/httpd.conf. Look for the line #LoadModule php5_module libexec/apache2/libphp5.so (line 114 for me), and remove the "#" at the beginning of the line. Save and close the file then run sudo apachectl restart to get the change to take effect. You now have PHP5 running on your local Mac.

MySQL

This one, to be frank, was a pain in the ass. Not so much installing MySQL itself, but getting it working with the Python MySQLdb library. Hopefully, my four hours of frustration can make this go much more smoothly for you. To start, download MySQL from their ridiculously complex downloads page. You want to download the "Mac OS X 10.5 (x8664)" version"Mac OS X 10.6 (x8664)" version. Once the disk image opens, double click "mysql-5.1.37-osx10.5-x86_64.pkg" to run the installer. Once that is finished, double click "MySQLStartupItem.pkg" to have MySQL run at startup. Finally, you can double click "MySQL.prefPane" to have a System Preferences pane for stopping and starting MySQL.

I'll note here that the preference pane is 32-bit, and Snow Leopard's System Preferences app is 64-bit. This means when you click on MySQL in System Preferences, it will give you a dialog about relaunching; this is fine and you can just click OK. Update: MySQL now ships a totally 64bit version of everything if you get the 10.6 version.

Another thing to be aware of, Snow Leopard doesn't do a traditional upgrade, it does what used to be called "Archive and Install". This means if you already had MySQL running on Leopard, that data will not migrate over automatically. The easiest fix is to export all of your data before you upgrade and then import it to your new setup after. If you're reading this post-upgrade and if you have a bootable backup of your old system, boot to it and do the export (this is what I had to do). If not, I'm very sorry.

At this point, run which mysql; if you get mysql not found as the result, edit your .zprofile again and add PATH=$PATH:/usr/local/mysql/bin; export PATH to it.

Python

Snow Leopard ships with Python 2.5 and 2.6, whereas Leopard had 2.4 and 2.5. Version 2.5 was the Leopard default so all my libraries were installed to it. I figured the easiest thing to do would be to change the default version Snow Leopard was using to 2.5. Unfortunately, Python 2.5 was shipped as 32-bit only. Using it would have required maintaining a whole separate setup of Apache, MySQL, PHP, etc. so I was forced over to 2.6. Most of my libraries couldn't care less and just worked when I moved them over to 2.6's site- packages folder. There were, of course, a few exceptions.

MySQLdb

There are severalgoodtutorials on setting up MySQLdb with OS X; none of them are current enough to work with Snow Leopard (much of the reason is that the library has now been updated to fix a couple of the previous OS X install bugs). Here's what you need to do:

  1. Run sudo ln -s /usr/local/mysql/lib /usr/local/mysql/lib/mysql
  2. Download MySQLdb from Source Forge
  3. Open the tar file if your browser didn't do that for you
  4. Inside the "MySQL-python-1.2.3c1" folder, edit "site.cfg" and change threadsafe = True to threadsafe = False and change line 13 to be mysql_config = /usr/local/mysql/bin/mysql_config. Save the file and close it.
  5. Edit "setup_posix.py" and change mysql_config.path = "mysql_config" to mysql_config.path = "/usr/local/mysql/bin/mysql_config"
  6. On the command line, cd into the "MySQL-python-1.2.3c1" folder (you can type cd (there's a space in there) then drag the folder onto the Terminal window to have the location entered for you)
  7. Run sudo python setup.py build. You may see some errors, it's typically ok to ignore them.
  8. Run sudo python setup.py install.

That should have everything running. To test it, run the following:

    
python
import MySQLdb

If that didn't throw any errors, pour yourself a drink because you deserve it; if it did throw errors, pour two because you'll need them. If you had no errors, press "ctrl+D" to exit out of the python shell. If it did have errors, check to make sure that mysql is running (you can use MySQL in System Preferences). If not, start it and repeat the last two commands. If it is running, copy and paste the errors into Google for some help. If you get stuck here, leave a comment below and I'll try to help.

Update: Learned the hard way today that MySQLdb adds both the installed location and the source code build to the Python egg path. After you successfully install MySQLdb, sudo rm -r dist in the MySQL-python-1.2.3c1 folder.

Python Imaging Library (PIL)

Django requires PIL for it's ImageFields. On Leopard this was a straightforward install; however the C library that PIL uses for jpg manipulation ships as 32-bit only in Snow Leopard. So before we can install PIL, we have to first install the newest version of libjpeg. This will require the OS X Developer tools to be installed. If you don't have them, grab them from the Apple Developer Connection. To install libjpeg:

  1. Download the source
  2. Open the tar file if your browser didn't do that for you
  3. On the command line cd in the "jpegsrc.v7" folder
  4. Run ln -s which glibtool ./libtool
  5. Run setenv MACOSX_DEPLOYMENT_TARGET 10.6
  6. Run ./configure --enable-shared && make && sudo make install

Now we can install PIL itself. There are instructions on the web for doing it with easy_install, but PIL isn't configured for that so it will not install properly. Instead we need to build it from source.

  1. Download the source
  2. Open the tar file if your browser didn't do that for you
  3. On the command line cd in the "Imaging-1.1.6" folder
  4. Run sudo python setup.py build. This will take a few minutes and you may see several warnings; that's ok. At the end, setup will output a build summary. If it says --- JPEG support ok, you can move on. If not, edit "setup.py" and change line 38 JPEG_ROOT = None to JPEG_ROOT = "/usr/local/lib/libjpeg.dylib" and run sudo python setup.py build again.
  5. Run sudo python setup.py install

Once that completes, PIL should be installed properly. To test out that theory, run python selftest.py in the "Imaging-1.1.6" directory. If it says 57 tests passed everything is good. If not, run the setup steps again.

Django

Compared to the last two steps, installing Django is a breeze. I'm basing these instructions on Brian Rossner's screencast but I'm making a few edits to make the download process a lot faster (if you actually plan on editing the django core and pushing patches back to the project, follow his instructions and not mine). You'll want to run the following commands in order, waiting for one to finish before running the next.

cd ~/Code
git svn clone -s -r11479:HEAD http://code.djangoproject.com/svn/django/

11479 was the latest revision at the time of posting. If you want to only get the latest revision and then keep up to date from there, go over to the Djanog SVN source and replace 11479 with whatever revision it says it is currently at.

If you get the error RA layer request failed: REPORT of '/svn/!svn/vcc/default'..., just keep running git svn clone -s -r11479:HEAD http://code.djangoproject.com/svn/django/ until you see Checked out HEAD:. Once you see that, keep going.

cd django
git svn rebase
git gc
cd 1.0.X
sudo python setup.py build
sudo python setup.py install

Now you have Django installed. To update to the latest revision later, run the following:

cd ~/Code/django/
git svn rebase
git gc
cd 1.0.X
sudo python setup.py clean
sudo python setup.py build
sudo python setup.py install

The makeproj Script

Anything we have to do more than once should be able to be automated. Setting up a project initially used to take me about 30 minutes. I'd create the folder, then the html subfolder, make a basic file structure, etc. Then put it into git, go make a repository on GitHub, push it there, make it a submodule in Sites. Having had enough of that, I wrote a shell script in python I call "makeproj". You can download makeproj if you want to use it, although I won't be supporting it. It's use-at-your-own-risk. If you want to use the GitHub features, you'll need py-github.

Conclusion

This is the first of a series of articles I'll be doing on my work process. Hopefully this one will show you some new tools and techniques you didn't know about and ease the pain of upgrading to Snow Leopard. One last tip: once you get your development environment working, grab a tool like Carbon Copy Cloner and make a bootable backup of your system. You'll then have it both for reference and to work from in a worst-case scenario. See you in the comments.