The ? means that the machine cannot find a drive with a valid operating system. This means either that your hard drive has failed, or the operating system on it is damaged.
What to do: http://support.apple.com/kb/TS1411?viewlocale=en_US
Safe Mode: http://support.apple.com/kb/HT1564
(Starting in Safe Mode will automatically run the fsck disk maintenance routine.)
If you can’t boot at all, and the lights are acting strangely or you don’t get the question mark, try a System Management Controller reset and try again
http://support.apple.com/kb/ht3964
First step: Boot from the DVD of OSX that came with your machine (insert the DVD, restart the machine with the power switch, hold the C key down throughout the restart to force it to boot from the DVD)
Once it has booted, answer the language preference screen, and it will take you to the Installer screen. DO NOT start an Install now. Instead, go to the Utilities menu and choose Disk Utility.
Does the icon of your hard drive show up on the left pane?
If not, then your hard drive is damaged and unreadable. Take the machine to a technician.
If your hard drive icon does show up, click it once to select it, and choose Repair Disk from the right hand pane of Disk Utility.
If it says that it repaired the problems, then try restarting again from the hard drive.
If it doesn’t restart, then you have to get more involved in repair.
If DiskUtility gave the drive a clean bill of health, you could try proceeding into the Installer, and do an Install (OSX 10.6) or “Archive and Install” (OSX 10.5 and earlier) (Do not choose Erase and Install!!) This will install a new OS X on the drive, and will attempt to preserve your settings, data, user accounts and programs.
Otherwise, consider getting one of the repair utilities listed below, or take to a technician.
Maintosh repair resources
Apple’s DiskUtility http://support.apple.com/kb/TS1417
Alsoft DiskWarrior http://www.alsoft.com/DiskWarrior/
TechTool Pro http://www.micromat.com/
Recovery
MiniTools Mac http://mac.powerdatarecovery.com/
DataRescue http://www.prosofteng.com/
FileSalvage (Mac) http://subrosasoft.com/OSXSoftware/index.php?main_page=product_info&products_id=1
Support
MacRumors http://forums.macrumors.com/
ehMac: http://www.ehmac.ca
MacFixIt http://www.macfixit.com/
Apple Support http://www.apple.com/support/
Apple Support Discussions http://discussions.apple.com/index.jspa
Your other option is to physically remove the hard drive and put it into a USB drive enclosure. Then hook that up to a different computer to copy your files off of the old hard drive.
You can boot a Macintosh with a Firewire port into Firewire Target Disk Mode (hold the “T” key down through boot up until the firewire symbol appears on the screen). If you cable your machine to another Mac with a Firewire cable (plug the cable in when both machines are off) then boot in Target Disk mode, the other machine will see your hard drive as if it was a Firewire external drive. You can then do disk repairs on it, or copy data off. If the two machines are the same model, then you could also reinstall the OS onto the target drive.