Installing CHDK on the Canon SX120IS Camera

Peter D. Hiscocks Royal Astronomical Society, Toronto Centre, Canada Syscomp Electronic Design Limited Email: phiscock@ee.ryerson.ca September 29, 2010

Introduction
The digital camera is a useful instrument for the measurement of light pollution. It can be used to simultaneously create a photographic record of a scene and perform the function of a light meter. If the camera is properly calibrated, the numerical value of a pixel in the image can be related to luminance (brightness) in the image. Then a digital camera becomes a low-cost measuring instrument, replacing specialized instruments of much higher cost. The Canon SX120IS is a point-and-shoot camera that normally produces JPEG formatted images. This exercise was undertaken in order to obtain RAW format images from the camera. The JPEG images have a non-linear luminance-pixelnumber response, with 8 bit (1:256) dynamic range. The RAW format images have a linear luminance-pixelnumber response, with 12 bit (1:4096) dynamic range. A linear response curve should simplify calibration. Fortunately, the Canon Hackers Development Kit [1] has generated software that supplements the Canon SX120IS operation, including the capability for on-screen live histograms and RAW format output. Everything is available to make this work, but nding and interpreting the material is somewhat challenging. This document will hopefully speed up the process. Much of this document is cut and paste from other sources, so the content is not original. I have organized the ow of information, the references and the specics of the SX120IS camera. 1. Obtain a suitable SD storage card. You may be able to use the card thats currently in the camera, but SD cards are very inexpensive and its convenient to have a card dedicated for this purpose. The best capacity is 2GB. A larger card must be partitioned, which complicates the install process. I obtained Kingston 2GB SD storage card SD/2GBCR. 2. Obtain a USB SD card reader Camera card readers are very inexpensive. You will need one to read the SD card. Mine is a SanDisk SDR-88. 3. Format the card The card I purchased was already formatted, as conrmed below. However, if the card must be formatted, the instructions are here [2]. Here are the instructions for formatting a card: Format the card to FAT16 with a desktop computer via a USB card reader and create the DCIM and MISC folders on the newly formatted card. Alternatively, you can format your

card with the camera, which in some models creates a FAT16 lesystem. (Note that the Canon SX200 IS, rmware version 1.00D formats the SD card to FAT32. Therefore formatting the SD card in an SX200 IS will not give you a partition from which CHDK can be booted, and you must use the terminal commands on a PC). You can check how an SD card has been formatted by running the "strings" Unix command on the volume, and looking at the output for FAT16 or FAT32. Important: Due to the limitations of FAT16 this will not work with cards bigger than 4G! I didnt have to actually do this, because the card was formatted to FAT16, which is conrmed below. 4. Make the card Bootable The camera needs to know that there is computer programming code on the SD card, so the card needs the magic word BOOTDISK must be on the card. I prefer to use Linux if possible, but in this case I couldnt determine the mount point of the card, so I decided to use Windows XP. Under Windows, I plugged in the card reader and plugged the card into the reader. The card plus USB reader appear as drive I:\. Basically, you are going to manually change a few bytes in the card header to make it bootable. Here are the steps I used to make the card bootable (cribbed from one of the above sites): (a) (b) (c) (d) (e) (f) (g) (h) HDHacker [3] is a free utility that allows direct manipulation of the information on a memory card. Download, install and run the HDHacker utility. Select the proper drive letter corresponding to your SD card in the card reader. Select logical drive and first sector (probably the default) Read sector from disk Save this sector to a le for safety (512 bytes) (I named it: BootSector_Drive_I.dat) Download and install the HexEdit le editor [4] We now use the hex editor to edit this le. Position the cursor at offset 0x40 (decimal 64) and type BOOTDISK in ASCII mode. Keep in mind that, if you do not see FAT12 or FAT16 around 0x36 (decimal 54) in the hex editor, you probably picked the wrong drive. Do not overwrite anything in that case. (i) Save it as a different le (I named it: SDBootSectorBootable.dat) (j) Use the HDHacker command Load sector from file to retrieve this new le containing BOOTDISK information. (k) Use the HDHacker command write sector on disk to write the new le to the SD card. Dont forget to select the correct drive letter, or you could trash your main hard drive!

The SD card is now bootable. 5. Determine version of the rmware on your camera In order to install the correct CHDK software, you need to know the version of the rmware in your camera. That information is in the metadata (exif le) stored with any photograph taken by that camera. On my linux machine, the program exiftool is installed [17], so I can run the command exiftool foo.jpg where foo.jpg is some image taken with the camera, and it will dump out the version number. Heres how you do it under Windows. Using the instructions in this document [6]: 2

CameraVersion is a small Windows tool that helps to get the real rmware version of a Canon Powershot camera. Based on Phil Harveys ExifTool, it shows the rmware version based on the EXIF data from a Canon JPG image. Download the app. from here [5]. The le is a self-extracting archive, it contains CameraVersion.exe (the executable le) OS requirements: MS Windows OS (2000 / XP / Vista / 2003 / 2008 / Seven) You will need to shoot a jpeg image and have it saved on the SD card. Have the card inserted into a card reader connected to a PC. Start the tool (CameraVersion.exe), select the le. In either case, Linux or Windows, we nd the following information in the exif le: Canon Firmware Version 1.00 Firmware revision 1.00 rev 2.00 Just to add to the fun, this is referred to elsewhere as rmrmware revision 1.00B. 6. Install CHDK Surprisingly, downloads of the CHDK software are not available on the CHDK website, possibly because the code is still in beta test. I found suitable versions on this website [7]. In this case, I downloaded sx120is-100b-0.9.9-912-full.zip Notice that the version number of the camera rmware (1.00B) is encoded into the name of the zip le. That assures us that we have the right le1 . Also of interest is the le named trunk912.zip, which contains the source code. (a) Download the latest version, unzip it and read the readme le. There is a lot of useful information on operating the software in the readme le. (b) Unzip the archive. Copy the CHDK folder (with its contents), the le DISKBOOT.BIN and the le vers.req to the SD card. The les in the CHDK folder are the new commands. DISKBOOT.BIN is the binary executable that interprets the new commands. vers.req is required for the camera to report its version on the LCD screen. (We dont need that, but its a useful conrmation.) (c) Ensure the the card is in the lock position. (Note: its very easy to misread the Lock/Unlock indicator. Look at it very, very closely.) Insert the card into the camera. 7. Verify the Firmware Version This is not essential. But it conrms the version number of the camera and it demonstrates that the camera is reading the SD card correctly. (a) Start camera IN PLAYBACK-MODE. To do that, do not power on the camera. Press the play button. (Rear panel of the camera, top left, button shows a square with a right arrow in it.) (b) When you extracted the archive it placed a le on your card called vers.req (or ver.req depending on the camera model). This will allow you to see, with the right camera-button presses, the TRUE rmware version of your camera (and other interesting information). (c) Press your Func Set button, hold it down. While holding down your Func Set button press your Display button. You will see the version of the rmware in your camera. Conrm that it is the right rmware to match up with the CHDK that you downloaded. Your real rmware version is NOT 1.0.1.0 nor 1.0.0.0 Your REAL rmware version will look something like: Firmware Ver GM1.00A.
1 You

could search for a later version of the le by Googling sx120is-100b.

(Pressing Display again a few times while still holding down your Func Set button will let you see even more information about your camera, including total number of shots taken since it was made.) In my case, camera version shows 1.00B. Total shoot is 550. 8. Run CHDK (a) Using the dial on the top of the camera, put the camera in some sort of image mode, such as faces. (b) Start the camera using power on button (top of camera). (c) You should briey see the CHDK menu, before it is overwhelmed by the usual Canon menu. If the camera starts showing the date and time entry widget, hit MENU to get out of it. (d) Hit the DISP button to get rid of the Canon displays. (e) Hit the <alt> button (the face symbol on the rear of the camera, above and to the left of the selection wheel) and then MENU. The CHDK menu should appear. Congratulations! You now have CHDK operating on your camera. The CHDK User Manual is here [8] 9. Image Processing Now we need to give some thought to which le formats to use and how to process the information. The ultimate goal is to get the image into the imagej image analysis program [11]. The Canon SX120IS can produce raw les in any of the following formats: CRW CR2 THM WAV DNG Canon raw format CRW is described here: [9] Canon raw format 2 CR2 is described here: [10] According to Wikipedia: [12] DNG (digital negative) is the only raw image format for which industry-wide buy-in is being sought. It is based upon, and compatible with, the ISO standard raw image format ISO 12234-2, TIFF/EP, and is being used by ISO in their revision of that standard. (See the Standardization section, below.) Also see [13] According to this, DNG is related to TIFF format. None of these formats can be read by the ImageJ analysis program. The formats that ImageJ can read are: TIFF GIF PGM FITS 4

 BMP Of these, TIFF seems like the best bet since DNG is based on TIFF, TIFF uses lossless compression and TIFF is supported by a standard. Can ImageJ load a DNC format image? Yes and no. It appears to load a thumbnail of 128 x 96 pixels. It can load JPG, even though thats not on the list. 10. Raw Files Here is the process to have the camera produce RAW les. The rst step is to create the badpixel.bin le. Ensure the memory card is in lock position. Startup the CHDK menu: Power on the camera Place in record mode by half pressing the shutter. Press <alt> (face) Create badpixel.bin appears in bottom left of display. Press Func Set. Menu appears for Load script from le. Press Func Set to select. Navigate to Test folder. Press Func Set to select. Navigate to Badpixel.lua. Press the shutter button to start the script. Wait. Shutter res. LED ashes. Busy indicates. Time passes. Wait until the message Press SET to save file. If the usual CANON menu appears and this message does not appear, enable the CHDK menu with <alt> <MENU>. Press Func Set. (l) Menu shows File is saved. (m) Press Menu to call up main menu. (n) To check that the le has been saved, put the card in the reader and open the CHDK directory. badpixel.bin should be there. If you decide to redo the process, it may be that the le is overwritten without any need to save it again. If you want to make sure you get a new version, erase this one. Youll need to unlock the SD card, erase badpixel.bin and then lock the card. Now we need to set up the RAW parameters: (a) Power on the camera. (b) Show the CHDK menu: <alt>, <MENU> (c) Enable Save RAW, DNG Format, DNG File Extension Notice that these settings are lost when the camera is powered off. Take some pictures. Place the SD card in the card reader. There should be some les ending in .DNG, and their corresponding .JPG les. 11. Using DCRAW to generate TIFF Format The program DCRAW [14] can convert from DNC to TIFF format. So we can use it to convert the DNC raw les from the camera into TIFF les for ImageJ. On our Suse Linux box, DCRAW is already installed. On a Windows machine, youll need to download the program. This website [15] has a description of doing that, along with a useful tutorial on using DCRAW. For example, to convert image crw_0569.dng to TIFF format, the command line is: 5 (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

dcraw -T crw_0569.dng After running this command, we nd the le crw_0569.tiff, which will then load into imagej. 12. How many bits to each pixel? How many bits per pixel do we obtain from RAW mode on the Canon SX120IS? You would think this is easy to nd out, but its not. According to [16]: Bits per unit, 12 bits The properties of the image may be explored using exiftool, which is already installed on this linux box [17]. Example: exiftool crw_0569.dng shows among many other properties Black Level White Level : 128 : 4095

which implies a 12 bit image (4096 distinct levels). The tiff image produced by dcraw is 8 bits unless specically asked to produce 16 bits: dcraw -T crw_0569.dng exiftool crw_0569.tiff Bits Per Sample : 8 8 8 dcraw -4 -T crw_0569.dng exiftool crw_0569.tiff Bits Per Sample : 16 16 16 Loading this into imagej shows a 16 bit image, histogram etc. So: the DNG (Raw) image from the camera has 12 bits of resolution. Converting it to TIFF, using DCRAW, the image can have 8 bits of resolution (not useful) or 16 bits of resolution. The image analysis program ImageJ automatically adjusts to process 16 bit TIFF raw les. 13. Workow Summary Canon SX120IS with CHDK software Configured to generate RAW files in DNG format | | file.dng, 12 bit V dcraw -4 -T crw_0569.dng | | file.tiff, 16 bit V imagej

References
[1] CHDK, the Canon Hacker Development Kit, home page http://chdk.wikia.com/wiki/CHDK [2] Formatting the SD card http://chdk.wikia.com/wiki/Bootable_SD_card [3] HD Hacker http://www.freedownloadscenter.com/Utilities/Disk_Maintenance_and_ Repair_Utilities/HDHacker.html [4] HexEdit, a utility that allows editing at the binary level http://www.physics.ohio-state.edu/~prewett/hexedit/ [5] Camera Version (for Windows) http://drop.io/chdkshellbeta/asset/cameraversion11-sfx-exe [6] CHDK Installation Guide Supplement to User Quick Start Guide (pdf) http://chdk.wikia.com/wiki/File:CHDK_Installation_Guide.pdf [7] Download site for CHDK software http://drop.io/6y1tiuz/ [8] CHDK User Manual http://chdk.wikia.com/wiki/File:CHDK_UserGuide_April_2009_A4.pdf [9] Canon raw format, CRW http://www.sno.phy.queensu.ca/~phil/exiftool/canon_raw.html [10] Canon raw format 2, CR2 http://lclevy.free.fr/cr2/#intro [11] The amazing ImageJ, image analysis program http://rsbweb.nih.gov/ij/ [12] Wikipedia: Raw Image Formats http://en.wikipedia.org/wiki/Raw_image_format [13] Wikipedia: Digital Negative Format http://en.wikipedia.org/wiki/Digital_Negative_(file_format) [14] DCRAW image le format conversion utility http://www.cybercom.net/~dcoffin/dcraw/ [15] Tutorial on using DCRAW http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm [16] Home page for CHDK on the Canon SX120IS http://chdk.wikia.com/wiki/SX120IS [17] EXIFTOOL, utility for reading metadata from images http://www.sno.phy.queensu.ca/~phil/exiftool/

Acknowledgement
Special thanks to the hard-working people who put together the amazing CHDK software, and to those people who documented the installation and operation. Every one of the programs described here is zero cost, and many of them are fully open source. That is amazing.