Thursday, May 7, 2009

How did I write my own macros for web scrapping from

I remember the following always, "Give a man a fish, he will not be hungry for the moment. Teach a man how to fish, he will never be hungry".

So in this blog we will see how to write macros using firefox using an example. The example is how to write a macro which you could use to shift all your photos in original quality from KodakGallery to Local drive/disk.

Items needed:
1. Firefox 3.0 or above (
2. iMacros for firefox (, Click "Install now" and then select the add-on from the dialog box and click "Install now" button. Restart firefox, once.)
3. MeasureIt (Goto, Click "Install now" and then select the add-on from the dialog box and click "Install now" button. Restart firefox, once.)

For the Do-It-Yourself folks:

Here are the detailed steps you should follow,
1. Open a new firefox window.
2. Visit ""
3. Click SignIn
4. Enter your E-mail address/mobile number and password and click "Sign-in"
5. Click "My Photos"
6. Click on one of the albums from which you need to download all the photos.
7. Click on the first image in the album
8. Click on the "iOpus iMacros" button, located beneath the menu bar. You should see a "iMacros" frame.

9. Click "Edit" tab in "iMacros" frame

10. Click "Options" button in "Edit" tab

11. Click "Paths" tab in the "Options dialog

12. Change Folder Downloads, to any folder you wish. Click "Apply" after the folder changes
13. Click "Rec" (Record) tab
14. Click "Click Mode" button in the "Rec" , select "Use X/Y position" radio button and click "Ok"
(Buckle up. We are ready to record the macro. :-) )
15. Click "Record" in the "Rec" tab. Now we are in the recording mode.
16. Context click (Right click for windows users) on "DownloadDownload high-resolution " link, select "Save link as", select the folder in which you need to save the JPG (same folder which you selected in Step 12) and click "Save"
17. Click on the "next" navigation button
18. Click "Stop" button in the "Rec" tab of iMacros frame
19. Now click MeasureIt tool in the bottom-left part of the browser's status bar. You should see a layer of brightness painted over the browser view panel.
20. Now you have to measure the width and height of the "Download" link. To do that, click on the link and drag your mouse towards top-left and make a note of the width and height. For me it was height=525 and width=568. I guess this value might vary based on your resolution.
21. Click "Edit" tab (by default our current macro named "#current.iim" would be preselected in the List box of the iMacros frame, if not, select it)
22. Click "Edit Macro" button of the "Edit" tab
23. Now the macro would have been opened with a file editor (for me it opened in Notepad).
a) Delete the line that looks like "URL GOTO="
b) Look for a line "CLICK X=undefined Y=undefined CONTENT=EVENT:SAVETARGETAS". Change it to "CLICK X=525Y=568CONTENT=EVENT:SAVETARGETAS"
25. Save the file and closer the file editor.
26. Now goto "Play" tab
27. Enter the number of photos you are yet to download in the "Max" textbox (it should be N-1, where N is the total number of photos in your album, considering you have already downloaded one)
28. Click "Play (Loop)"

That's it folks. Now you know how to write a iMacros yourself. Happy web scrapping.

No comments:

Post a Comment