LJ Downloader - Journal of Omnifarious
Jun. 7th, 2003
12:53 am - LJ Downloader
Well, the LJ downloading program is now about 1/3rd done. It's a huge pain. The interface they have for LJ client programs is not at all suited to downloading an LJ, despite appearances. There is critical information that's missing. It's designed too much for the convenience of clients instead of as a complete interface to LJ.
I had to pretend to be a web browser and go to http://www.livejournal.com/export.bml. I reverse engineered the form (not very hard, though I had to break out a packet sniffer) and now I can query it just like someone filled in the form and pressed the button.
But, that download interface doesn't get you the comments people left, it just gets what you put in. So, I have to write another piece which takes the itemid from the XML download and uses that to pull up the page with the comments on it. If there are a LOT of comments, it will still not work as the comments will be collapsed, but for most pages it'll be fine.
I place a high value on the comments on my LJ entries, so getting the comments is important to me.