?

Log in

No account? Create an account

LJ Downloader - Journal of Omnifarious

Jun. 7th, 2003

12:53 am - LJ Downloader

Previous Entry Share Next Entry

Well, the LJ downloading program is now about 1/3rd done. It's a huge pain. The interface they have for LJ client programs is not at all suited to downloading an LJ, despite appearances. There is critical information that's missing. It's designed too much for the convenience of clients instead of as a complete interface to LJ.

I had to pretend to be a web browser and go to http://www.livejournal.com/export.bml. I reverse engineered the form (not very hard, though I had to break out a packet sniffer) and now I can query it just like someone filled in the form and pressed the button.

But, that download interface doesn't get you the comments people left, it just gets what you put in. So, I have to write another piece which takes the itemid from the XML download and uses that to pull up the page with the comments on it. If there are a LOT of comments, it will still not work as the comments will be collapsed, but for most pages it'll be fine.

I place a high value on the comments on my LJ entries, so getting the comments is important to me.

Current Mood: [mood icon] accomplished
Current Music: Dido - All You Want

Comments:

[User Picture]
From:prettydark
Date:June 7th, 2003 12:33 am (UTC)
(Link)
have i told you lately how much ass you kick? :)
(Reply) (Thread)
[User Picture]
From:omnifarious
Date:June 7th, 2003 08:03 am (UTC)
(Link)

No. :-) Thank you! :-)

(Reply) (Parent) (Thread)
From:(Anonymous)
Date:June 7th, 2003 11:50 pm (UTC)

sweeeeeet candy

(Link)
omg please post this program when you are done. it would be so slick. i have several friends that also want to 'download' their journals but have taken to copy-pasting it into word2000 or something. u r0x0rs dude.
(Reply) (Parent) (Thread)
[User Picture]
From:herbstkind
Date:September 10th, 2003 07:13 am (UTC)
(Link)
I was searching google for a program to download my lj entries and ended up here. I'm really looking for something like the program you seem to develop. So, out of curiosity I'm commenting on your entry to ask, whether you have already finished.
(Reply) (Thread)
[User Picture]
From:omnifarious
Date:September 10th, 2003 07:53 am (UTC)
(Link)

It has been finished, but it's not very user friendly. :-) I need to put it up for public download though. Maybe someone else will add a nice UI to it. It's written in Python, so it should work on a Mac and under Windows. I was actually a little careful to not use anything Unix/Linux specific.

(Reply) (Parent) (Thread)
[User Picture]
From:herbstkind
Date:September 10th, 2003 07:56 am (UTC)
(Link)
If you ever put it up for download I would be very happy if you tell me... :)
(Reply) (Parent) (Thread)
From:j_shaver
Date:March 9th, 2008 08:30 pm (UTC)

News?

(Link)
Just ran into this, what ever happened to it?
(Reply) (Thread)
[User Picture]
From:omnifarious
Date:March 9th, 2008 08:34 pm (UTC)

Re: News?

(Link)

I finished it, but it's really sub-optimal in many ways. I discovered LJ Dump which is really a lot better in almost every way.

(Reply) (Parent) (Thread)
From:j_shaver
Date:April 6th, 2008 07:35 pm (UTC)

Re: News?

(Link)
Ok, Thanks, I found LJDump, but was hoping for a tool that did not require logging into the account.
(Reply) (Parent) (Thread)
[User Picture]
From:omnifarious
Date:April 6th, 2008 09:26 pm (UTC)

Re: News?

(Link)

You want to download all the public posts of a random LJ that isn't yours? Even the tool I developed wouldn't do that. I actually think doing that is ethically questionable and I would not generally be interested in developing such a tool.

Is there a different reason you wanted to not have to log into the account?

(Reply) (Parent) (Thread)
From:j_shaver
Date:April 12th, 2008 04:29 pm (UTC)

Re: News?

(Link)
It is not really a big deal. It just seems silly to have one more config file with passwords lying around in order to download information that is -- basically by its nature -- already public.
(Reply) (Parent) (Thread)