I’m surprised (and somewhat impressed) that it works for Wordpress blogs too.

Good work!

greywulf 2010-06-01 05:51 UTC

Now that the coding has been done, I need to do actual text assembly. Yikes!

AlexSchroeder 2010-06-01 17:52 UTC

If you’re wondering how to do this… Assume you want to pull a copy of A Hamsterish Hoard of Dungeons and Dragons. Examine the source code and you’ll find a link to the atom feed within blogger. This is important, because it’ll provide us with the blog Id! In this case:

<link rel="alternate" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://hamsterhoard.blogspot.com/feeds/posts/default" /> <link rel="alternate" type="application/rss+xml" title="A Hamsterish Hoard of Dungeons and Dragons - RSS" href="http://hamsterhoard.blogspot.com/feeds/posts/default?alt=rss" /> <link rel="service.post" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://www.blogger.com/feeds/5373792969086619654/posts/default" /> ← that’s the one we’re looking for!

for i in seq 4; do
start=$((($i-1)*25+1))
curl -o taichara-$i.atom "http://www.blogger.com/feeds/5373792969086619654/posts/default?start-index=$start&amp;amp;max-results=25"
done

Save it in a script such as download-atom.sh and run it using bash download-atom.sh. You’ll end up with the files taichara-1.atom taichara-2.atom taichara-3.atom taichara-4.atom.

Now take the Perl script from the main page and save it as url.pl. It will extract the page URLs from the Atom files.

for f in *.atom; do
for p in perl url.pl &lt; $f; do wget$p
done
done

Once you’ve verified it, you can fetch more Atom pages.

AlexSchroeder 2011-11-16 19:53 UTC

Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit this page if you need to fix typos. You can subscribe to updates by email without leaving a comment.