Diary SiteMap RecentChanges About Contact Calendar

Search:

Matching Pages:

Comments on 2010-05-31 Blognapping

Last edit

Summary: If you're wondering how to do this… Assume you want to pull a copy of A Hamsterish Hoard of Dungeons and Dragons. Examine the source code and you'll find a link to the atom feed within blogger. . . .

Deleted:

Changed:

< ##&lt;link rel="alternate" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://hamsterhoard.blogspot.com/feeds/posts/default" /&gt;
< &lt;link rel="alternate" type="application/rss+xml" title="A Hamsterish Hoard of Dungeons and Dragons - RSS" href="http://hamsterhoard.blogspot.com/feeds/posts/default?alt=rss" /&gt;
< &lt;link rel="service.post" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://www.blogger.com/feeds/5373792969086619654/posts/default" /&gt;## &lt;- that's the one we're looking for!

to

> ##<link rel="alternate" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://hamsterhoard.blogspot.com/feeds/posts/default" />
> <link rel="alternate" type="application/rss+xml" title="A Hamsterish Hoard of Dungeons and Dragons - RSS" href="http://hamsterhoard.blogspot.com/feeds/posts/default?alt=rss" />
> <link rel="service.post" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://www.blogger.com/feeds/5373792969086619654/posts/default" />## <- that's the one we're looking for!


I’m surprised (and somewhat impressed) that it works for Wordpress blogs too.

Good work!

greywulf 2010-06-01 05:51 UTC


Now that the coding has been done, I need to do actual text assembly. Yikes! :)

AlexSchroeder 2010-06-01 17:52 UTC



AlexSchroeder
If you’re wondering how to do this… Assume you want to pull a copy of A Hamsterish Hoard of Dungeons and Dragons. Examine the source code and you’ll find a link to the atom feed within blogger. This is important, because it’ll provide us with the blog Id! In this case:

<link rel="alternate" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://hamsterhoard.blogspot.com/feeds/posts/default" /> <link rel="alternate" type="application/rss+xml" title="A Hamsterish Hoard of Dungeons and Dragons - RSS" href="http://hamsterhoard.blogspot.com/feeds/posts/default?alt=rss" /> <link rel="service.post" type="application/atom+xml" title="A Hamsterish Hoard of Dungeons and Dragons - Atom" href="http://www.blogger.com/feeds/5373792969086619654/posts/default" /> ← that’s the one we’re looking for!

Start with a small set: the last 100 entries:

for i in `seq 4`; do
  start=$((($i-1)*25+1))
  curl -o taichara-$i.atom "http://www.blogger.com/feeds/5373792969086619654/posts/default?start-index=$start&amp;amp;max-results=25"
done

Save it in a script such as download-atom.sh and run it using bash download-atom.sh. You’ll end up with the files taichara-1.atom taichara-2.atom taichara-3.atom taichara-4.atom.

Now take the Perl script from the main page and save it as url.pl. It will extract the page URLs from the Atom files.

for f in *.atom; do
  for p in `perl url.pl &lt; $f`; do
    wget $p
  done
done

Once you’ve verified it, you can fetch more Atom pages.

AlexSchroeder 2011-11-16 19:53 UTC


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit this page if you need to fix typos. You can subscribe to updates by email without leaving a comment.

A carefully selected set of emoji for you to copy and paste: • ✕ ✓ ✝ ☠ ☢ ☣ ⚛ ☭ ☮ ☯ ⚒ ⚓ ⚔⚙ ★ ☆ ✨ 🌟 🐣 🐤 🐥 🐦 🐧 🐨 🐷 🐻 🐼 🐢 🐝 🐛 🐙 🐒 🐌 🐋 👑 ✊ 👊 ✌ 👋 👌 👍 👎 👏 👸 🍵 🍷 👹 👺 👻 👽 👾 👿 💀 ❤ ❦ ♥ 💔 📓 📖 📜 📝 🔒 🔓 🔔 🔥 🔨 🔪 🔫 🔮 😁 😂 😃 😄 😅 😆 😉 😊 😋 😌 😍 😏 😒 😓 😔 😖 😘 😚 😜 😝 😞 😠 😡 😢 😣 😤 😥 😨 😩 😪 😫 😭 😰 😱 😲 😳 😵 😷 😸 😹 😺 😻 😼 😽 😾 😿 🙀 🙇 🙈 🙉 🙊 🚶 🚲 🚀 🚽

To save this page you must answer this question:

Please say HELLO.