I ask only once a year: please help the Internet Archive today. Right now, we have a 2-to-1 Matching Gift Campaign, so you can triple your impact! Most can’t afford to give, but we hope you can. The average donation is $45. If everyone reading this chips in just $5, we can end this fundraiser today. All we need is the price of a paperback book to sustain a non-profit website the whole world depends on. We have only 150 staff but run one of the world’s top websites. We’re dedicated to reader privacy so we never track you. We never accept ads. But we still need to pay for servers and staff. I know we could charge money, but then we couldn’t achieve our mission: a free online library for everyone. This is our day. Today. To bring the best, most trustworthy information to every internet reader. I believe all of this is doable, if we pull together to create the internet as it was meant to be. The Great Library for all. The Internet Archive is a bargain, but we need your help. If you find our site useful, please chip in. Thank you.
—Brewster Kahle, Founder, Internet Archive
Dear Internet Archive Supporter,
I ask only once a year: please help the Internet Archive today. Right now, we have a 2-to-1 Matching Gift Campaign, so you can triple your impact!The average donation is $45. If everyone reading this chips in just $5, we can end this fundraiser today. All we need is the price of a paperback book to sustain a non-profit website the whole world depends on. We’re dedicated to reader privacy so we never track you. We never accept ads. But we still need to pay for servers and staff. I know we could charge money, but then we couldn’t achieve our mission. To bring the best, most trustworthy information to every internet reader. The Great Library for all. The Internet Archive is a bargain, but we need your help. If you find our site useful, please chip in. Thank you.
—Brewster Kahle, Founder, Internet Archive
Dear Internet Archive Supporter,
I ask only once a year: please help the Internet Archive today. Right now, we have a 2-to-1 Matching Gift Campaign, so you can triple your impact!The average donation is $45. If everyone reading this chips in just $5, we can end this fundraiser today. All we need is the price of a paperback book to sustain a non-profit website the whole world depends on. We’re dedicated to reader privacy so we never track you. We never accept ads. But we still need to pay for servers and staff. I know we could charge money, but then we couldn’t achieve our mission. To bring the best, most trustworthy information to every internet reader. The Great Library for all. The Internet Archive is a bargain, but we need your help. If you find our site useful, please chip in. Thank you.
—Brewster Kahle, Founder, Internet Archive
Dear Internet Archive Supporter,
I ask only once a year: please help the Internet Archive today. Right now, we have a 2-to-1 Matching Gift Campaign, so you can triple your impact!The average donation is $45. If everyone chips in just $5, we can end this fundraiser today. All we need is the price of a paperback book to sustain a non-profit library the whole world depends on. We’re dedicated to reader privacy. We never accept ads. But we still need to pay for servers and staff. I know we could charge money, but then we couldn’t achieve our mission. To bring the best, most trustworthy information to every internet reader. The Great Library for all. We need your help. If you find our site useful, please chip in.
—Brewster Kahle, Founder, Internet Archive
Thanks for donating. Would you consider becoming a monthly donor starting next month?
Monthly support helps ensure that anyone curious enough to seek knowledge will be able to
find it here. For free.
Together we are building the public libraries of the future.
Dump of Hacker News stories and comments up to 2014-05-29 From the HN post: Downloading All of Hacker News Posts and Comments https://news.ycombinator.com/item?id=7835605 http://shitalshah.com/p/downloading-all-of-hacker-news-posts-and-comments/ ( 1 reviews ) Topics: hackernews, archive, stories, comments
Dagobah is a large archive of ancient 4chan flash animations, dating all the way back to 2008 when the site was first founded. Anyone can upload files to this site. Because of it's 13099+ collection containing flash animations that date from 4chan's earliest history, the Bibliotheca Anonoma is conducting a contingency archival of the site. We used custom built Python scraping scripts to reduce strain on the server, and avoid the many pitfalls encountered by scraping an automatically generated...
I took the Reddit comment archive and converted all the JSON into one SQLite database using this program that I wrote: https://gist.github.com/ers35/3b615a75fa0ed5e6d5cc I ran a few tests to make sure the number of database rows matches the number of JSON records. "SELECT MAX(rowid) FROM comment" and "SELECT COUNT(id) FROM comment" both return 1659361605. This gives me some confidence as to the integrity of the dataset, but I cannot be 100% sure. The compressed size is 163G....
