Archiving and You

[Kourai]

Text (Mostly Tweet) Archiving and You​

Why?​

People delete things. To discuss news and drama accurately, it's important to have a record of who said what.

How?​

Archive sites:​

Archive sites take a snapshot of a web page as it was on the day you accessed it. Unlike screenshots, they are not easily faked.

I know of three archive sites. There may be others. Unfortunately all three sites have their own quirks when it comes to tweets, which are the majority of what should be archived.

• archive.today - Paste URL in. Press Save. The Firefox add-on in the post above reduces this to one button. When the page changes to archive.ph/wip/[characters], copy that URL and put it in your post. You do not need to wait for it to finish.
  • Only preserves embedded images as thumbnails
  • Does not preserve replies to a tweet (currently)
• ghostarchive.org - Paste URL in. Press Submit for Archival. When the page changes to ghostarchive.org/archive/[characters], copy that URL and put it in your post. You do not need to wait for it to finish.
  • Preserves replies to a tweet (currently)
  • Seems slower than archive.today
  • Seems to intend to preserve full embedded images in tweets, but gets stuck on loading
• archive.org - Click on Web. Paste URL into Save Page Now and press Save.
  • Has a process for takedown requests, so not the first choice. If you're paranoid that a page which has been deleted is only on archive.org, you can re-archive it using archive.today.
  • Useless for new tweets when I tested it
  • Very useful for edge cases like video, audio, Google Docs that require you to be signed in

Important embedded media, like images of ominous white pages with text, should be saved separately if possible.

All three sites can be searched for existing archives as well, provided you have a URL.

One weird trick (the TVA embed method):​

If you have a lot of tweets you want to archive, paste them all into a post on this forum and press Post. Then use the Archive Page button on the top right of the page. This opens a new archive.today tab. Edit that archive URL into your post. The embedded tweets will still be visible, even if the original tweets are deleted. However, anything in spoiler tags is not archived.

Screenshots:​

Archive sites are an independent record. Screenshots are not independent; they can be faked. But sometimes you have no choice (e.g. site that requires login), and in those cases screenshots are better than nothing.

Edge: ... (Menu) > Web Capture, or Ctrl+Shift+S - Capture Full Page is an option
Firefox: Right-click > Take Screenshot - Save Full Page is an option (but see the post below)
Chrome/Brave/etc. : GoFullPage add-on

I have not looked into mobile browser options. If nothing else, you can screenshot your phone screen and crop it down as needed.

 

[Porean]

I've been searching for a way to mass download videos and images from twitter for a while and i have now found a solution;

WFDownloader App - Free bulk image downloader and multi-purpose bulk downloader

Bulk download list of urls, generate sequential urls, download images, wallpapers, videos, documents, etc, from forums, open directories, search engines and supported sites such as instagram, twitter, pinterest, tumblr, artstation, deviantart, etc.

www.wfdownloader.xyz

Now, you might click that and say to yourself; "Porean is trying to give me a virus and make me pay to remove it via Google store gift cards."
To that i say; "fuck you bloody banchod bitch show bob and vagene"

But for real, it might look scummy but it works, and as a man with an Oshi who constantly posts drawings and I HAVE TO HAVE THEM ALL it's been very useful, not only does it let you download an entire Twitter accounts media tab, you can also use it on a specific hashtag, useful for art-tags you want to archive. You have to give the programme your cookies for twitter for it to work properly, if you don't know how it tells you in the programme itself.

I will attach an image here to show you my folder after I've downloaded Amiya's media tab and her fan-art hashtag;

As you can see it sorts via username if you tell it to, which i love, so i very much recommend this if you have a lot of images you wanna download from Vtubers you follow. It is slow if it has to search through many posts (Ami posts a LOT.) It also works on other sites, how-ever i have not used it on other sites so i cannot speak for it's efficacy there.

 

[GOD'S STRONGEST BUILDERBEAR]

Hate to bother you again, but is there anyway make the ytarchive setup work with member streams too? ytarchive is what I have been using. I setup yt-dlp, but I'd like another option just in case I fucked it up.

yes if you use

    --members-only
        Only download members-only streams. Can only be used with channel URLs
        such as /live, /streams, etc, and requires cookies.
        Useful when monitoring channels and you only want membership streams.

and

--cookies COOKIES_FILE
        Give a cookies.txt file that has your youtube cookies. Allows
        the script to access members-only content if you are a member
        for the given stream's user. Must be netscape cookie format.

 

[Willemshaven]

Spoiler: Original post

Im currently having this issue with one of Kuri Rinji's twitter thread that i have shared on the niji thread. I can archive but it doesnt load. I press "Archived page not showing up" but since it flagged as sensitive content for some reason, the screenshots wont show up. The archive basically become useless.

Well, I've tried archiving the Tweet on Firefox, making sure the unnecessary bits in the link are removed (hover here). Not only does the Tweet load on first try, the "Show" button fully works. (FYI, I've opened the archived page back in Chromium.)

​

LAST UPDATED on 19-2-2025
IMPORTANT UPDATE
Searching in Ghostarchive works without requiring the input to start with 'http' or 'https', so a Tweet ID from an embed is enough/ It's still good practice to embed the Tweet URL somewhere in your post in case this situation changes again. [Example]

Archiving Tweet threads using Ghostarchive reliably
Version 8 by Willemshaven - Appendix about Nitter​

Ghostarchive is currently the only remaining web archiver that can save entire Twitter threads and NSFW-marked Tweets, but it can be a crapshoot to successfully get it to archive Tweets.
That is unless you're using Firefox. It's by far the most reliable way to archive Tweets with Ghostarchive for some reason. FYI, my Firefox setup also has uBlock Origin and LocalCDN add-ons installed.
Despite it being very good tool, there're still a few important things and caveats to keep in mind:

• Remove the tracking parts and other unnecessary stuff in a URL when archiving. This example has the Tweet not show up unless you click "Archived page not showing up?" in the left menu, which is a limb mode with reduced functionality. Such as saving less of a Tweet thread. A proper example is something like this.
• Saving reply Tweets always cause the archived Tweet to not display unless you click the aforementioned "Archived page not showing up?" in the left menu. [Example of archived reply Tweet] [Example of archived "master" Tweet]
  NSFW-marked reply Tweets are sometimes an exception for some odd reason. [Unsuccessful attempt] (The "Tweet not available" is a unrelated issue.)
  • This has been resolved as of the switch to the x.com domain.
• NSFW-marked account pages also won't work without clicking "Archived page not showing up?". In that mode, the timeline won't be consistent with how it looks like when visiting using the original link.
• Direct archives to Tweets with videos that are too large/too long won't work at all, even when clicking the "not showing up" link. Tweets with smaller/shorter videos archive fine, but usually requires clicking the aforementioned link in the left menu. Archived Tweets with video Tweets inside the replies and Tweets quoting video Tweets usually also requires clicking the "not showing up" link. I did manage to get a Tweet with a video attachment, including the video itself, archived with no caveats.
  • As of the switch to the x.com domain, the caveats of the above cases no longer applies. [Other case]
• Direct archives of Premium Tweets (they display a "show more" link when having a reply Tweet open) is flakey. Sometimes it won't work at all. Sometimes it works perfectly. [Reply Tweet] You can archive the profile page of the Tweet OP, but it won't work without clicking the "not showing up" link. This isn't ideal as it only shows the first four Tweets in the feed and sometimes won't show the full Tweet anyway. The ideal result is to archive a reply Tweet and hope that you can click the original Tweet to show the full text. This example requires you to click the "not showing up" link, so it won't work.
  • As of the switch to the x.com domain, Premium Tweets have been consistently archiving without needing to click the "not showing up" link.
• Non-cropped Images AREN'T archived. Archiving a Tweet with a open media display like this will just archive the original Tweet. Unfortunately, there's no way to open a larger-sized image through a archived Tweet, so you will have to use "right click>save as" or tools like gallery-dl mentioned in the page prior.
  • On a similar note: videos are likely to not be archived either.
• When you try to save too much Tweets (or pages in general) at the time, which usually results in being offered a CAPTCHA queue, the reliability will drop off sharply.

And there you have it. You can now reliably utilize Ghostarchive to archive entire Tweet threads at a time. I will keep updating this post when new developments happen.

Spoiler: Changelog

Version 1
-Initial version

Version 2 (11-4-2024)
-Updated the situation on archiving Tweets with videos.
-Re-worded some of the items.
-Changed closing words.

Version 3 (12-4-2024)
-Added situation of archiving quote Tweets of Tweets with video's.

Status Update (14-4-2024)
-Added notice regarding Twitter archiving not working.
-Update ~8PM UTC: Twitter archives work again.

Version 4 (15-4-2024)
-Added example of a fully successful archive of a video Tweet. Revised images and videos item accordingly.
-Updated archiving status to edit time.

Version 5 (18-4-2024)
-Added another scenario where archiving won't work fully.

Status Update (19-4-2024)
-Added notice regarding Twitter archiving not working, again.

Status Update (20-4-2024)
-Added notice that archiving Twitter with Ghostarchive works again.

Version 6 (20-4-2024)
-Added link to my new Appendix about Nitter.
-Updated status on "read more" Tweets.
-Shortening of most "Archived page not showing up?" parts.

Status Update (24-4-2024)
-I went away and Twitter archiving broke again.
-It worked again since ~9PM UTC, but it's in CAPTCHA hell since ~11PM UTC.

Status Update (25-4-2024)
-I went away for a long time and Ghostarchive is no longer loading.

Status Update (26-4-2024)
-Ghostarchive is working again.

Update (01-5-2024)
-I won't update the status of Twitter archiving anymore. I will only update the guide itself from now on.

Version 7 (25-6-2024)
-As of the switch to x.com, Premium Tweets have been consistently archiving without needing to click the "not showing up" link.

Version 8 (12-8-2024)
-Archiving Tweets has become more relieable than ever before. Most of the caveats in the initial guide no longer applies.

Important Update (01-9-2024)
-Added notice that Tweet ID's aren't enough anymore to search on Ghostarchive.

Status Update (19-2-2025)
-The search function in Ghostarchive has been restored to the previous situation.