You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hydrator doesn't do that currently, but that would be a good option to add potentially. If you need to do it now you can use twarc's conversation command.
Actually I am trying to identify some good case studies for the conversation threads.
I have 33 .csv files (each around 300-500MB) and I would like to reconstruct all the threads (to better identify the case studies). Do you know how I can do this?
Do you have access to the Academic Research product track, which allows searching the historical archive?
In theory it ought to be possible if you extract the tweet ids from your CSVs into a file e.g ids.txt. And then run twarc conversations ids.txt --archive conversations.json to collect all the threads. It could take a while depending on the sizes of the threads you encounter. But these are all questions for the twarc issue tracker I guess.
If you are studying or working at a university you can apply. The main difference is that you can access 10 million tweets a month from Twitter's V2 API (usually limited to 100,000/month). The V2 API includes things like reply_count for tweets, as well as the conversation_id for a tweet which lets you easily collect all the tweets in a thread. And most importantly, Academic Research track lets you search the full archive of tweets rather than just the last week.
Activity
edsu commentedon Aug 24, 2021
Hydrator doesn't do that currently, but that would be a good option to add potentially. If you need to do it now you can use twarc's
conversationcommand.mcolemann commentedon Aug 24, 2021
Thank you very much!
Is twarc compatible with the .csv files from the Hydrator?
edsu commentedon Aug 24, 2021
It depends on what you are doing. Do you want to get all the conversation threads in the CSV file you generated?
mcolemann commentedon Aug 24, 2021
Actually I am trying to identify some good case studies for the conversation threads.
I have 33 .csv files (each around 300-500MB) and I would like to reconstruct all the threads (to better identify the case studies). Do you know how I can do this?
edsu commentedon Aug 24, 2021
Do you have access to the Academic Research product track, which allows searching the historical archive?
In theory it ought to be possible if you extract the tweet ids from your CSVs into a file e.g
ids.txt. And then runtwarc conversations ids.txt --archive conversations.jsonto collect all the threads. It could take a while depending on the sizes of the threads you encounter. But these are all questions for the twarc issue tracker I guess.mcolemann commentedon Aug 25, 2021
Unfortunately I don't have (I think so) access to the Academic Research product track...
How can I have access to it?
Thanks a lot! Then I will open an issue for twarc.
edsu commentedon Aug 25, 2021
If you are studying or working at a university you can apply. The main difference is that you can access 10 million tweets a month from Twitter's V2 API (usually limited to 100,000/month). The V2 API includes things like reply_count for tweets, as well as the conversation_id for a tweet which lets you easily collect all the tweets in a thread. And most importantly, Academic Research track lets you search the full archive of tweets rather than just the last week.