Knowledgebase:
twitter.com: Advanced features
Posted by pspzockerscene psp, Last modified by pspzockerscene psp on 07 July 2023 10:30 AM

twitter.com: Advanced features

Our twitter.com Plugins support some advanced features which will be explained here.


Special URL parameters which can be used as abort conditions for profile crawling

  • maxitems -> Define how many tweets of a profile should be crawled at most
  • max_date -> Only crawl items until end of day before this date

Usage example:
If you want to crawl twitter.com/exampleProfile and only want to crawl the first 50 items, add it like this:
twitter.com/exampleProfile?maxitems=50

Of course you can include both abort conditions at the same time like this:
twitter.com/exampleProfile?max_date=2022-01-15&maxitems=50
This will crawl up to 50 items or tweets until max posted date of 2022-01-14 11:59PM.
The abort condition which is reached first will abort the crawl process.


Special plugin properties that e.g. for custom filenames via Packagizer

Property key Description Example
username Profile name of added profile to crawl / owner of added tweet. exampleusername
date Date when content was posted in form yyyy-MM-dd 2022-01-17
date_timestamp Timestamp in milliseconds of date when content was posted. 1688670637000
mediaindex

Index of media in context of single tweet.

Useful when adding tweets containing multiple media items.

0
media_count

Total number of media items which the tweet has [regardless of how many were returned].

For example 2 images, 1 thumbnail and 1 video -> That number will be 4

4
mediaid Internal id of photo/video 1234561234561234561
bitrate Bitrate of crawled video objects 832000
tweetid id of the related tweet 1234567890123456789
tweet_text Text of tweet containing crawled media example
crawlerfilename Filename set by crawler plugin.
Available for all items added via crawler.
example.mp4
type Type of the added item.
Only available for items processed via crawler.
Possible values: video, photo, text
photo, text, video
retweet Boolean: true if that item is a retweet. False or null if it is a normal tweet. true, false, null
reply Boolean: true if that item is a reply to another tweet. False or null if it is not. true, false, null
crawlerfilename

DEPRECATED: Full filename which is set in crawler.

Only available for older items.

somestring.mp4
related_original_filename

Original filename of item related to this item.

This is typically only available for .txt files with tweet post content.

somestring.mp4, somestring.jpg