Simple Tweets collection using Twitter’s Stream API and Java using Twitter4j

Fairly simple tweet collection (you will need to add to file/database/other storage) using Twitter’s stream API and Twitter4j

You need to add the Twitter4j core and stream Jars to your java application & you need to add in your own Twitter application tokens and user tokens (you should change this to not hardcode in the user tokens)

package com.crowley.simplestream;

import twitter4j.FilterQuery;
import twitter4j.Status;
import twitter4j.StatusDeletionNotice;
import twitter4j.StatusListener;
import twitter4j.TwitterStream;
import twitter4j.TwitterStreamFactory;
import twitter4j.User;
import twitter4j.conf.ConfigurationBuilder;

public class SimpleStream {
    public static void main(String[] args) {
        ConfigurationBuilder cb = new ConfigurationBuilder();
        cb.setDebugEnabled(true);
        cb.setOAuthConsumerKey("*************");
        cb.setOAuthConsumerSecret("*************");
        cb.setOAuthAccessToken("*************");
        cb.setOAuthAccessTokenSecret("*************");

        TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();

        StatusListener listener = new StatusListener() {

            @Override
            public void onException(Exception arg0) {
                // TODO Auto-generated method stub

            }

            @Override
            public void onDeletionNotice(StatusDeletionNotice arg0) {
                // TODO Auto-generated method stub

            }

            @Override
            public void onScrubGeo(long arg0, long arg1) {
                // TODO Auto-generated method stub

            }

            @Override
            public void onStatus(Status status) {
                User user = status.getUser();
                
                // gets Username
                String username = status.getUser().getScreenName();
                System.out.println(username);
                String profileLocation = user.getLocation();
                System.out.println(profileLocation);
                long tweetId = status.getId(); 
                System.out.println(tweetId);
                String content = status.getText();
                System.out.println(content +"\n");

            }

            @Override
            public void onTrackLimitationNotice(int arg0) {
                // TODO Auto-generated method stub

            }

        };
        FilterQuery fq = new FilterQuery();
    
        String keywords[] = {"ireland"};

        fq.track(keywords);

        twitterStream.addListener(listener);
        twitterStream.filter(fq);  

    }
}

165 Responses

  1. Adel October 18, 2012 / 08:45

    Very nice exemple thank’s so much

    • Navindran April 25, 2014 / 04:42

      twitterStream.addListener(listener) method not found error

      • Easwar October 17, 2014 / 16:34

        Same problem here. How o fix it?

        • DCrowley October 17, 2014 / 16:39

          Make sure you have the Twitter4j jars added to the project

          • pratik December 30, 2014 / 11:03

            I am getting the same error even after adding jars. Any clue?

  2. franjo December 28, 2012 / 14:42

    Great code!. Do you know a way to get ‘all’ tweets that have a certain hashtag? I mean, already tweeted, not ‘on the fly’ like this example.
    Thanks.

    • DCrowley January 5, 2013 / 16:38

      As far as I know – no. You can get recent ones but not ones in the past. Not without going to one of the companies that allow you access to the Twitter stream fully like Gnip

  3. neni February 7, 2013 / 14:13

    i have a problem
    Exception in thread “main” java.lang.IllegalArgumentException: Invalid access token format.
    at twitter4j.auth.AccessToken.(AccessToken.java:50)
    at twitter4j.auth.OAuthAuthorization.(OAuthAuthorization.java:69)
    at twitter4j.auth.AuthorizationFactory.getInstance(AuthorizationFactory.java:40)
    at twitter4j.TwitterStreamFactory.getInstance(TwitterStreamFactory.java:77)
    at coba2.SimpleStream.main(SimpleStream.java:30)
    Java Result: 1

    • DCrowley February 7, 2013 / 14:59

      I’d guess that you don’t have the tokens set for your application – you can get tokens when you set up an application on Twitter’s development pages

  4. TropinR April 18, 2013 / 19:57

    TY so much But….
    How to use 2 filters?

    • DCrowley April 20, 2013 / 13:10

      Two filters?
      You can track two words by String keywords[] = {“ireland”, “weather”}; or do you mean track “ireland” && “weather”?

  5. Vijay Baswal April 25, 2013 / 10:53

    I’m trying to run this code having twitter4j-core-2.2.6.jar and twitter4j-stream-2.2.4.jar but this gives me error saying :
    1. Status Listener is not set StatusListener listener = new StatusListener()
    2. gives error at twitterStream.addListener(listener); – saying not a method of twitterStream.

    Please help as i’m jostling with this problem since last 2 days. Thank you.

    • DCrowley May 7, 2013 / 21:05

      I’ll have a look tomorrow at my code and see if anything has changed within twitter4j to cause these errors

      DC

    • DCrowley May 8, 2013 / 18:27

      Maybe the mismatch in versions is the problem? I am using 2.2.5 core and stream. Maybe use 2.2.6 core and stream?

      DC

  6. Roberto Pérez May 6, 2013 / 07:19

    Hi,
    This example allows to filter tweets in real time, but how search for previous tweets
    (using streaming interface)?
    Thanks

  7. Thiago May 8, 2013 / 18:54

    Nice man. Very good, and it works perfectly. Thank you very much.

  8. Gabriel M. Maia May 15, 2013 / 18:55

    I am using twitter4j 3.0.3 and the classes StatusListener; TwitterStream;TwitterStreamFactory; are not getting recognized what wrong am i doing?

    • DCrowley May 16, 2013 / 11:42

      Hi Gabriel,

      I changed my jars to Twitter4j 3.03 core and stream and it worked fine – I did have to add an unimplemented method for StatusListener but that was the only change I made,

      Regards,

      DC

  9. muthu May 31, 2013 / 03:51

    I am getting the error
    HTTP ERROR: 401
    Problem accessing ‘/1.1/statuses/filter.json’. Reason:
    Unauthorized

    TwitterStreamFactory twitterStreamFactory = new TwitterStreamFactory(configuration);
    TwitterStream twitterStreamClient = twitterStreamFactory.getInstance();
    FilterQuery filterQuery = new FilterQuery(followersId);
    twitterStreamClient.addListener(statusListener);
    twitterStreamClient.filter(filterQuery);

    But the same Configuration is working for twitter API

    • DCrowley May 31, 2013 / 08:39

      a 401 means you can connect but you haven’t authenticated? Did you enter in your own secret keys?

      DC

  10. Karunjay Anand May 31, 2013 / 05:39

    Hello David,

    Every time i am getting this in eclipse console
    [Fri May 31 11:01:02 IST 2013]Establishing connection.
    [Fri May 31 11:01:22 IST 2013]connect timed out
    [Fri May 31 11:01:22 IST 2013]Waiting for 16000 milliseconds
    [Fri May 31 11:01:38 IST 2013]Establishing connection.
    [Fri May 31 11:01:58 IST 2013]connect timed out
    [Fri May 31 11:01:58 IST 2013]Waiting for 16000 milliseconds
    [Fri May 31 11:02:14 IST 2013]Establishing connection.
    [Fri May 31 11:02:34 IST 2013]connect timed out
    [Fri May 31 11:02:34 IST 2013]Waiting for 16000 milliseconds
    [Fri May 31 11:02:50 IST 2013]Establishing connection.
    [Fri May 31 11:03:11 IST 2013]connect timed out
    [Fri May 31 11:03:11 IST 2013]Waiting for 16000 milliseconds
    [Fri May 31 11:03:27 IST 2013]Establishing connection.
    [Fri May 31 11:03:47 IST 2013]connect timed out

    Help me to fix this.

    • DCrowley May 31, 2013 / 08:38

      Hi,

      Make sure you have internet?! 😛 and make sure your time on your computer is right as often it won’t allow a connection if it is wrong. Also make sure you are connecting to the right addresses

      DC

    • Lalitha October 28, 2013 / 10:47

      Hi Karunjay Anand, did you found any fix for this.
      Even am facing with the same issue. I think its because of proxy.

      • Lalitha October 28, 2013 / 11:04

        Hurray!!! i fixed it by setting the proxy 🙂

        • jx8772 March 13, 2014 / 16:59

          Hello Lalitha,

          I am using a school ubuntu server to collect tweets, and I have the same problem. Can you specify how you set the proxy? Thanks!

    • adarsh January 21, 2014 / 10:06

      You would be in corporate network. Switch to Simple internet.

  11. Rashmi Rai June 14, 2013 / 06:48

    HI HOW TO ADD BOUNDING BOX WITH TRACK.???

    • DCrowley June 19, 2013 / 21:04

      Do you mean a geolocated bounding box?

  12. Casey French June 18, 2013 / 06:39

    Hey just used this as a template to start a project I’ve been wanting to do forever – found a site that charges at least 50 a month (at min) and you can only search for two hashtags – this option is infinitely more flexible and suitable for a custom application – just wanted to say thanks for the post, so awesome and I’ll check back if anyone needs help with theirs
    (was not paid to write this lol)

  13. Spyros June 18, 2013 / 10:44

    many thanks!!! 🙂

  14. RG July 17, 2013 / 15:38

    Is there a way to retrieve tweets based on multiple keywords, like –
    String keywords[] = {“ireland”,”belgium”,”london”};
    fq.track(keywords);

    Pls let me know? I’m struggling to achieve this using Streaming API.

    • DCrowley July 17, 2013 / 17:15

      Yes – a comma separated list should work like

      String keywords[] = {“ireland”, “kerry”};

      But the tweets don’t have to contain both Ireland & Kerry

      • Gaston September 7, 2013 / 19:57

        Any idea how to search for tweets that contain both Ireland & Kerry?

      • Gaston Martin September 9, 2013 / 20:23

        Any idea how to make it search for both Ireland and Kerry?

        • DCrowley September 9, 2013 / 20:44

          From looking at this -https://dev.twitter.com/docs/streaming-apis/parameters#track I am not sure if you can – you might have to use the search API. I’ll have to look into it more to be sure

  15. RG July 17, 2013 / 17:41

    Thank you so much!!!

  16. Matthew Pintor July 19, 2013 / 08:10

    How does FilterQuery filter the streamed data? Coz every post returned does not contain my filter strings. And why does it sometimes return some post with ????? user and ????????? post?

    • DCrowley July 19, 2013 / 09:48

      It isn’t a logical AND – it returns any tweet with string1 or string2 or string3.

      ???? could be an encoding problem?

  17. Brendon Unland July 22, 2013 / 12:38

    Thanks DC for this code! I would like to simply get a specific user’s tweets from the streaming API. How would I change the code to do so? Many tks!

    • DCrowley July 30, 2013 / 09:58

      There used to be a method called “getUserTimeLine” but might be deprecated?

  18. baran July 29, 2013 / 23:40

    Thank you.
    I need to have something like a big corpus from old tweets and then streaming new tweets .
    I am confused .I looked it up but it seems that it needs some permissions.am I right?

    • DCrowley July 30, 2013 / 09:56

      Not sure what you mean by permissions? permission to publish the tweets?

      Or do you mean authenticate your application/user?

  19. baran ghasemi July 30, 2013 / 20:40

    I am not sure too.But I think some kind of authentication from twitter website.
    anyways,the only thing I want to do is collect old tweets and streaming new ones which have a special hashtag and make a simple text file from all of them.can you save my life please?
    Thank you so much

    • DCrowley July 31, 2013 / 15:11

      That is more difficult – you will need to keep track of “where” you are – so you don’t store duplicates. For that you might just use search and not use the streaming API at all.

      • baran ghasemi July 31, 2013 / 20:06

        Could you be more specific ?I am new to these stuff.Could you send me some sort of code?
        Thank you so much

  20. shamsan August 2, 2013 / 19:27

    it works, awesome. thanks for sharing the code 🙂

  21. baran ghasemi August 8, 2013 / 02:13

    Thank you for your code.I tried your code and the output is weird .here is a sample of output:

    19:12:05.229 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – User-Agent: twitter4j http://twitter4j.org/ /3.0.3
    19:12:05.229 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-Version: 3.0.3
    19:12:05.230 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Connection: close
    19:12:05.230 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Post Params: count=0&track=happy&stall_warnings=true
    19:12:05.300 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:05.300 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – Waiting for 16000 milliseconds
    19:12:05.300 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Waiting for 16000 milliseconds]
    19:12:21.301 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:21.301 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] INFO twitter4j.TwitterStreamImpl – Establishing connection.
    19:12:21.302 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Establishing connection]
    19:12:21.302 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Request:
    19:12:21.302 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – POST https://stream.twitter.com/1.1/statuses/filter.json
    19:12:21.303 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth base string: POST&https%3A%2F%2Fstream.twitter.com%2F1.1%2Fstatuses%2Ffilter.json&count%3D0%26oauth_consumer_key%3DVzdS7zf33IyW8IdFZYhHg%26oauth_nonce%3D1768804082%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1375927941%26oauth_token%3D1594456190-lXS9kTMkl0x7ttPxcr0RwIEjXix9k3g2xaGikiy%26oauth_version%3D1.0%26stall_warnings%3Dtrue%26track%3Dhappy
    19:12:21.304 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth signature: y6LmSWcLBff6pPxUQ5RZz636T0w=
    19:12:21.304 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Authorization: *******************************************************************************************************************************************************************************************************************************************************************************
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-URL: http://twitter4j.org/en/twitter4j-3.0.3.xml
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client: Twitter4J
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Accept-Encoding: gzip
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – User-Agent: twitter4j http://twitter4j.org/ /3.0.3
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-Version: 3.0.3
    19:12:21.305 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Connection: close
    19:12:21.306 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Post Params: count=0&track=happy&stall_warnings=true
    19:12:21.377 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:21.377 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – Waiting for 16000 milliseconds
    19:12:21.377 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Waiting for 16000 milliseconds]
    19:12:37.378 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:37.378 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] INFO twitter4j.TwitterStreamImpl – Establishing connection.
    19:12:37.378 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Establishing connection]
    19:12:37.379 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Request:
    19:12:37.379 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – POST https://stream.twitter.com/1.1/statuses/filter.json
    19:12:37.380 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth base string: POST&https%3A%2F%2Fstream.twitter.com%2F1.1%2Fstatuses%2Ffilter.json&count%3D0%26oauth_consumer_key%3DVzdS7zf33IyW8IdFZYhHg%26oauth_nonce%3D1789025394%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1375927957%26oauth_token%3D1594456190-lXS9kTMkl0x7ttPxcr0RwIEjXix9k3g2xaGikiy%26oauth_version%3D1.0%26stall_warnings%3Dtrue%26track%3Dhappy
    19:12:37.381 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth signature: g+U46S4Wet5ZN8Ru6BrY7Zo+rkY=
    19:12:37.381 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Authorization: ***********************************************************************************************************************************************************************************************************************************************************************************
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-URL: http://twitter4j.org/en/twitter4j-3.0.3.xml
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client: Twitter4J
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Accept-Encoding: gzip
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – User-Agent: twitter4j http://twitter4j.org/ /3.0.3
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-Version: 3.0.3
    19:12:37.382 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Connection: close
    19:12:37.383 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Post Params: count=0&track=happy&stall_warnings=true
    19:12:37.452 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:37.452 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – Waiting for 16000 milliseconds
    19:12:37.452 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Waiting for 16000 milliseconds]
    19:12:53.453 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:53.453 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] INFO twitter4j.TwitterStreamImpl – Establishing connection.
    19:12:53.453 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Establishing connection]
    19:12:53.454 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Request:
    19:12:53.454 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – POST https://stream.twitter.com/1.1/statuses/filter.json
    19:12:53.455 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth base string: POST&https%3A%2F%2Fstream.twitter.com%2F1.1%2Fstatuses%2Ffilter.json&count%3D0%26oauth_consumer_key%3DVzdS7zf33IyW8IdFZYhHg%26oauth_nonce%3D-171852571%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1375927973%26oauth_token%3D1594456190-lXS9kTMkl0x7ttPxcr0RwIEjXix9k3g2xaGikiy%26oauth_version%3D1.0%26stall_warnings%3Dtrue%26track%3Dhappy
    19:12:53.456 [Twitter Stream consumer-1[Establishing connection]] DEBUG twitter4j.auth.OAuthAuthorization – OAuth signature: qnPDnV9plkCmdzZzoTQa/7EDyWk=
    19:12:53.456 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Authorization: *********************************************************************************************************************************************************************************************************************************************************************************
    19:12:53.456 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-URL: http://twitter4j.org/en/twitter4j-3.0.3.xml
    19:12:53.456 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client: Twitter4J
    19:12:53.457 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Accept-Encoding: gzip
    19:12:53.457 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – User-Agent: twitter4j http://twitter4j.org/ /3.0.3
    19:12:53.457 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – X-Twitter-Client-Version: 3.0.3
    19:12:53.457 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Connection: close
    19:12:53.457 [Twitter Stream consumer-1[Establishing connection]] DEBUG t.internal.http.HttpClientImpl – Post Params: count=0&track=happy&stall_warnings=true
    19:12:53.528 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    19:12:53.529 [Twitter Stream consumer-1[Establishing connection]] INFO twitter4j.TwitterStreamImpl – Waiting for 16000 milliseconds
    19:12:53.529 [Twitter Stream consumer-1[Waiting for 16000 milliseconds]] DEBUG twitter4j.TwitterStreamImpl – Twitter Stream consumer-1[Waiting for 16000 milliseconds]

    • DCrowley August 8, 2013 / 21:14

      Exit your IDE and run it again – seen this error message when you have already made a connection and it didn’t close correctly

      • baran ghasemi August 8, 2013 / 21:18

        Tank you for your fast reply.

  22. Manish Kamboj August 21, 2013 / 05:34

    Hi DC,
    I tried the code and I am getting following result:

    [Wed Aug 21 10:58:49 IST 2013]Establishing connection.
    [Wed Aug 21 10:58:53 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:58:53 IST 2013]Waiting for 250 milliseconds
    [Wed Aug 21 10:58:53 IST 2013]Establishing connection.
    [Wed Aug 21 10:58:54 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:58:54 IST 2013]Waiting for 500 milliseconds
    [Wed Aug 21 10:58:55 IST 2013]Establishing connection.
    [Wed Aug 21 10:58:55 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:58:55 IST 2013]Waiting for 1000 milliseconds
    [Wed Aug 21 10:58:56 IST 2013]Establishing connection.
    [Wed Aug 21 10:58:57 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:58:57 IST 2013]Waiting for 2000 milliseconds
    [Wed Aug 21 10:58:59 IST 2013]Establishing connection.
    [Wed Aug 21 10:59:00 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:59:00 IST 2013]Waiting for 4000 milliseconds
    [Wed Aug 21 10:59:04 IST 2013]Establishing connection.
    [Wed Aug 21 10:59:05 IST 2013]Extensions not allowed in v2 certificate
    [Wed Aug 21 10:59:05 IST 2013]Waiting for 8000 milliseconds

    Please Help!

    • DCrowley August 21, 2013 / 09:54

      That looks like you are trying to use HTTPS?

  23. Amitkumar September 24, 2013 / 08:47

    Thanks a lot….it helps me

  24. James Barret September 27, 2013 / 18:44

    Hi David Crowley,

    I wanna to get the tweets from people following with stream instead of keywords. How to do? Thank you.

  25. Sagun October 20, 2013 / 05:50

    How can we get the latest tweet from public twitter profile using the streaming api? How do i filter content to get the tweets from a specific public twitter profile.

  26. Lalitha October 28, 2013 / 11:38

    Hi, Thanks a lot for your code.
    Any idea on how to get the tweets using # tag?

    • DCrowley October 29, 2013 / 20:58

      this code should work with #tags as well as without #hashtags

      • Lalitha November 6, 2013 / 09:50

        Yup, it works. Problem was with the #tag i gave to search. At that moment, there were no recent tweets related to that #tag. Hence, thought it doesn’t work with hastags.
        Thanks 🙂

  27. Mahesh October 28, 2013 / 14:19

    Hi David
    That’s a great piece of code . I am completely new to this . I tried the code . I wanted to stop the code after a particular interval of time. But unable to do so. Any help is appreciated.
    Thanks

    • DCrowley October 29, 2013 / 20:53

      You could loop until you have a certain amount of tweets? or you could use time to do it for so many seconds. Depends on what you are trying to do

  28. su October 29, 2013 / 00:33

    Hello David,

    there is a compilation error with your code:

    SimpleStream.java:22: error: is not abstract and does not override abstract method onStallWarning(StallWarning) in StatusListener
    StatusListener listener = new StatusListener() {
    ^
    Can it be fixed?

    • DCrowley October 29, 2013 / 20:59

      haven’t seen this error – what version of Twitter4j are you using?

  29. su October 29, 2013 / 22:13

    I tried 3.0.3, 3.0.4 and 3.0.5, all came with this error. I use javac compiler (java 7 sdk) on CentOS 6.4

    • DCrowley October 30, 2013 / 18:02

      Using Java 1.6/JRE7 and 3.03 using eclipse 3.7 working fine – just tried it there.

  30. Saad October 30, 2013 / 09:39

    thanks nice code

  31. Saad October 30, 2013 / 10:55

    is there any way to get tweets from specific user ?

    • DCrowley October 30, 2013 / 18:14

      Do you want it for a logged in user? or any user?

  32. Rob November 7, 2013 / 23:56

    Hi. I’m very comfortable with Java and have done some successful testing with Twitter4J from my IDE, but I’m at a loss when searching for resources on how to deploy a Java that could monitor a twitter feed. Can you point me to some useful resources (i.e. where/how to host, deployment how-tos)? Thanks

  33. Dale Leonard December 11, 2013 / 23:10

    Hello:

    Have you found any way to run multiple listeners with just one TwitterStream connection. ?

    Trying to creatively get around some of the limitations that Twitter is imposing, such as only one concurrent TwitterStream connection, only 400 keywords via the fq.track(keywords) method, etc.

    I tried creating 2 separate listeners in one method but since they are both connected to the same TwitterStream it treated it as if it were two separate connections and disconnects.

    Im beginning to work on a multi-threaded approach to see it that has any merit.
    Just curious if you have any ideas.

    Thanks

  34. Pourneema January 27, 2014 / 11:07

    Hey, thanks for your code. When I ran your code am getting the error as
    ” sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target”
    Am using twitter4j-3.0.5, NetBeansIDE7.4, https internet connection in Windows7. Could you pls help me to resolve this issue?

  35. Boris April 2, 2014 / 18:45

    Hey David i just copied your code… But i still get this error below… It looks like, i miss some methods in my Jar-Files. Can u check this out and tell me what i do wrong? If it’s juste the Jar-Files… Can send my the right one.
    I use :
    twitter4j-stream-3.0.3.jar
    twitter4j-core-4.0.1.jar

    Exception in thread “main” java.lang.NoClassDefFoundError: twitter4j/internal/http/HttpClientWrapperConfiguration
    at twitter4j.TwitterStreamFactory.(TwitterStreamFactory.java:40)
    at twitter.SimpleStream.main(SimpleStream.java:23)
    Caused by: java.lang.ClassNotFoundException: twitter4j.internal.http.HttpClientWrapperConfiguration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    … 2 more

    • DCrowley April 3, 2014 / 15:28

      At a guess I would say that using twitter4j-stream-3.0.3.jar
      twitter4j-core-4.0.1.jar non matching jars would be the problem if you download Twitter4j then the jars should be of the same version?

      • Boris April 4, 2014 / 09:00

        I guessed it too, but i also tried each at the time… Then i got other issues: it doesn’t recognize some imported sources. Can u send me the Referenced Libraries u used to compile your code? It’ll be nice, if i can get your code running.

        thanks.

  36. Boris April 4, 2014 / 11:06

    I have got this problem fixed by using these 2 Jar-Files:
    – twitter4j-stream-3.0.3.jar
    – twitter4j-core-3.0.5.jar

    Now i get some Twitter-API struggles. And i don’t know what wrong with that problem… It looks like my OAuthConsumerKey, OAuthConsumerSecret, … are false. But that not true, then i used them with the Rest-API and they worked very well.

    Please help!

    [Fri Apr 04 12:51:54 CEST 2014]Establishing connection.
    [Fri Apr 04 12:51:57 CEST 2014]401:Authentication credentials (https://dev.twitter.com/pages/auth) were missing or incorrect. Ensure that you have set valid consumer key/secret, access token/secret, and the system clock is in sync.
    \n\n\nError 401 Unauthorized

    HTTP ERROR: 401
    Problem accessing ‘/1.1/statuses/filter.json’. Reason:
    Unauthorized

    [Fri Apr 04 12:51:57 CEST 2014]Waiting for 10000 milliseconds
    [Fri Apr 04 12:52:07 CEST 2014]Establishing connection.
    [Fri Apr 04 12:52:07 CEST 2014]401:Authentication credentials (https://dev.twitter.com/pages/auth) were missing or incorrect. Ensure that you have set valid consumer key/secret, access token/secret, and the system clock is in sync.
    \n\n\nError 401 Unauthorized

    HTTP ERROR: 401
    Problem accessing ‘/1.1/statuses/filter.json’. Reason:
    Unauthorized

    [Fri Apr 04 12:52:07 CEST 2014]Waiting for 20000 milliseconds

  37. Navindran April 23, 2014 / 09:23

    I am getting the below two lines

    [Wed Apr 23 14:54:48 IST 2014]Will use class twitter4j.internal.logging.StdOutLoggerFactory as logging factory.
    [Wed Apr 23 14:54:48 IST 2014]Will use twitter4j.internal.http.HttpClientImpl as HttpClient implementation.

    • DCrowley April 23, 2014 / 21:29

      That seems ok? Does it run after that?

      • Navindran April 25, 2014 / 04:43

        I dont get any tweets

        • DCrowley April 29, 2014 / 12:49

          Have you done a manual search on Twitter.com to see there is related tweets to your search?

  38. Pavan Atri April 29, 2014 / 18:46

    Thanks for the valuable information!
    I want to know we can limit the number of tweets we get by setting the listener. I mean for a given filter, the onStatus method keeps getting called infinitely. Can we like limit the onStatus method being called for a fixed number of times without using System.exit? Thanks!

    • DCrowley April 29, 2014 / 19:42

      You could build some kind of Tweet counter and stop the listener after that amount. Have never tried it. You could also use a Timer and only collect for a set time scale.

  39. Amrutha Benny June 20, 2014 / 18:36

    Respected Sir,

    First of all thanks for your code. But I have a problem. When I run the program it shows exceptions like

    Exception in thread “main” java.lang.NoClassDefFoundError: SimpleStream (wrong name: com/crowley/simplestream/SimpleStream)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:472)

    I used twitter4j. Please help me.

    • DCrowley June 22, 2014 / 12:53

      Looking at this line Exception in thread “main” java.lang.NoClassDefFoundError: SimpleStream (wrong name: com/crowley/simplestream/SimpleStream) i wonder is this the problem?

  40. sravanthi June 21, 2014 / 09:11

    HI
    thanks for the code, could you just explain in detail, how to link the twitter4j files to the program

    • DCrowley June 22, 2014 / 12:52

      You need to add the twitter4j jars to the project

  41. aniket June 27, 2014 / 05:26

    i want change the keyword after every 600 seconds but i m not able to do it.Whenever i cleanup and shutdown streamfactory after every keyword it says inflator has been closed.
    It only able to fetch 1st keyword.

  42. sara July 1, 2014 / 07:41

    I am getting Estabilishing Connection. Waiting for 16000 milliseconds continously.
    How to bypass proxy in Ubantu ?
    Please experts share your ideas…..

  43. niceguy August 13, 2014 / 16:09

    Thanks for this great tutorial

  44. Satish September 19, 2014 / 05:52

    Hi

    This was an awesome tutorial for newbies like me.
    I want to store the output Tweet stream to a JSON file,could you please share any code for that?

    Thanks in Advance!

  45. Danae Rikou (@DanaeRikou) October 7, 2014 / 10:35

    Hi
    thanks for this great post. I have a little different problem here. I am trying to get all tweets that are streaming and print it in real time using twitter4j. This means that i should use the listener without any filter, but i don t know how to implement this. Can you help me?

  46. Charlotte October 16, 2014 / 04:26

    Hi, thanks for the codes… But there’re some problems with my output like:

    [Thu Oct 16 12:30:39 SGT 2014]Establishing connection.
    [Thu Oct 16 12:30:41 SGT 2014]401:Authentication credentials (https://dev.twitter.com/pages/auth) were missing or incorrect. Ensure that you have set valid consumer key/secret, access token/secret, and the system clock is in sync.
    \n\n\nError 401 Unauthorized

    HTTP ERROR: 401
    Problem accessing ‘/1.1/statuses/filter.json’. Reason:
    Unauthorized

    I’ve applied for the keys and tokens yesterday, but I still get a 401. Can you please help me figure it out? Thank you.

    • DCrowley October 17, 2014 / 14:45

      The obvious things to check are that the keys are in the right place and that your system clock is correct?

      • manu May 14, 2019 / 08:48

        sir i have a problem
        19/05/14 14:15:43 INFO twitter4j.TwitterStreamImpl: Establishing connection.
        19/05/14 14:15:44 INFO twitter4j.TwitterStreamImpl: https://stream.twitter.com/1/statuses/filter.json
        19/05/14 14:15:44 INFO twitter4j.TwitterStreamImpl: Waiting for 16000 milliseconds

        this is showing from longer time
        please help

  47. sowjanya January 7, 2015 / 15:28

    can anyone tell me how to add jar files and setting path through command prompt..please give me the clear steps about what to do after downloading twitter4j 4.0.2 API.Thanks!

  48. Lubu January 12, 2015 / 11:04

    I use your code, but have this error:

    Exception in thread “main” java.lang.NoClassDefFoundError: twitter4j/internal/http/HttpClientWrapperConfiguration
    at twitter4j.TwitterStreamFactory.(TwitterStreamFactory.java:40)
    at javaapplication3.JavaApplication3.main(JavaApplication3.java:35)
    Caused by: java.lang.ClassNotFoundException: twitter4j.internal.http.HttpClientWrapperConfiguration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    … 2 more
    Java Result: 1

    The line which caused the error is:
    TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();

    Thank you very much!

  49. Pritam February 9, 2015 / 10:36

    How can i fetch the images from timeline.

  50. Nilang March 26, 2015 / 12:12

    Hey, great code helped me a lot! I wanted to know that is there any way to insert these incoming tweet stream to directly into a mysql table using jdbc?

  51. Vishwanath Patil April 3, 2015 / 10:57

    Hello DC,

    Any thoughts on how to stream tweets with just the geo-location (geo-boxing) and not using the search keywords inside the track method.

    Basically want to stream tweets from a particular country

  52. Aravind June 12, 2015 / 07:40

    How can i save the output as a .txt file and use it rather than seeing in console..I am new to this help me.

  53. Aravind June 15, 2015 / 07:42

    Thank you.But When i run your above program and save it in a file it overwrites the previous tweets.Any Solution for that?? I need to save all the streaming Tweets in a .txt file

    • Nilang June 15, 2015 / 09:29

      Hey, Aravind you need to change the name of txt file every time in program if you don’t want your collected tweets to be overwritten. Whenever you stop the streaming and a bunch of tweets are collected and next time when you start again to collect new set of tweets just change the name of text file to whatever you want to. This happens because in program PrintStream will first find whether the file with given name on given path exists or not. If file is present then it will overwrite the contents, if not then it will make a new txt file and writes new content(here tweets) in newly created text file. Hope this helps.

      • Aravind June 16, 2015 / 06:53

        Yeah it helps…Thank You 🙂

  54. sadhana January 26, 2016 / 11:55

    hi,
    will you please provide code for collecting geo-tagged tweets

      • sadhana January 27, 2016 / 05:07

        Thank you

    • DCrowley February 8, 2016 / 19:36

      You need to copy the code into an IDE and use the external Twitter4j jars

  55. sadhana February 20, 2016 / 19:28

    hi,
    i am using eclipse tool and twitter 4j api to collect the tweets.How to store the collected tweets in excel sheet ad how to apply the sentiment analysis.please do let me know how to do this.

    thanks in advance

    • DCrowley February 22, 2016 / 10:12

      You wouldn’t save them in an Excel format. Save them in CSV format. Then you can pick an already existing sentiment analysis package and use that. There is a lot of examples and lots of libraries depending on what language you are comfortable with (Python or R maybe)

      • sadhana April 12, 2016 / 15:34

        Sir i did’nt get how to store the data in CSV format please help me. I have used lexicon based approach to calucate polarity of tweets.

        Thank you

  56. rajeswarilakshman March 2, 2016 / 09:00

    getting following error

    ” The method addListener(StatusListener) is undefined for the type TwitterStream ” when i try to run this as java application

    • DCrowley March 2, 2016 / 12:18

      What version of Twitter4j are you using?

  57. mohit March 21, 2016 / 14:36

    crowley sir pls help its very urgent i have hadoop 1.2.1 and ubuntu 14.04 flume 1.6 i m trying to get twitter data in hdfs but

    previously i got errror while running flume agent “NO SUCH METHOD FOUND ” which i corrected by googling by replacing twitter jar of version 3..*.* with 2.*.*
    BUT NOW I M GETTING ERROR “JAVA.LANG.NULLPOINTEREXCEPTION consumer secret is null”
    pls i have tried a lot u caN Screenshots are below
    ALSO TO TELL U THAT PREVIOUSLY I GOT ERROR COULD NOT LOAD MAIN CLASS SO I UNSET FLUME HOME

    21 Mar 2016 07:51:25,851 ERROR [lifecycleSupervisor-1-5] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) – Unable to start EventDrivenSourceRunner: { source:com.cloudera.flume.source.TwitterSource{name:Twitter,state:IDLE} } – Exception follows.
    java.lang.NullPointerException: consumer secret is null
    at twitter4j.TwitterBaseImpl.setOAuthConsumer(TwitterBaseImpl.java:250)
    at com.cloudera.flume.source.TwitterSource.start(TwitterSource.java:129)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

  58. mohit March 21, 2016 / 14:51

    pls help me i have found it everywhere i can find no answer pls i will be shattered i don’t get help this is my twitter.conf
    TwitterAgent.sources= Twitter
    TwitterAgent.channels= MemChannel
    TwitterAgent.sinks=HDFS
    TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
    TwitterAgent.sources.Twitter.channels= MemChannel

    TwitterAgent.sources.Twitter.consumerSecret=
    XXX
    TwitterAgent.sources.Twitter.consumerKey= XXX
    TwitterAgent.sources.Twitter.accessTokenSecret= XXXX
    TwitterAgent.sources.Twitter.accessToken=
    XXXX

    TwitterAgent.sources.Twitter.keywords= samsung galaxy,samsung phone launch,samsung mobile

    TwitterAgent.sinks.HDFS.channel=MemChannel
    TwitterAgent.sinks.HDFS.type=hdfs
    TwitterAgent.sinks.HDFS.hdfs.path=hdfs://localhost:9000/winit/%Y/%m/%d/%H/
    TwitterAgent.sinks.HDFS.hdfs.fileType=DataStream
    TwitterAgent.sinks.HDFS.hdfs.writeformat=Text
    TwitterAgent.sinks.HDFS.hdfs.batchSize=1000
    TwitterAgent.sinks.HDFS.hdfs.rollSize=0
    TwitterAgent.sinks.HDFS.hdfs.rollCount=10000
    TwitterAgent.sinks.HDFS.hdfs.rollInterval=600
    TwitterAgent.channels.MemChannel.type=memory
    TwitterAgent.channels.MemChannel.capacity=10000
    TwitterAgent.channels.MemChannel.transactionCapacity=1000

    • DCrowley April 1, 2016 / 10:37

      Please don’t post with your keys shown as these should be private.

  59. rafael April 1, 2016 / 09:13

    Can you send me code to print all tweets on son format;
    What must change on this to code to work;
    Thanks you very much

    • DCrowley April 1, 2016 / 10:36

      In what format? JSON?

      • rafael April 2, 2016 / 11:59

        yes json! Sorry for my mistake

  60. rafael April 3, 2016 / 11:33

    cannot find symbol
    symbol: class Tweet
    location: class TwitterApp2
    Can use functional operators
    And when i press to fix the bug :
    Create a local variable “Tweet”
    Use functional Operators
    But nothing works!
    if you can help me to fix this bug and start download data from twitter in json format.
    Thank you

    • rafael April 4, 2016 / 12:15

      i fix it. The problem was the the twitter4j realase. I do some changes and work

  61. Ishan April 5, 2016 / 12:05

    Thank you very much for this example, David!
    I had to set proxy host and port to make this program work on my machine:
    cb.setHttpProxyHost(“proxy”);
    cb.setHttpProxyPort(8080);

    • Ishan April 5, 2016 / 12:20

      I had also set VM args and I think that did the trick instead of cb.setHttp… methods:
      -Dhttp.proxyPort=8080 -Dhttp.proxyHost=proxy -Dhttps.proxyPort=8080 -Dhttps.proxyHost=proxy

  62. sadhana April 12, 2016 / 08:58

    how to store the twitter data after collecting using twitter4j in database.

    • DCrowley April 12, 2016 / 09:05

      It depends on how many tweets are coming in – some help here – https://dev.twitter.com/streaming/overview/processing

      Trying to store them in a database on the fly is probably not going to work. You probably need to collect, then process as different tasks.

      • sadhana April 12, 2016 / 15:50

        Please let me know how to store the data in csv format.

        Thank you sir

  63. charlah malika April 13, 2016 / 21:19

    how to add the method of geolcalisation and the language to the program

  64. Nilang May 8, 2016 / 03:23

    Hi Sadhana,

    Try out these lines just above ConfigurationBuilder in the code. I have not tried this in csv format, but still try this. When I was working with this, I stored data in txt file. While storing you might face mapping issues.

    FileOutputStream f = new FileOutputStream(“C:\\your_path\\your_file_name.csv”);
    System.setOut(new PrintStream(f));

    • sadhana July 28, 2016 / 16:50

      Thank you Nilang

  65. iin August 27, 2016 / 05:12

    Hi Sir i have a problem ,,

    Exception in thread “main” java.lang.RuntimeException: Uncompilable source code – is not abstract and does not override abstract method onStallWarning(twitter4j.StallWarning) in twitter4j.StatusListener
    at Simple.main(Simple.java:31)

    how o fix it ?

    • DCrowley August 29, 2016 / 08:46

      You don’t have the method implemented – and it must be for StatusListener

  66. Srdjan (@srga1993) September 15, 2016 / 15:25

    Hello sir,

    I have a question. I was able to configure everything successfully and collect tweets based on hashtags that I provided in my default list of keywords. Now besides that I am also collecting other hashtags that appear in tweets, the ones that are not in my default keyword list, and then I add them to my default list so that I can receive tweets with those hashtags too.
    But my problem is I am focusing on specific event (let’s say olympics) and sometimes I get some hashtags that are not really related to olympics, but they were tweeted with some of the hashtags from my default list. Do you have any idea maybe from statistical or algorithmic point, how could I solve this problem, or I mean how can I decide whether specific hashtags should be added to my default list of keywords or not (is it relevant to the event for which I am trying to collect data)?

    I know this might not be related to this discussion, but I would appreciate any kind of advice or suggestion.

    Thanks in advance.

    • DCrowley September 21, 2016 / 07:56

      This is much more of a research question then a coding question. I think looking at your data would help – collect specific tweets with the hashtag. Look at other hashtag use then evolve the system. Spammers seem to use hashtags a lot – but they often use just trending topics so you could examine both using inclusion of hashtags and then removing tweets that have unrelated trending hashtags in them. Again this could be difficult as hashtags could evolve. Semi related is work by Pavan Kapanipathi & Fabrizio Orlandi http://corescholar.libraries.wright.edu/cgi/viewcontent.cgi?article=2025&context=knoesis using semantic technologies to take tweet content and tie them to external concepts. This might help

  67. Gaurav October 8, 2016 / 15:41

    Thanks for the blog…sir…
    I am getting rate limit exception ….
    Could you please tell me how can I avoid this….

    • DCrowley October 9, 2016 / 11:34

      Read the Twitter API guidelines – depends on what you want to do. You could do a request (timed) to stop the rate limiting

  68. KetkiK February 27, 2017 / 06:46

    I am getting the following error :
    I have added java-json.jar

    [Mon Feb 27 12:14:10 IST 2017]Establishing connection.
    Exception in thread “Twitter Stream consumer-1[Establishing connection]” java.lang.NoClassDefFoundError: twitter4j/internal/org/json/JSONException
    at twitter4j.TwitterStreamImpl.getFilterStream(TwitterStreamImpl.java:290)
    at twitter4j.TwitterStreamImpl$7.getStream(TwitterStreamImpl.java:279)
    at twitter4j.TwitterStreamImpl$7.getStream(TwitterStreamImpl.java:277)
    at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:427)
    Caused by: java.lang.ClassNotFoundException: twitter4j.internal.org.json.JSONException
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    … 4 more

    • DCrowley February 27, 2017 / 09:57

      You could possibly be using mismatched versions? Check if your streaming and search versions of twitter4j are the same

      • KetkiK February 27, 2017 / 10:31

        I am using different versions of stream and core files .
        If I use the latest version 4.0.4 I get errors at .addListener.
        All the errors get resolved after adding “twitter4j-core-3.0.4” , “twitter4j-stream-2.2.5” , “twitter-4j”.

        But I get the following error now :

        [Mon Feb 27 15:57:10 IST 2017]Establishing connection.
        [Mon Feb 27 15:57:13 IST 2017]Connection established.
        [Mon Feb 27 15:57:13 IST 2017]Receiving status stream.

        Exception in thread “Twitter4J Async Dispatcher[0]” java.lang.NoSuchMethodError: twitter4j.json.JSONObjectType.determine(Ltwitter4j/internal/org/json/JSONObject;)Ltwitter4j/json/JSONObjectType;
        at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100)
        at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116)

  69. DCrowley February 27, 2017 / 10:47

    I would use all matching jars – as methods/class structures will have changed between 2.2.5 and 3.0.4

    • KetkiK February 27, 2017 / 11:07

      Can you tell me a stable version of core and stream jar files ?

  70. DCrowley February 27, 2017 / 11:12

    From http://twitter4j.org/en/ – it looks like 4.0.4. But I have yet to try this version with my codebase

    • KetkiK February 27, 2017 / 11:18

      4.0.4 doesn’t work well.
      I tried using 3.0.4 core and stream jar files and it worked well for me .
      Getting all the live tweets.
      Thanks alot .
      It would be really helpful if you would describe how the code works exactly.

      • KetkiK March 1, 2017 / 12:21

        I get the following response for content :
        RT @farukonalan: 2015’de Daesh’in infaz görüntülerini paylasan Le Pen’in dokunulmazligi “terör propagandasi” kapsaminda kaldiriliyor. https…

        Here the URL received is incomplete.This happens alot of times.I wanted to know what is the issue with the data?
        Why do I get incomplete tweets ?

        • DCrowley March 1, 2017 / 12:34

          Ask Twitter! or the Twitter4j developers – or start looking at the raw JSON data returned by the request.

          • KetkiK March 1, 2017 / 13:01

            Thank you.

  71. DCrowley March 1, 2017 / 13:06

    Also if you look at the original tweet – https://twitter.com/farukonalan/status/836848809426763779 there is no link. So is the link to the images contained in the tweet or maybe the person retweeting it added a link? Then it might have been over the 140 character limit. But a good place to start would be to examine the actual retweet and see what it actually contains

  72. gurpreet kaur April 9, 2017 / 13:47

    i am getting 404 error…requested resource is not available

    • DCrowley April 9, 2017 / 18:52

      Are you online? Using a proxy?

  73. Rahul Sharma July 10, 2017 / 06:28

    Hello,

    I have implemented this and it works like charm. Now if I want to dynamically add more user to follow, how should I go about it? ie The updated list of users to follow gets stored in DB and from there if a new user is added or an old user got removed I need to update my follow predicates and restart the stream in graceful manner. Can you provide a code sample for this

    Regards
    Rahul

    • DCrowley July 18, 2017 / 11:42

      I could – but I am not a contract developer. Probably adding them daily to a list/db and restarting the process when you have updated the list of users you want to follow.

  74. Anca-Elena December 9, 2017 / 19:00

    Ty for this tutorial. But what if I want to get all the tweets ever posted on Twitter. Is there a way to do that? I want to analyse all the tweets on Twitter (text mining) that contain a certain keyword (for my project).

  75. manu May 14, 2019 / 08:51

    it showing from longer time
    19/05/14 14:15:43 INFO twitter4j.TwitterStreamImpl: Establishing connection.
    19/05/14 14:15:44 INFO twitter4j.TwitterStreamImpl: https://stream.twitter.com/1/statuses/filter.json
    19/05/14 14:15:44 INFO twitter4j.TwitterStreamImpl: Waiting for 16000 milliseconds

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.