Raazesh Sainudiin

Raazesh Sainudiin

I work at the interface of mathematics, computing and statistics to solve real-world problems through custom-built mathematical and statistical models.

Getting your own Twitter Developer Credentials for sds-2.x

Summary

Please follow the instructions below to get your own Twitter Developer Credentials.

You need this for Structured Streaming Examples and Exercises in the sequel of sds-2.x, after we have covered Spark core and SparkSQL, you will be needing to set up a developer account with Twitter. It is a good idea to get this out of the way at your earliest convenience as Twitter may take some time to review your application for the developer account.

As long as you are clear to twitter.com by answering the questions in their online application that you only need this Twitter developer account for educational purposes, then you should not have any problem.

You need to follow these instructions to get Twitter Developer Accout so you can continue as an independent social-media resarcher/developer during and after learning Apache Sparks’s Legacy and Structured Streaming modules in the course.

  1. Go to https://developer.twitter.com
    • When twitter.com ask about your use case, choose the student project / learning to code option,
      • You can inform them that you are planning to practice organizing Twitter data from Twitter’s Streaming API with various search criteria, including hashtags, language, words, etc., as a part of your assigned exercise in the course “Scalable Data Science” by lamastex.org with instructor Raazesh Sainudiin.
      • Choose no when asked if you will be providing this data to a government entity.
    • Here are some possible answers you might need to provide, in some cases:
      • I am using the Twitter API to:
    • learn the Spark Streaming (legacy) via Twitter Streaming API
    • learn the Spark Structured Streaming via Twitter Streaming API - What I am likely to do with the collected Tweets:
    • learn Spark Structured/Streaming and downstream SparkSQL analysis - What I am NOT going to do with the collected Tweets:
    • programmatically ‘update status’ in Twitter using various Twitter APIs
    • publicly display Tweets I’ll be consuming via Twitter STreaming API - Why I need the Twitter Developer account/credentials?
    • for learning Apache Spark’s Streaming and SQL libraries
    • prototype Proof-of-Concept codes for scalable streaming analytics

Updated: