Twitter research platform
From Federal Burro of Information
Version 1.0
- fetch the data /twitter_streaming_es.py
- do queries on the data
- search eater
- queue processor
- rate limiting aware.
---
- installed elastics search. mostly rooted in /data/elasticsearch
- https://github.com/mobz/elasticsearch-head
- prototype http://adilmoujahid.com/posts/2014/07/twitter-analytics/
- elk https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04
- http://192.168.1.98:9200/_plugin/head/
version 2.0
- proxy the queries
- store the results
- don't store the data yourself.
Sentiment notes
http://www.cs.columbia.edu/~julia/papers/Agarwaletal11.pdf
http://www.csc.ncsu.edu/faculty/healey/tweet_viz/
http://www.cs.cornell.edu/home/llee/papers/sentiment.pdf ( ML for Sentiment )
databases: https://www.quora.com/Is-there-a-downloadable-database-of-positive-and-negative-words
deployment notes
working dir: /home/david/work/trp
running stream collector:
while true; do python mydatetime.py ; sleep 5; done
sends data to elastic search.
