Mirrors
/
the-algorithm
mirror of https://github.com/twitter/the-algorithm.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
the-algorithm/src/scala/com/twitter/recos/user_tweet_graph
History
twitter-team ef4c5eb65e Twitter Recommendation Algorithm 
Please note we have force-pushed a new initial commit in order to remove some publicly-available Twitter user information. Note that this process may be required in the future.
 		2023-03-31 17:36:31 -05:00
..
relatedTweetHandlers Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
store Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
util Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
BUILD Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
Main.scala Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
README.md Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
UserTweetGraph.scala Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
UserTweetGraphConfig.scala Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00
UserTweetGraphWriter.scala Twitter Recommendation Algorithm 2023-03-31 17:36:31 -05:00

README.md

UserTweetGraph (UTG)

What is it

User Tweet Graph (UTG) is a Finalge thrift service built on the GraphJet framework. In maintains a graph of user-tweet engagements and serves user recommendations based on traversals of this graph.

How is it used on Twitter

UTG recommends tweets based on collaborative filtering & random walks. UTG takes a set of seed users or seed tweets as input, and performs 1-hop, 2-hop, or even 3+hop traversals on the engagement graph. UTG's user-tweet engagement edges are bi-directional, and this enables it to perform flexible multi-hop traversals. The flipside to this is UTG is more memory demanding compared to other GraphJet services like UTEG, whose engagement edges are single directional.

UTG is a stateful service and relies on a Kafka stream to ingest & persist states. The Kafka stream is processed and generated by Recos-Injector. It maintains an in-memory user engagements over the past 24-48 hours. Older events are dropped and GC'ed.

For full details on storage & processing, please check out our open-sourced project GraphJet, a general-purpose high performance in-memory storage engine.