mirror of
https://github.com/twitter/the-algorithm.git
synced 2024-06-02 09:18:49 +02:00
ef4c5eb65e
Please note we have force-pushed a new initial commit in order to remove some publicly-available Twitter user information. Note that this process may be required in the future.
18 lines
1.2 KiB
Markdown
18 lines
1.2 KiB
Markdown
# UserTweetGraph (UTG)
|
|
|
|
## What is it
|
|
User Tweet Graph (UTG) is a Finalge thrift service built on the GraphJet framework. In maintains a graph of user-tweet engagements and serves user recommendations based on traversals of this graph.
|
|
|
|
## How is it used on Twitter
|
|
UTG recommends tweets based on collaborative filtering & random walks. UTG takes a set of seed users or seed tweets as input, and performs
|
|
1-hop, 2-hop, or even 3+hop traversals on the engagement graph.
|
|
UTG's user-tweet engagement edges are bi-directional, and this enables it to perform flexible multi-hop traversals. The flipside to this is
|
|
UTG is more memory demanding compared to other GraphJet services like UTEG, whose engagement edges are single directional.
|
|
|
|
UTG is a stateful service and relies on a Kafka stream to ingest & persist states. The Kafka stream is processed and generated by Recos-Injector.
|
|
It maintains an in-memory user engagements over the past 24-48 hours. Older events are dropped and GC'ed.
|
|
|
|
For full details on storage & processing, please check out our open-sourced project GraphJet, a general-purpose high performance in-memory storage engine.
|
|
- https://github.com/twitter/GraphJet
|
|
- http://www.vldb.org/pvldb/vol9/p1281-sharma.pdf
|