[opensource] Update README to include all new modules

Since the first batch of open sourcing, we have added the following components:
- User signal service
- Unified user actions
- Topic social proof service

Update the README to include these.

README.md

@@ -1,22 +1,39 @@
 # Twitter's Recommendation Algorithm
 
-Twitter's Recommendation Algorithm is a set of services and jobs that are responsible for constructing and serving the
+Twitter's Recommendation Algorithm is a set of services and jobs that are responsible for serving feeds of Tweets and other content across all Twitter product surfaces (e.g. For You Timeline, Search, Explore). For an introduction to how the algorithm works, please refer to our [engineering blog](https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm).
-Home Timeline. For an introduction to how the algorithm works, please refer to our [engineering blog](https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm). The
-diagram below illustrates how major services and jobs interconnect.
 
-![](docs/system-diagram.png)
+## Architecture
 
-These are the main components of the Recommendation Algorithm included in this repository:
+Product surfaces at Twitter are built on a shared set of data, models, and software frameworks. The shared components included in this repository are listed below:
 
 | Type | Component | Description |
 |------------|------------|------------|
-| Feature | [SimClusters](src/scala/com/twitter/simclusters_v2/README.md) | Community detection and sparse embeddings into those communities. |
+| Data | [unified-user-actions](unified_user_actions/README.md) | Real-time stream of user actions on Twitter. |
+|      | [user-signal-service](user-signal-service/README.md) | Centralized platform to retrieve explicit (e.g. likes, replies) and implicit (e.g. profile visits, tweet clicks) user signals. |
+| Model | [SimClusters](src/scala/com/twitter/simclusters_v2/README.md) | Community detection and sparse embeddings into those communities. |
 |       | [TwHIN](https://github.com/twitter/the-algorithm-ml/blob/main/projects/twhin/README.md) | Dense knowledge graph embeddings for Users and Tweets. |
 |       | [trust-and-safety-models](trust_and_safety_models/README.md) | Models for detecting NSFW or abusive content. |
 |       | [real-graph](src/scala/com/twitter/interaction_graph/README.md) | Model to predict the likelihood of a Twitter User interacting with another User. |
 |       | [tweepcred](src/scala/com/twitter/graph/batch/job/tweepcred/README) | Page-Rank algorithm for calculating Twitter User reputation. |
 |       | [recos-injector](recos-injector/README.md) | Streaming event processor for building input streams for [GraphJet](https://github.com/twitter/GraphJet) based services. |
 |       | [graph-feature-service](graph-feature-service/README.md) | Serves graph features for a directed pair of Users (e.g. how many of User A's following liked Tweets from User B). |
+|       | [topic-social-proof](topic-social-proof/README.md) | Identifies topics related to individual Tweets. |
+| Software framework | [navi](navi/README.md) | High performance, machine learning model serving written in Rust. |
+|                    | [product-mixer](product-mixer/README.md) | Software framework for building feeds of content. |
+|                    | [twml](twml/README.md) | Legacy machine learning framework built on TensorFlow v1. |
+
+The product surface currently included in this repository is the For You Timeline.
+
+### For You Timeline
+
+The diagram below illustrates how major services and jobs interconnect to construct a For You Timeline.
+
+![](docs/system-diagram.png)
+
+The core components of the For You Timeline included in this repository are listed below:
+
+| Type | Component | Description |
+|------------|------------|------------|
 | Candidate Source | [search-index](src/java/com/twitter/search/README.md) | Find and rank In-Network Tweets. ~50% of Tweets come from this candidate source. |
 |                  | [cr-mixer](cr-mixer/README.md) | Coordination layer for fetching Out-of-Network tweet candidates from underlying compute services. |
 |                  | [user-tweet-entity-graph](src/scala/com/twitter/recos/user_tweet_entity_graph/README.md) (UTEG)| Maintains an in memory User to Tweet interaction graph, and finds candidates based on traversals of this graph. This is built on the [GraphJet](https://github.com/twitter/GraphJet) framework. Several other GraphJet based features and candidate sources are located [here](src/scala/com/twitter/recos). |
@@ -26,11 +43,10 @@ These are the main components of the Recommendation Algorithm included in this r
 | Tweet mixing & filtering | [home-mixer](home-mixer/README.md) | Main service used to construct and serve the Home Timeline. Built on [product-mixer](product-mixer/README.md). |
 |                          | [visibility-filters](visibilitylib/README.md) | Responsible for filtering Twitter content to support legal compliance, improve product quality, increase user trust, protect revenue through the use of hard-filtering, visible product treatments, and coarse-grained downranking. |
 |                          | [timelineranker](timelineranker/README.md) | Legacy service which provides relevance-scored tweets from the Earlybird Search Index and UTEG service. |
-| Software framework | [navi](navi/README.md) | High performance, machine learning model serving written in Rust. |
-|                    | [product-mixer](product-mixer/README.md) | Software framework for building feeds of content. |
-|                    | [twml](twml/README.md) | Legacy machine learning framework built on TensorFlow v1. |
 
-We include Bazel BUILD files for most components, but not a top-level BUILD or WORKSPACE file.
+## Build and test code
+
+We include Bazel BUILD files for most components, but not a top-level BUILD or WORKSPACE file. We plan to add a more complete build and test system in the future.
 
 ## Contributing
 

topic-social-proof/README.md

@@ -0,0 +1,8 @@
+# Topic Social Proof Service (TSPS)
+=================
+
+**Topic Social Proof Service** (TSPS) serves as a centralized source for verifying topics related to Timelines and Notifications. By analyzing user's topic preferences, such as following or unfollowing, and employing semantic annotations and tweet embeddings from SimClusters, or other machine learning models, TSPS delivers highly relevant topics tailored to each user's interests.
+
+For instance, when a tweet discusses Stephen Curry, the service determines if the content falls under topics like "NBA" and/or "Golden State Warriors" while also providing relevance scores based on SimClusters Embedding. Additionally, TSPS evaluates user-specific topic preferences to offer a comprehensive list of available topics, only those the user is currently following, or new topics they have not followed but may find interesting if recommended on specific product surfaces.
+
+

topic-social-proof/server/BUILD

@@ -0,0 +1,24 @@
+jvm_binary(
+    name = "bin",
+    basename = "topic-social-proof",
+    main = "com.twitter.tsp.TopicSocialProofStratoFedServerMain",
+    runtime_platform = "java11",
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "strato/src/main/scala/com/twitter/strato/logging/logback",
+        "topic-social-proof/server/src/main/resources",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp",
+    ],
+)
+
+#  Aurora Workflows build phase convention requires a jvm_app named with ${project-name}-app
+jvm_app(
+    name = "topic-social-proof-app",
+    archive = "zip",
+    binary = ":bin",
+    tags = [
+        "bazel-compatible",
+    ],
+)

topic-social-proof/server/src/main/resources/BUILD

@@ -0,0 +1,8 @@
+resources(
+    sources = [
+        "*.xml",
+        "*.yml",
+        "config/*.yml",
+    ],
+    tags = ["bazel-compatible"],
+)

topic-social-proof/server/src/main/resources/config/decider.yml

@@ -0,0 +1,61 @@
+# Keys are sorted in an alphabetical order
+
+enable_topic_social_proof_score:
+  comment : "Enable the calculation of <topic, tweet> cosine similarity score in TopicSocialProofStore. 0 means do not calculate the score and use a random rank to generate topic social proof"
+  default_availability: 0
+
+enable_tweet_health_score:
+  comment: "Enable the calculation for health scores in tweetInfo. By enabling this decider, we will compute TweetHealthModelScore"
+  default_availability: 0
+
+enable_user_agatha_score:
+  comment: "Enable the calculation for health scores in tweetInfo. By enabling this decider, we will compute UserHealthModelScore"
+  default_availability: 0
+
+enable_loadshedding_HomeTimeline:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineRecommendTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_MagicRecsRecommendTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_TopicLandingPage:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineFeatures:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineTopicTweetsMetrics:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineUTEGTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_HomeTimelineSimClusters:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_ExploreTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_MagicRecsTopicTweets:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0
+
+enable_loadshedding_Search:
+  comment: "Enable loadshedding (from 0% to 100%). Requests that have been shed will return an empty response"
+  default_availability: 0

topic-social-proof/server/src/main/resources/logback.xml

@@ -0,0 +1,155 @@
+<configuration>
+  <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>
+  <property name="async_queue_size" value="${queue.size:-50000}"/>
+  <property name="async_max_flush_time" value="${max.flush.time:-0}"/>
+  <!-- ===================================================== -->
+  <!-- Structured Logging -->
+  <!-- ===================================================== -->
+  <!-- Only sample 0.1% of the requests -->
+  <property name="splunk_sampling_rate" value="${splunk_sampling_rate:-0.001}"/>
+  <include resource="structured-logger-logback.xml"/>
+  <!-- ===================================================== -->
+  <!-- Service Config -->
+  <!-- ===================================================== -->
+  <property name="DEFAULT_SERVICE_PATTERN"
+            value="%-16X{transactionId} %logger %msg"/>
+
+  <!-- ===================================================== -->
+  <!-- Common Config -->
+  <!-- ===================================================== -->
+
+  <!-- JUL/JDK14 to Logback bridge -->
+  <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
+    <resetJUL>true</resetJUL>
+  </contextListener>
+
+  <!-- Service Log (Rollover every 50MB, max 11 logs) -->
+  <appender name="SERVICE" class="ch.qos.logback.core.rolling.RollingFileAppender">
+    <file>${log.service.output}</file>
+    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
+      <fileNamePattern>${log.service.output}.%i</fileNamePattern>
+      <minIndex>1</minIndex>
+      <maxIndex>10</maxIndex>
+    </rollingPolicy>
+    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
+      <maxFileSize>50MB</maxFileSize>
+    </triggeringPolicy>
+    <encoder>
+      <pattern>%date %.-3level ${DEFAULT_SERVICE_PATTERN}%n</pattern>
+    </encoder>
+  </appender>
+
+  <!-- Strato package only log (Rollover every 50MB, max 11 logs) -->
+  <appender name="STRATO-ONLY" class="ch.qos.logback.core.rolling.RollingFileAppender">
+    <file>${log.strato_only.output}</file>
+    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
+      <fileNamePattern>${log.strato_only.output}.%i</fileNamePattern>
+      <minIndex>1</minIndex>
+      <maxIndex>10</maxIndex>
+    </rollingPolicy>
+    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
+      <maxFileSize>50MB</maxFileSize>
+    </triggeringPolicy>
+    <encoder>
+      <pattern>%date %.-3level ${DEFAULT_SERVICE_PATTERN}%n</pattern>
+    </encoder>
+  </appender>
+
+  <!-- LogLens -->
+  <appender name="LOGLENS" class="com.twitter.loglens.logback.LoglensAppender">
+    <mdcAdditionalContext>true</mdcAdditionalContext>
+    <category>loglens</category>
+    <index>${log.lens.index}</index>
+    <tag>${log.lens.tag}/service</tag>
+    <encoder>
+      <pattern>%msg%n</pattern>
+    </encoder>
+    <turboFilter class="ch.qos.logback.classic.turbo.DuplicateMessageFilter">
+      <cacheSize>500</cacheSize>
+      <allowedRepetitions>50</allowedRepetitions>
+    </turboFilter>
+    <filter class="com.twitter.strato.logging.logback.RegexFilter">
+      <forLogger>manhattan-client</forLogger>
+      <excludeRegex>.*InvalidRequest.*</excludeRegex>
+    </filter>
+  </appender>
+
+  <!-- ===================================================== -->
+  <!-- Primary Async Appenders -->
+  <!-- ===================================================== -->
+
+  <appender name="ASYNC-SERVICE" class="ch.qos.logback.classic.AsyncAppender">
+    <queueSize>${async_queue_size}</queueSize>
+    <maxFlushTime>${async_max_flush_time}</maxFlushTime>
+    <appender-ref ref="SERVICE"/>
+  </appender>
+
+  <appender name="ASYNC-STRATO-ONLY" class="ch.qos.logback.classic.AsyncAppender">
+    <queueSize>${async_queue_size}</queueSize>
+    <maxFlushTime>${async_max_flush_time}</maxFlushTime>
+    <appender-ref ref="STRATO-ONLY"/>
+  </appender>
+
+  <appender name="ASYNC-LOGLENS" class="ch.qos.logback.classic.AsyncAppender">
+    <queueSize>${async_queue_size}</queueSize>
+    <maxFlushTime>${async_max_flush_time}</maxFlushTime>
+    <appender-ref ref="LOGLENS"/>
+  </appender>
+
+  <!-- ===================================================== -->
+  <!-- Package Config -->
+  <!-- ===================================================== -->
+
+  <!-- Per-Package Config (shared) -->
+  <logger name="com.twitter" level="info"/>
+
+  <!--
+    By default, we leave the strato package at INFO level.
+    However, this line allows us to set the entire strato package, or a subset of it, to
+    a specific level. For example, if you pass -Dstrato_log_package=streaming -Dstrato_log_level=DEBUG
+    only loggers under com.twitter.strato.streaming.* will be set to DEBUG level. Passing only
+    -Dstrato_log_level will set all of strato.* to the specified level.
+  -->
+  <logger name="com.twitter.strato${strato_log_package:-}" level="${strato_log_level:-INFO}"/>
+
+  <logger name="com.twitter.wilyns" level="warn"/>
+  <logger name="com.twitter.finagle.mux" level="warn"/>
+  <logger name="com.twitter.finagle.serverset2" level="warn"/>
+  <logger name="com.twitter.logging.ScribeHandler" level="warn"/>
+  <logger name="com.twitter.zookeeper.client.internal" level="warn"/>
+  <logger name="com.twitter.decider.StoreDecider" level="warn"/>
+
+  <!-- Per-Package Config (Strato) -->
+  <logger name="com.twitter.distributedlog.client" level="warn"/>
+  <logger name="com.twitter.finagle.mtls.authorization.config.AccessControlListConfiguration" level="warn"/>
+  <logger name="com.twitter.finatra.kafka.common.kerberoshelpers" level="warn"/>
+  <logger name="com.twitter.finatra.kafka.utils.BootstrapServerUtils" level="warn"/>
+  <logger name="com.twitter.server.coordinate" level="error"/>
+  <logger name="com.twitter.zookeeper.client" level="info"/>
+  <logger name="org.apache.zookeeper" level="error"/>
+  <logger name="org.apache.zookeeper.ClientCnxn" level="warn"/>
+  <logger name="ZkSession" level="info"/>
+  <logger name="OptimisticLockingCache" level="off"/>
+  <logger name="manhattan-client" level="warn"/>
+  <logger name="strato.op" level="warn"/>
+  <logger name="org.apache.kafka.clients.NetworkClient" level="error"/>
+  <logger name="org.apache.kafka.clients.consumer.internals" level="error"/>
+  <logger name="org.apache.kafka.clients.producer.internals" level="error"/>
+  <!-- produce a lot of messages like: Building client authenticator with server name kafka -->
+  <logger name="org.apache.kafka.common.network" level="warn"/>
+
+  <!-- Root Config -->
+  <root level="${log_level:-INFO}">
+    <appender-ref ref="ASYNC-SERVICE"/>
+    <appender-ref ref="ASYNC-LOGLENS"/>
+  </root>
+
+  <!-- Strato package only logging-->
+  <logger name="com.twitter.strato"
+          level="info"
+          additivity="true">
+    <appender-ref ref="ASYNC-STRATO-ONLY" />
+  </logger>
+
+
+</configuration>

topic-social-proof/server/src/main/scala/com/twitter/tsp/BUILD

@@ -0,0 +1,12 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "finatra/inject/inject-thrift-client",
+        "strato/src/main/scala/com/twitter/strato/fed",
+        "strato/src/main/scala/com/twitter/strato/fed/server",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/columns",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/TopicSocialProofStratoFedServer.scala

@@ -0,0 +1,56 @@
+package com.twitter.tsp
+
+import com.google.inject.Module
+import com.twitter.strato.fed._
+import com.twitter.strato.fed.server._
+import com.twitter.strato.warmup.Warmer
+import com.twitter.tsp.columns.TopicSocialProofColumn
+import com.twitter.tsp.columns.TopicSocialProofBatchColumn
+import com.twitter.tsp.handlers.UttChildrenWarmupHandler
+import com.twitter.tsp.modules.RepresentationScorerStoreModule
+import com.twitter.tsp.modules.GizmoduckUserModule
+import com.twitter.tsp.modules.TSPClientIdModule
+import com.twitter.tsp.modules.TopicListingModule
+import com.twitter.tsp.modules.TopicSocialProofStoreModule
+import com.twitter.tsp.modules.TopicTweetCosineSimilarityAggregateStoreModule
+import com.twitter.tsp.modules.TweetInfoStoreModule
+import com.twitter.tsp.modules.TweetyPieClientModule
+import com.twitter.tsp.modules.UttClientModule
+import com.twitter.tsp.modules.UttLocalizationModule
+import com.twitter.util.Future
+
+object TopicSocialProofStratoFedServerMain extends TopicSocialProofStratoFedServer
+
+trait TopicSocialProofStratoFedServer extends StratoFedServer {
+  override def dest: String = "/s/topic-social-proof/topic-social-proof"
+
+  override val modules: Seq[Module] =
+    Seq(
+      GizmoduckUserModule,
+      RepresentationScorerStoreModule,
+      TopicSocialProofStoreModule,
+      TopicListingModule,
+      TopicTweetCosineSimilarityAggregateStoreModule,
+      TSPClientIdModule,
+      TweetInfoStoreModule,
+      TweetyPieClientModule,
+      UttClientModule,
+      UttLocalizationModule
+    )
+
+  override def columns: Seq[Class[_ <: StratoFed.Column]] =
+    Seq(
+      classOf[TopicSocialProofColumn],
+      classOf[TopicSocialProofBatchColumn]
+    )
+
+  override def configureWarmer(warmer: Warmer): Unit = {
+    warmer.add(
+      "uttChildrenWarmupHandler",
+      () => {
+        handle[UttChildrenWarmupHandler]()
+        Future.Unit
+      }
+    )
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/columns/BUILD

@@ -0,0 +1,12 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "stitch/stitch-storehaus",
+        "strato/src/main/scala/com/twitter/strato/fed",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/service",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/columns/TopicSocialProofBatchColumn.scala

@@ -0,0 +1,84 @@
+package com.twitter.tsp.columns
+
+import com.twitter.stitch.SeqGroup
+import com.twitter.stitch.Stitch
+import com.twitter.strato.catalog.Fetch
+import com.twitter.strato.catalog.OpMetadata
+import com.twitter.strato.config._
+import com.twitter.strato.config.AllowAll
+import com.twitter.strato.config.ContactInfo
+import com.twitter.strato.config.Policy
+import com.twitter.strato.data.Conv
+import com.twitter.strato.data.Description.PlainText
+import com.twitter.strato.data.Lifecycle.Production
+import com.twitter.strato.fed.StratoFed
+import com.twitter.strato.thrift.ScroogeConv
+import com.twitter.tsp.thriftscala.TopicSocialProofRequest
+import com.twitter.tsp.thriftscala.TopicSocialProofOptions
+import com.twitter.tsp.service.TopicSocialProofService
+import com.twitter.tsp.thriftscala.TopicWithScore
+import com.twitter.util.Future
+import com.twitter.util.Try
+import javax.inject.Inject
+
+class TopicSocialProofBatchColumn @Inject() (
+  topicSocialProofService: TopicSocialProofService)
+    extends StratoFed.Column(TopicSocialProofBatchColumn.Path)
+    with StratoFed.Fetch.Stitch {
+
+  override val policy: Policy =
+    ReadWritePolicy(
+      readPolicy = AllowAll,
+      writePolicy = AllowKeyAuthenticatedTwitterUserId
+    )
+
+  override type Key = Long
+  override type View = TopicSocialProofOptions
+  override type Value = Seq[TopicWithScore]
+
+  override val keyConv: Conv[Key] = Conv.ofType
+  override val viewConv: Conv[View] = ScroogeConv.fromStruct[TopicSocialProofOptions]
+  override val valueConv: Conv[Value] = Conv.seq(ScroogeConv.fromStruct[TopicWithScore])
+  override val metadata: OpMetadata =
+    OpMetadata(
+      lifecycle = Some(Production),
+      Some(PlainText("Topic Social Proof Batched Federated Column")))
+
+  case class TspsGroup(view: View) extends SeqGroup[Long, Fetch.Result[Value]] {
+    override protected def run(keys: Seq[Long]): Future[Seq[Try[Result[Seq[TopicWithScore]]]]] = {
+      val request = TopicSocialProofRequest(
+        userId = view.userId,
+        tweetIds = keys.toSet,
+        displayLocation = view.displayLocation,
+        topicListingSetting = view.topicListingSetting,
+        context = view.context,
+        bypassModes = view.bypassModes,
+        tags = view.tags
+      )
+
+      val response = topicSocialProofService
+        .topicSocialProofHandlerStoreStitch(request)
+        .map(_.socialProofs)
+      Stitch
+        .run(response).map(r =>
+          keys.map(key => {
+            Try {
+              val v = r.get(key)
+              if (v.nonEmpty && v.get.nonEmpty) {
+                found(v.get)
+              } else {
+                missing
+              }
+            }
+          }))
+    }
+  }
+
+  override def fetch(key: Key, view: View): Stitch[Result[Value]] = {
+    Stitch.call(key, TspsGroup(view))
+  }
+}
+
+object TopicSocialProofBatchColumn {
+  val Path = "topic-signals/tsp/topic-social-proof-batched"
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/columns/TopicSocialProofColumn.scala

@@ -0,0 +1,47 @@
+package com.twitter.tsp.columns
+
+import com.twitter.stitch
+import com.twitter.stitch.Stitch
+import com.twitter.strato.catalog.OpMetadata
+import com.twitter.strato.config._
+import com.twitter.strato.config.AllowAll
+import com.twitter.strato.config.ContactInfo
+import com.twitter.strato.config.Policy
+import com.twitter.strato.data.Conv
+import com.twitter.strato.data.Description.PlainText
+import com.twitter.strato.data.Lifecycle.Production
+import com.twitter.strato.fed.StratoFed
+import com.twitter.strato.thrift.ScroogeConv
+import com.twitter.tsp.thriftscala.TopicSocialProofRequest
+import com.twitter.tsp.thriftscala.TopicSocialProofResponse
+import com.twitter.tsp.service.TopicSocialProofService
+import javax.inject.Inject
+
+class TopicSocialProofColumn @Inject() (
+  topicSocialProofService: TopicSocialProofService)
+    extends StratoFed.Column(TopicSocialProofColumn.Path)
+    with StratoFed.Fetch.Stitch {
+
+  override type Key = TopicSocialProofRequest
+  override type View = Unit
+  override type Value = TopicSocialProofResponse
+
+  override val keyConv: Conv[Key] = ScroogeConv.fromStruct[TopicSocialProofRequest]
+  override val viewConv: Conv[View] = Conv.ofType
+  override val valueConv: Conv[Value] = ScroogeConv.fromStruct[TopicSocialProofResponse]
+  override val metadata: OpMetadata =
+    OpMetadata(lifecycle = Some(Production), Some(PlainText("Topic Social Proof Federated Column")))
+
+  override def fetch(key: Key, view: View): Stitch[Result[Value]] = {
+    topicSocialProofService
+      .topicSocialProofHandlerStoreStitch(key)
+      .map { result => found(result) }
+      .handle {
+        case stitch.NotFound => missing
+      }
+  }
+}
+
+object TopicSocialProofColumn {
+  val Path = "topic-signals/tsp/topic-social-proof"
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/BUILD

@@ -0,0 +1,23 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "configapi/configapi-abdecider",
+        "configapi/configapi-core",
+        "content-recommender/thrift/src/main/thrift:thrift-scala",
+        "decider/src/main/scala",
+        "discovery-common/src/main/scala/com/twitter/discovery/common/configapi",
+        "featureswitches/featureswitches-core",
+        "finatra/inject/inject-core/src/main/scala",
+        "frigate/frigate-common:base",
+        "frigate/frigate-common:util",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/candidate",
+        "interests-service/thrift/src/main/thrift:thrift-scala",
+        "src/scala/com/twitter/simclusters_v2/common",
+        "src/thrift/com/twitter/simclusters_v2:simclusters_v2-thrift-scala",
+        "stitch/stitch-storehaus",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/DeciderConstants.scala

@@ -0,0 +1,19 @@
+package com.twitter.tsp.common
+
+import com.twitter.servo.decider.DeciderKeyEnum
+
+object DeciderConstants {
+  val enableTopicSocialProofScore = "enable_topic_social_proof_score"
+  val enableHealthSignalsScoreDeciderKey = "enable_tweet_health_score"
+  val enableUserAgathaScoreDeciderKey = "enable_user_agatha_score"
+}
+
+object DeciderKey extends DeciderKeyEnum {
+
+  val enableHealthSignalsScoreDeciderKey: Value = Value(
+    DeciderConstants.enableHealthSignalsScoreDeciderKey
+  )
+  val enableUserAgathaScoreDeciderKey: Value = Value(
+    DeciderConstants.enableUserAgathaScoreDeciderKey
+  )
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/FeatureSwitchesBuilder.scala

@@ -0,0 +1,34 @@
+package com.twitter.tsp.common
+
+import com.twitter.abdecider.LoggingABDecider
+import com.twitter.featureswitches.v2.FeatureSwitches
+import com.twitter.featureswitches.v2.builder.{FeatureSwitchesBuilder => FsBuilder}
+import com.twitter.featureswitches.v2.experimentation.NullBucketImpressor
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.util.Duration
+
+case class FeatureSwitchesBuilder(
+  statsReceiver: StatsReceiver,
+  abDecider: LoggingABDecider,
+  featuresDirectory: String,
+  addServiceDetailsFromAurora: Boolean,
+  configRepoDirectory: String = "/usr/local/config",
+  fastRefresh: Boolean = false,
+  impressExperiments: Boolean = true) {
+
+  def build(): FeatureSwitches = {
+    val featureSwitches = FsBuilder()
+      .abDecider(abDecider)
+      .statsReceiver(statsReceiver)
+      .configRepoAbsPath(configRepoDirectory)
+      .featuresDirectory(featuresDirectory)
+      .limitToReferencedExperiments(shouldLimit = true)
+      .experimentImpressionStatsEnabled(true)
+
+    if (!impressExperiments) featureSwitches.experimentBucketImpressor(NullBucketImpressor)
+    if (addServiceDetailsFromAurora) featureSwitches.serviceDetailsFromAurora()
+    if (fastRefresh) featureSwitches.refreshPeriod(Duration.fromSeconds(10))
+
+    featureSwitches.build()
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/LoadShedder.scala

@@ -0,0 +1,44 @@
+package com.twitter.tsp.common
+
+import com.twitter.decider.Decider
+import com.twitter.decider.RandomRecipient
+import com.twitter.util.Future
+import javax.inject.Inject
+import scala.util.control.NoStackTrace
+
+/*
+  Provides deciders-controlled load shedding for a given displayLocation
+  The format of the decider keys is:
+
+    enable_loadshedding_<display location>
+  E.g.:
+    enable_loadshedding_HomeTimeline
+
+  Deciders are fractional, so a value of 50.00 will drop 50% of responses. If a decider key is not
+  defined for a particular displayLocation, those requests will always be served.
+
+  We should therefore aim to define keys for the locations we care most about in decider.yml,
+  so that we can control them during incidents.
+ */
+class LoadShedder @Inject() (decider: Decider) {
+  import LoadShedder._
+
+  // Fall back to False for any undefined key
+  private val deciderWithFalseFallback: Decider = decider.orElse(Decider.False)
+  private val keyPrefix = "enable_loadshedding"
+
+  def apply[T](typeString: String)(serve: => Future[T]): Future[T] = {
+    /*
+    Per-typeString level load shedding: enable_loadshedding_HomeTimeline
+    Checks if per-typeString load shedding is enabled
+     */
+    val keyTyped = s"${keyPrefix}_$typeString"
+    if (deciderWithFalseFallback.isAvailable(keyTyped, recipient = Some(RandomRecipient)))
+      Future.exception(LoadSheddingException)
+    else serve
+  }
+}
+
+object LoadShedder {
+  object LoadSheddingException extends Exception with NoStackTrace
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/ParamsBuilder.scala

@@ -0,0 +1,98 @@
+package com.twitter.tsp.common
+
+import com.twitter.abdecider.LoggingABDecider
+import com.twitter.abdecider.UserRecipient
+import com.twitter.contentrecommender.thriftscala.DisplayLocation
+import com.twitter.discovery.common.configapi.FeatureContextBuilder
+import com.twitter.featureswitches.FSRecipient
+import com.twitter.featureswitches.Recipient
+import com.twitter.featureswitches.UserAgent
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.interests.thriftscala.TopicListingViewerContext
+import com.twitter.timelines.configapi
+import com.twitter.timelines.configapi.Params
+import com.twitter.timelines.configapi.RequestContext
+import com.twitter.timelines.configapi.abdecider.LoggingABDeciderExperimentContext
+
+case class ParamsBuilder(
+  featureContextBuilder: FeatureContextBuilder,
+  abDecider: LoggingABDecider,
+  overridesConfig: configapi.Config,
+  statsReceiver: StatsReceiver) {
+
+  def buildFromTopicListingViewerContext(
+    topicListingViewerContext: Option[TopicListingViewerContext],
+    displayLocation: DisplayLocation,
+    userRoleOverride: Option[Set[String]] = None
+  ): Params = {
+
+    topicListingViewerContext.flatMap(_.userId) match {
+      case Some(userId) =>
+        val userRecipient = ParamsBuilder.toFeatureSwitchRecipientWithTopicContext(
+          userId,
+          userRoleOverride,
+          topicListingViewerContext,
+          Some(displayLocation)
+        )
+
+        overridesConfig(
+          requestContext = RequestContext(
+            userId = Some(userId),
+            experimentContext = LoggingABDeciderExperimentContext(
+              abDecider,
+              Some(UserRecipient(userId, Some(userId)))),
+            featureContext = featureContextBuilder(
+              Some(userId),
+              Some(userRecipient)
+            )
+          ),
+          statsReceiver
+        )
+      case _ =>
+        throw new IllegalArgumentException(
+          s"${this.getClass.getSimpleName} tried to build Param for a request without a userId"
+        )
+    }
+  }
+}
+
+object ParamsBuilder {
+
+  def toFeatureSwitchRecipientWithTopicContext(
+    userId: Long,
+    userRolesOverride: Option[Set[String]],
+    context: Option[TopicListingViewerContext],
+    displayLocationOpt: Option[DisplayLocation]
+  ): Recipient = {
+    val userRoles = userRolesOverride match {
+      case Some(overrides) => Some(overrides)
+      case _ => context.flatMap(_.userRoles.map(_.toSet))
+    }
+
+    val recipient = FSRecipient(
+      userId = Some(userId),
+      userRoles = userRoles,
+      deviceId = context.flatMap(_.deviceId),
+      guestId = context.flatMap(_.guestId),
+      languageCode = context.flatMap(_.languageCode),
+      countryCode = context.flatMap(_.countryCode),
+      userAgent = context.flatMap(_.userAgent).flatMap(UserAgent(_)),
+      isVerified = None,
+      isTwoffice = None,
+      tooClient = None,
+      highWaterMark = None
+    )
+    displayLocationOpt match {
+      case Some(displayLocation) =>
+        recipient.withCustomFields(displayLocationCustomFieldMap(displayLocation))
+      case None =>
+        recipient
+    }
+  }
+
+  private val DisplayLocationCustomField = "display_location"
+
+  def displayLocationCustomFieldMap(displayLocation: DisplayLocation): (String, String) =
+    DisplayLocationCustomField -> displayLocation.toString
+
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/RecTargetFactory.scala

@@ -0,0 +1,65 @@
+package com.twitter.tsp.common
+
+import com.twitter.abdecider.LoggingABDecider
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.base.TargetUser
+import com.twitter.frigate.common.candidate.TargetABDecider
+import com.twitter.frigate.common.util.ABDeciderWithOverride
+import com.twitter.gizmoduck.thriftscala.User
+import com.twitter.simclusters_v2.common.UserId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.timelines.configapi.Params
+import com.twitter.tsp.thriftscala.TopicSocialProofRequest
+import com.twitter.util.Future
+
+case class DefaultRecTopicSocialProofTarget(
+  topicSocialProofRequest: TopicSocialProofRequest,
+  targetId: UserId,
+  user: Option[User],
+  abDecider: ABDeciderWithOverride,
+  params: Params
+)(
+  implicit statsReceiver: StatsReceiver)
+    extends TargetUser
+    with TopicSocialProofRecRequest
+    with TargetABDecider {
+  override def globalStats: StatsReceiver = statsReceiver
+  override val targetUser: Future[Option[User]] = Future.value(user)
+}
+
+trait TopicSocialProofRecRequest {
+  tuc: TargetUser =>
+
+  val topicSocialProofRequest: TopicSocialProofRequest
+}
+
+case class RecTargetFactory(
+  abDecider: LoggingABDecider,
+  userStore: ReadableStore[UserId, User],
+  paramBuilder: ParamsBuilder,
+  statsReceiver: StatsReceiver) {
+
+  type RecTopicSocialProofTarget = DefaultRecTopicSocialProofTarget
+
+  def buildRecTopicSocialProofTarget(
+    request: TopicSocialProofRequest
+  ): Future[RecTopicSocialProofTarget] = {
+    val userId = request.userId
+    userStore.get(userId).map { userOpt =>
+      val userRoles = userOpt.flatMap(_.roles.map(_.roles.toSet))
+
+      val context = request.context.copy(userId = Some(request.userId)) // override to make sure
+
+      val params = paramBuilder
+        .buildFromTopicListingViewerContext(Some(context), request.displayLocation, userRoles)
+
+      DefaultRecTopicSocialProofTarget(
+        request,
+        userId,
+        userOpt,
+        ABDeciderWithOverride(abDecider, None)(statsReceiver),
+        params
+      )(statsReceiver)
+    }
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/TopicSocialProofDecider.scala

@@ -0,0 +1,26 @@
+package com.twitter.tsp
+package common
+
+import com.twitter.decider.Decider
+import com.twitter.decider.RandomRecipient
+import com.twitter.decider.Recipient
+import com.twitter.simclusters_v2.common.DeciderGateBuilderWithIdHashing
+import javax.inject.Inject
+
+case class TopicSocialProofDecider @Inject() (decider: Decider) {
+
+  def isAvailable(feature: String, recipient: Option[Recipient]): Boolean = {
+    decider.isAvailable(feature, recipient)
+  }
+
+  lazy val deciderGateBuilder = new DeciderGateBuilderWithIdHashing(decider)
+
+  /**
+   * When useRandomRecipient is set to false, the decider is either completely on or off.
+   * When useRandomRecipient is set to true, the decider is on for the specified % of traffic.
+   */
+  def isAvailable(feature: String, useRandomRecipient: Boolean = true): Boolean = {
+    if (useRandomRecipient) isAvailable(feature, Some(RandomRecipient))
+    else isAvailable(feature, None)
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/common/TopicSocialProofParams.scala

@@ -0,0 +1,104 @@
+package com.twitter.tsp.common
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.logging.Logger
+import com.twitter.timelines.configapi.BaseConfig
+import com.twitter.timelines.configapi.BaseConfigBuilder
+import com.twitter.timelines.configapi.FSBoundedParam
+import com.twitter.timelines.configapi.FSParam
+import com.twitter.timelines.configapi.FeatureSwitchOverrideUtil
+
+object TopicSocialProofParams {
+
+  object TopicTweetsSemanticCoreVersionId
+      extends FSBoundedParam[Long](
+        name = "topic_tweets_semantic_core_annotation_version_id",
+        default = 1433487161551032320L,
+        min = 0L,
+        max = Long.MaxValue
+      )
+  object TopicTweetsSemanticCoreVersionIdsSet
+      extends FSParam[Set[Long]](
+        name = "topic_tweets_semantic_core_annotation_version_id_allowed_set",
+        default = Set(TopicTweetsSemanticCoreVersionId.default))
+
+  /**
+   * Controls the Topic Social Proof cosine similarity threshold for the Topic Tweets.
+   */
+  object TweetToTopicCosineSimilarityThreshold
+      extends FSBoundedParam[Double](
+        name = "topic_tweets_cosine_similarity_threshold_tsp",
+        default = 0.0,
+        min = 0.0,
+        max = 1.0
+      )
+
+  object EnablePersonalizedContextTopics // master feature switch to enable backfill
+      extends FSParam[Boolean](
+        name = "topic_tweets_personalized_contexts_enable_personalized_contexts",
+        default = false
+      )
+
+  object EnableYouMightLikeTopic
+      extends FSParam[Boolean](
+        name = "topic_tweets_personalized_contexts_enable_you_might_like",
+        default = false
+      )
+
+  object EnableRecentEngagementsTopic
+      extends FSParam[Boolean](
+        name = "topic_tweets_personalized_contexts_enable_recent_engagements",
+        default = false
+      )
+
+  object EnableTopicTweetHealthFilterPersonalizedContexts
+      extends FSParam[Boolean](
+        name = "topic_tweets_personalized_contexts_health_switch",
+        default = true
+      )
+
+  object EnableTweetToTopicScoreRanking
+      extends FSParam[Boolean](
+        name = "topic_tweets_enable_tweet_to_topic_score_ranking",
+        default = true
+      )
+
+}
+
+object FeatureSwitchConfig {
+  private val enumFeatureSwitchOverrides = FeatureSwitchOverrideUtil
+    .getEnumFSOverrides(
+      NullStatsReceiver,
+      Logger(getClass),
+    )
+
+  private val intFeatureSwitchOverrides = FeatureSwitchOverrideUtil.getBoundedIntFSOverrides()
+
+  private val longFeatureSwitchOverrides = FeatureSwitchOverrideUtil.getBoundedLongFSOverrides(
+    TopicSocialProofParams.TopicTweetsSemanticCoreVersionId
+  )
+
+  private val doubleFeatureSwitchOverrides = FeatureSwitchOverrideUtil.getBoundedDoubleFSOverrides(
+    TopicSocialProofParams.TweetToTopicCosineSimilarityThreshold,
+  )
+
+  private val longSetFeatureSwitchOverrides = FeatureSwitchOverrideUtil.getLongSetFSOverrides(
+    TopicSocialProofParams.TopicTweetsSemanticCoreVersionIdsSet,
+  )
+
+  private val booleanFeatureSwitchOverrides = FeatureSwitchOverrideUtil.getBooleanFSOverrides(
+    TopicSocialProofParams.EnablePersonalizedContextTopics,
+    TopicSocialProofParams.EnableYouMightLikeTopic,
+    TopicSocialProofParams.EnableRecentEngagementsTopic,
+    TopicSocialProofParams.EnableTopicTweetHealthFilterPersonalizedContexts,
+    TopicSocialProofParams.EnableTweetToTopicScoreRanking,
+  )
+  val config: BaseConfig = BaseConfigBuilder()
+    .set(enumFeatureSwitchOverrides: _*)
+    .set(intFeatureSwitchOverrides: _*)
+    .set(longFeatureSwitchOverrides: _*)
+    .set(doubleFeatureSwitchOverrides: _*)
+    .set(longSetFeatureSwitchOverrides: _*)
+    .set(booleanFeatureSwitchOverrides: _*)
+    .build()
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/handlers/BUILD

@@ -0,0 +1,14 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "src/thrift/com/twitter/simclusters_v2:simclusters_v2-thrift-scala",
+        "stitch/stitch-storehaus",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/common",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/stores",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+        "topiclisting/topiclisting-core/src/main/scala/com/twitter/topiclisting",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/handlers/TopicSocialProofHandler.scala

@@ -0,0 +1,587 @@
+package com.twitter.tsp.handlers
+
+import com.twitter.conversions.DurationOps._
+import com.twitter.finagle.mux.ClientDiscardedRequestException
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.util.StatsUtil
+import com.twitter.simclusters_v2.common.SemanticCoreEntityId
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.thriftscala.EmbeddingType
+import com.twitter.simclusters_v2.thriftscala.ModelVersion
+import com.twitter.strato.response.Err
+import com.twitter.storehaus.ReadableStore
+import com.twitter.timelines.configapi.Params
+import com.twitter.topic_recos.common.Configs.ConsumerTopicEmbeddingType
+import com.twitter.topic_recos.common.Configs.DefaultModelVersion
+import com.twitter.topic_recos.common.Configs.ProducerTopicEmbeddingType
+import com.twitter.topic_recos.common.Configs.TweetEmbeddingType
+import com.twitter.topiclisting.TopicListingViewerContext
+import com.twitter.topic_recos.common.LocaleUtil
+import com.twitter.topiclisting.AnnotationRuleProvider
+import com.twitter.tsp.common.DeciderConstants
+import com.twitter.tsp.common.LoadShedder
+import com.twitter.tsp.common.RecTargetFactory
+import com.twitter.tsp.common.TopicSocialProofDecider
+import com.twitter.tsp.common.TopicSocialProofParams
+import com.twitter.tsp.stores.TopicSocialProofStore
+import com.twitter.tsp.stores.TopicSocialProofStore.TopicSocialProof
+import com.twitter.tsp.stores.UttTopicFilterStore
+import com.twitter.tsp.stores.TopicTweetsCosineSimilarityAggregateStore.ScoreKey
+import com.twitter.tsp.thriftscala.MetricTag
+import com.twitter.tsp.thriftscala.TopicFollowType
+import com.twitter.tsp.thriftscala.TopicListingSetting
+import com.twitter.tsp.thriftscala.TopicSocialProofRequest
+import com.twitter.tsp.thriftscala.TopicSocialProofResponse
+import com.twitter.tsp.thriftscala.TopicWithScore
+import com.twitter.tsp.thriftscala.TspTweetInfo
+import com.twitter.tsp.utils.HealthSignalsUtils
+import com.twitter.util.Future
+import com.twitter.util.Timer
+import com.twitter.util.Duration
+import com.twitter.util.TimeoutException
+
+import scala.util.Random
+
+class TopicSocialProofHandler(
+  topicSocialProofStore: ReadableStore[TopicSocialProofStore.Query, Seq[TopicSocialProof]],
+  tweetInfoStore: ReadableStore[TweetId, TspTweetInfo],
+  uttTopicFilterStore: UttTopicFilterStore,
+  recTargetFactory: RecTargetFactory,
+  decider: TopicSocialProofDecider,
+  statsReceiver: StatsReceiver,
+  loadShedder: LoadShedder,
+  timer: Timer) {
+
+  import TopicSocialProofHandler._
+
+  def getTopicSocialProofResponse(
+    request: TopicSocialProofRequest
+  ): Future[TopicSocialProofResponse] = {
+    val scopedStats = statsReceiver.scope(request.displayLocation.toString)
+    scopedStats.counter("fanoutRequests").incr(request.tweetIds.size)
+    scopedStats.stat("numTweetsPerRequest").add(request.tweetIds.size)
+    StatsUtil.trackBlockStats(scopedStats) {
+      recTargetFactory
+        .buildRecTopicSocialProofTarget(request).flatMap { target =>
+          val enableCosineSimilarityScoreCalculation =
+            decider.isAvailable(DeciderConstants.enableTopicSocialProofScore)
+
+          val semanticCoreVersionId =
+            target.params(TopicSocialProofParams.TopicTweetsSemanticCoreVersionId)
+
+          val semanticCoreVersionIdsSet =
+            target.params(TopicSocialProofParams.TopicTweetsSemanticCoreVersionIdsSet)
+
+          val allowListWithTopicFollowTypeFut = uttTopicFilterStore
+            .getAllowListTopicsForUser(
+              request.userId,
+              request.topicListingSetting,
+              TopicListingViewerContext
+                .fromThrift(request.context).copy(languageCode =
+                  LocaleUtil.getStandardLanguageCode(request.context.languageCode)),
+              request.bypassModes.map(_.toSet)
+            ).rescue {
+              case _ =>
+                scopedStats.counter("uttTopicFilterStoreFailure").incr()
+                Future.value(Map.empty[SemanticCoreEntityId, Option[TopicFollowType]])
+            }
+
+          val tweetInfoMapFut: Future[Map[TweetId, Option[TspTweetInfo]]] = Future
+            .collect(
+              tweetInfoStore.multiGet(request.tweetIds.toSet)
+            ).raiseWithin(TweetInfoStoreTimeout)(timer).rescue {
+              case _: TimeoutException =>
+                scopedStats.counter("tweetInfoStoreTimeout").incr()
+                Future.value(Map.empty[TweetId, Option[TspTweetInfo]])
+              case _ =>
+                scopedStats.counter("tweetInfoStoreFailure").incr()
+                Future.value(Map.empty[TweetId, Option[TspTweetInfo]])
+            }
+
+          val definedTweetInfoMapFut =
+            keepTweetsWithTweetInfoAndLanguage(tweetInfoMapFut, request.displayLocation.toString)
+
+          Future
+            .join(definedTweetInfoMapFut, allowListWithTopicFollowTypeFut).map {
+              case (tweetInfoMap, allowListWithTopicFollowType) =>
+                val tweetIdsToQuery = tweetInfoMap.keys.toSet
+                val topicProofQueries =
+                  tweetIdsToQuery.map { tweetId =>
+                    TopicSocialProofStore.Query(
+                      TopicSocialProofStore.CacheableQuery(
+                        tweetId = tweetId,
+                        tweetLanguage = LocaleUtil.getSupportedStandardLanguageCodeWithDefault(
+                          tweetInfoMap.getOrElse(tweetId, None).flatMap {
+                            _.language
+                          }),
+                        enableCosineSimilarityScoreCalculation =
+                          enableCosineSimilarityScoreCalculation
+                      ),
+                      allowedSemanticCoreVersionIds = semanticCoreVersionIdsSet
+                    )
+                  }
+
+                val topicSocialProofsFut: Future[Map[TweetId, Seq[TopicSocialProof]]] = {
+                  Future
+                    .collect(topicSocialProofStore.multiGet(topicProofQueries)).map(_.map {
+                      case (query, results) =>
+                        query.cacheableQuery.tweetId -> results.toSeq.flatten.filter(
+                          _.semanticCoreVersionId == semanticCoreVersionId)
+                    })
+                }.raiseWithin(TopicSocialProofStoreTimeout)(timer).rescue {
+                  case _: TimeoutException =>
+                    scopedStats.counter("topicSocialProofStoreTimeout").incr()
+                    Future(Map.empty[TweetId, Seq[TopicSocialProof]])
+                  case _ =>
+                    scopedStats.counter("topicSocialProofStoreFailure").incr()
+                    Future(Map.empty[TweetId, Seq[TopicSocialProof]])
+                }
+
+                val random = new Random(seed = request.userId.toInt)
+
+                topicSocialProofsFut.map { topicSocialProofs =>
+                  val filteredTopicSocialProofs = filterByAllowedList(
+                    topicSocialProofs,
+                    request.topicListingSetting,
+                    allowListWithTopicFollowType.keySet
+                  )
+
+                  val filteredTopicSocialProofsEmptyCount: Int =
+                    filteredTopicSocialProofs.count {
+                      case (_, topicSocialProofs: Seq[TopicSocialProof]) =>
+                        topicSocialProofs.isEmpty
+                    }
+
+                  scopedStats
+                    .counter("filteredTopicSocialProofsCount").incr(filteredTopicSocialProofs.size)
+                  scopedStats
+                    .counter("filteredTopicSocialProofsEmptyCount").incr(
+                      filteredTopicSocialProofsEmptyCount)
+
+                  if (isCrTopicTweets(request)) {
+                    val socialProofs = filteredTopicSocialProofs.mapValues(_.flatMap { topicProof =>
+                      val topicWithScores = buildTopicWithRandomScore(
+                        topicProof,
+                        allowListWithTopicFollowType,
+                        random
+                      )
+                      topicWithScores
+                    })
+                    TopicSocialProofResponse(socialProofs)
+                  } else {
+                    val socialProofs = filteredTopicSocialProofs.mapValues(_.flatMap { topicProof =>
+                      getTopicProofScore(
+                        topicProof = topicProof,
+                        allowListWithTopicFollowType = allowListWithTopicFollowType,
+                        params = target.params,
+                        random = random,
+                        statsReceiver = statsReceiver
+                      )
+
+                    }.sortBy(-_.score).take(MaxCandidates))
+
+                    val personalizedContextSocialProofs =
+                      if (target.params(TopicSocialProofParams.EnablePersonalizedContextTopics)) {
+                        val personalizedContextEligibility =
+                          checkPersonalizedContextsEligibility(
+                            target.params,
+                            allowListWithTopicFollowType)
+                        val filteredTweets =
+                          filterPersonalizedContexts(socialProofs, tweetInfoMap, target.params)
+                        backfillPersonalizedContexts(
+                          allowListWithTopicFollowType,
+                          filteredTweets,
+                          request.tags.getOrElse(Map.empty),
+                          personalizedContextEligibility)
+                      } else {
+                        Map.empty[TweetId, Seq[TopicWithScore]]
+                      }
+
+                    val mergedSocialProofs = socialProofs.map {
+                      case (tweetId, proofs) =>
+                        (
+                          tweetId,
+                          proofs
+                            ++ personalizedContextSocialProofs.getOrElse(tweetId, Seq.empty))
+                    }
+
+                    // Note that we will NOT filter out tweets with no TSP in either case
+                    TopicSocialProofResponse(mergedSocialProofs)
+                  }
+                }
+            }
+        }.flatten.raiseWithin(Timeout)(timer).rescue {
+          case _: ClientDiscardedRequestException =>
+            scopedStats.counter("ClientDiscardedRequestException").incr()
+            Future.value(DefaultResponse)
+          case err: Err if err.code == Err.Cancelled =>
+            scopedStats.counter("CancelledErr").incr()
+            Future.value(DefaultResponse)
+          case _ =>
+            scopedStats.counter("FailedRequests").incr()
+            Future.value(DefaultResponse)
+        }
+    }
+  }
+
+  /**
+   * Fetch the Score for each Topic Social Proof
+   */
+  private def getTopicProofScore(
+    topicProof: TopicSocialProof,
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]],
+    params: Params,
+    random: Random,
+    statsReceiver: StatsReceiver
+  ): Option[TopicWithScore] = {
+    val scopedStats = statsReceiver.scope("getTopicProofScores")
+    val enableTweetToTopicScoreRanking =
+      params(TopicSocialProofParams.EnableTweetToTopicScoreRanking)
+
+    val minTweetToTopicCosineSimilarityThreshold =
+      params(TopicSocialProofParams.TweetToTopicCosineSimilarityThreshold)
+
+    val topicWithScore =
+      if (enableTweetToTopicScoreRanking) {
+        scopedStats.counter("enableTweetToTopicScoreRanking").incr()
+        buildTopicWithValidScore(
+          topicProof,
+          TweetEmbeddingType,
+          Some(ConsumerTopicEmbeddingType),
+          Some(ProducerTopicEmbeddingType),
+          allowListWithTopicFollowType,
+          DefaultModelVersion,
+          minTweetToTopicCosineSimilarityThreshold
+        )
+      } else {
+        scopedStats.counter("buildTopicWithRandomScore").incr()
+        buildTopicWithRandomScore(
+          topicProof,
+          allowListWithTopicFollowType,
+          random
+        )
+      }
+    topicWithScore
+
+  }
+
+  private[handlers] def isCrTopicTweets(
+    request: TopicSocialProofRequest
+  ): Boolean = {
+    // CrTopic (across a variety of DisplayLocations) is the only use case with TopicListingSetting.All
+    request.topicListingSetting == TopicListingSetting.All
+  }
+
+  /**
+   * Consolidate logics relevant to whether only quality topics should be enabled for Implicit Follows
+   */
+
+  /***
+   * Consolidate logics relevant to whether Personalized Contexts backfilling should be enabled
+   */
+  private[handlers] def checkPersonalizedContextsEligibility(
+    params: Params,
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]]
+  ): PersonalizedContextEligibility = {
+    val scopedStats = statsReceiver.scope("checkPersonalizedContextsEligibility")
+    val isRecentFavInAllowlist = allowListWithTopicFollowType
+      .contains(AnnotationRuleProvider.recentFavTopicId)
+
+    val isRecentFavEligible =
+      isRecentFavInAllowlist && params(TopicSocialProofParams.EnableRecentEngagementsTopic)
+    if (isRecentFavEligible)
+      scopedStats.counter("isRecentFavEligible").incr()
+
+    val isRecentRetweetInAllowlist = allowListWithTopicFollowType
+      .contains(AnnotationRuleProvider.recentRetweetTopicId)
+
+    val isRecentRetweetEligible =
+      isRecentRetweetInAllowlist && params(TopicSocialProofParams.EnableRecentEngagementsTopic)
+    if (isRecentRetweetEligible)
+      scopedStats.counter("isRecentRetweetEligible").incr()
+
+    val isYMLInAllowlist = allowListWithTopicFollowType
+      .contains(AnnotationRuleProvider.youMightLikeTopicId)
+
+    val isYMLEligible =
+      isYMLInAllowlist && params(TopicSocialProofParams.EnableYouMightLikeTopic)
+    if (isYMLEligible)
+      scopedStats.counter("isYMLEligible").incr()
+
+    PersonalizedContextEligibility(isRecentFavEligible, isRecentRetweetEligible, isYMLEligible)
+  }
+
+  private[handlers] def filterPersonalizedContexts(
+    socialProofs: Map[TweetId, Seq[TopicWithScore]],
+    tweetInfoMap: Map[TweetId, Option[TspTweetInfo]],
+    params: Params
+  ): Map[TweetId, Seq[TopicWithScore]] = {
+    val filters: Seq[(Option[TspTweetInfo], Params) => Boolean] = Seq(
+      healthSignalsFilter,
+      tweetLanguageFilter
+    )
+    applyFilters(socialProofs, tweetInfoMap, params, filters)
+  }
+
+  /** *
+   * filter tweets with None tweetInfo and undefined language
+   */
+  private def keepTweetsWithTweetInfoAndLanguage(
+    tweetInfoMapFut: Future[Map[TweetId, Option[TspTweetInfo]]],
+    displayLocation: String
+  ): Future[Map[TweetId, Option[TspTweetInfo]]] = {
+    val scopedStats = statsReceiver.scope(displayLocation)
+    tweetInfoMapFut.map { tweetInfoMap =>
+      val filteredTweetInfoMap = tweetInfoMap.filter {
+        case (_, optTweetInfo: Option[TspTweetInfo]) =>
+          if (optTweetInfo.isEmpty) {
+            scopedStats.counter("undefinedTweetInfoCount").incr()
+          }
+
+          optTweetInfo.exists { tweetInfo: TspTweetInfo =>
+            {
+              if (tweetInfo.language.isEmpty) {
+                scopedStats.counter("undefinedLanguageCount").incr()
+              }
+              tweetInfo.language.isDefined
+            }
+          }
+
+      }
+      val undefinedTweetInfoOrLangCount = tweetInfoMap.size - filteredTweetInfoMap.size
+      scopedStats.counter("undefinedTweetInfoOrLangCount").incr(undefinedTweetInfoOrLangCount)
+
+      scopedStats.counter("TweetInfoCount").incr(tweetInfoMap.size)
+
+      filteredTweetInfoMap
+    }
+  }
+
+  /***
+   * filter tweets with NO evergreen topic social proofs by their health signal scores & tweet languages
+   * i.e., tweets that are possible to be converted into Personalized Context topic tweets
+   * TBD: whether we are going to apply filters to all topic tweet candidates
+   */
+  private def applyFilters(
+    socialProofs: Map[TweetId, Seq[TopicWithScore]],
+    tweetInfoMap: Map[TweetId, Option[TspTweetInfo]],
+    params: Params,
+    filters: Seq[(Option[TspTweetInfo], Params) => Boolean]
+  ): Map[TweetId, Seq[TopicWithScore]] = {
+    socialProofs.collect {
+      case (tweetId, socialProofs) if socialProofs.nonEmpty || filters.forall { filter =>
+            filter(tweetInfoMap.getOrElse(tweetId, None), params)
+          } =>
+        tweetId -> socialProofs
+    }
+  }
+
+  private def healthSignalsFilter(
+    tweetInfoOpt: Option[TspTweetInfo],
+    params: Params
+  ): Boolean = {
+    !params(
+      TopicSocialProofParams.EnableTopicTweetHealthFilterPersonalizedContexts) || HealthSignalsUtils
+      .isHealthyTweet(tweetInfoOpt)
+  }
+
+  private def tweetLanguageFilter(
+    tweetInfoOpt: Option[TspTweetInfo],
+    params: Params
+  ): Boolean = {
+    PersonalizedContextTopicsAllowedLanguageSet
+      .contains(tweetInfoOpt.flatMap(_.language).getOrElse(LocaleUtil.DefaultLanguage))
+  }
+
+  private[handlers] def backfillPersonalizedContexts(
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]],
+    socialProofs: Map[TweetId, Seq[TopicWithScore]],
+    metricTagsMap: scala.collection.Map[TweetId, scala.collection.Set[MetricTag]],
+    personalizedContextEligibility: PersonalizedContextEligibility
+  ): Map[TweetId, Seq[TopicWithScore]] = {
+    val scopedStats = statsReceiver.scope("backfillPersonalizedContexts")
+    socialProofs.map {
+      case (tweetId, topicWithScores) =>
+        if (topicWithScores.nonEmpty) {
+          tweetId -> Seq.empty
+        } else {
+          val metricTagContainsTweetFav = metricTagsMap
+            .getOrElse(tweetId, Set.empty[MetricTag]).contains(MetricTag.TweetFavorite)
+          val backfillRecentFav =
+            personalizedContextEligibility.isRecentFavEligible && metricTagContainsTweetFav
+          if (metricTagContainsTweetFav)
+            scopedStats.counter("MetricTag.TweetFavorite").incr()
+          if (backfillRecentFav)
+            scopedStats.counter("backfillRecentFav").incr()
+
+          val metricTagContainsRetweet = metricTagsMap
+            .getOrElse(tweetId, Set.empty[MetricTag]).contains(MetricTag.Retweet)
+          val backfillRecentRetweet =
+            personalizedContextEligibility.isRecentRetweetEligible && metricTagContainsRetweet
+          if (metricTagContainsRetweet)
+            scopedStats.counter("MetricTag.Retweet").incr()
+          if (backfillRecentRetweet)
+            scopedStats.counter("backfillRecentRetweet").incr()
+
+          val metricTagContainsRecentSearches = metricTagsMap
+            .getOrElse(tweetId, Set.empty[MetricTag]).contains(
+              MetricTag.InterestsRankerRecentSearches)
+
+          val backfillYML = personalizedContextEligibility.isYMLEligible
+          if (backfillYML)
+            scopedStats.counter("backfillYML").incr()
+
+          tweetId -> buildBackfillTopics(
+            allowListWithTopicFollowType,
+            backfillRecentFav,
+            backfillRecentRetweet,
+            backfillYML)
+        }
+    }
+  }
+
+  private def buildBackfillTopics(
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]],
+    backfillRecentFav: Boolean,
+    backfillRecentRetweet: Boolean,
+    backfillYML: Boolean
+  ): Seq[TopicWithScore] = {
+    Seq(
+      if (backfillRecentFav) {
+        Some(
+          TopicWithScore(
+            topicId = AnnotationRuleProvider.recentFavTopicId,
+            score = 1.0,
+            topicFollowType = allowListWithTopicFollowType
+              .getOrElse(AnnotationRuleProvider.recentFavTopicId, None)
+          ))
+      } else { None },
+      if (backfillRecentRetweet) {
+        Some(
+          TopicWithScore(
+            topicId = AnnotationRuleProvider.recentRetweetTopicId,
+            score = 1.0,
+            topicFollowType = allowListWithTopicFollowType
+              .getOrElse(AnnotationRuleProvider.recentRetweetTopicId, None)
+          ))
+      } else { None },
+      if (backfillYML) {
+        Some(
+          TopicWithScore(
+            topicId = AnnotationRuleProvider.youMightLikeTopicId,
+            score = 1.0,
+            topicFollowType = allowListWithTopicFollowType
+              .getOrElse(AnnotationRuleProvider.youMightLikeTopicId, None)
+          ))
+      } else { None }
+    ).flatten
+  }
+
+  def toReadableStore: ReadableStore[TopicSocialProofRequest, TopicSocialProofResponse] = {
+    new ReadableStore[TopicSocialProofRequest, TopicSocialProofResponse] {
+      override def get(k: TopicSocialProofRequest): Future[Option[TopicSocialProofResponse]] = {
+        val displayLocation = k.displayLocation.toString
+        loadShedder(displayLocation) {
+          getTopicSocialProofResponse(k).map(Some(_))
+        }.rescue {
+          case LoadShedder.LoadSheddingException =>
+            statsReceiver.scope(displayLocation).counter("LoadSheddingException").incr()
+            Future.None
+          case _ =>
+            statsReceiver.scope(displayLocation).counter("Exception").incr()
+            Future.None
+        }
+      }
+    }
+  }
+}
+
+object TopicSocialProofHandler {
+
+  private val MaxCandidates = 10
+  // Currently we do hardcode for the language check of PersonalizedContexts Topics
+  private val PersonalizedContextTopicsAllowedLanguageSet: Set[String] =
+    Set("pt", "ko", "es", "ja", "tr", "id", "en", "hi", "ar", "fr", "ru")
+
+  private val Timeout: Duration = 200.milliseconds
+  private val TopicSocialProofStoreTimeout: Duration = 40.milliseconds
+  private val TweetInfoStoreTimeout: Duration = 60.milliseconds
+  private val DefaultResponse: TopicSocialProofResponse = TopicSocialProofResponse(Map.empty)
+
+  case class PersonalizedContextEligibility(
+    isRecentFavEligible: Boolean,
+    isRecentRetweetEligible: Boolean,
+    isYMLEligible: Boolean)
+
+  /**
+   * Calculate the Topic Scores for each (tweet, topic), filter out topic proofs whose scores do not
+   * pass the minimum threshold
+   */
+  private[handlers] def buildTopicWithValidScore(
+    topicProof: TopicSocialProof,
+    tweetEmbeddingType: EmbeddingType,
+    maybeConsumerEmbeddingType: Option[EmbeddingType],
+    maybeProducerEmbeddingType: Option[EmbeddingType],
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]],
+    simClustersModelVersion: ModelVersion,
+    minTweetToTopicCosineSimilarityThreshold: Double
+  ): Option[TopicWithScore] = {
+
+    val consumerScore = maybeConsumerEmbeddingType
+      .flatMap { consumerEmbeddingType =>
+        topicProof.scores.get(
+          ScoreKey(consumerEmbeddingType, tweetEmbeddingType, simClustersModelVersion))
+      }.getOrElse(0.0)
+
+    val producerScore = maybeProducerEmbeddingType
+      .flatMap { producerEmbeddingType =>
+        topicProof.scores.get(
+          ScoreKey(producerEmbeddingType, tweetEmbeddingType, simClustersModelVersion))
+      }.getOrElse(0.0)
+
+    val combinedScore = consumerScore + producerScore
+    if (combinedScore > minTweetToTopicCosineSimilarityThreshold || topicProof.ignoreSimClusterFiltering) {
+      Some(
+        TopicWithScore(
+          topicId = topicProof.topicId.entityId,
+          score = combinedScore,
+          topicFollowType =
+            allowListWithTopicFollowType.getOrElse(topicProof.topicId.entityId, None)))
+    } else {
+      None
+    }
+  }
+
+  private[handlers] def buildTopicWithRandomScore(
+    topicSocialProof: TopicSocialProof,
+    allowListWithTopicFollowType: Map[SemanticCoreEntityId, Option[TopicFollowType]],
+    random: Random
+  ): Option[TopicWithScore] = {
+
+    Some(
+      TopicWithScore(
+        topicId = topicSocialProof.topicId.entityId,
+        score = random.nextDouble(),
+        topicFollowType =
+          allowListWithTopicFollowType.getOrElse(topicSocialProof.topicId.entityId, None)
+      ))
+  }
+
+  /**
+   * Filter all the non-qualified Topic Social Proof
+   */
+  private[handlers] def filterByAllowedList(
+    topicProofs: Map[TweetId, Seq[TopicSocialProof]],
+    setting: TopicListingSetting,
+    allowList: Set[SemanticCoreEntityId]
+  ): Map[TweetId, Seq[TopicSocialProof]] = {
+    setting match {
+      case TopicListingSetting.All =>
+        // Return all the topics
+        topicProofs
+      case _ =>
+        topicProofs.mapValues(
+          _.filter(topicProof => allowList.contains(topicProof.topicId.entityId)))
+    }
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/handlers/UttChildrenWarmupHandler.scala

@@ -0,0 +1,40 @@
+package com.twitter.tsp.handlers
+
+import com.twitter.inject.utils.Handler
+import com.twitter.topiclisting.FollowableTopicProductId
+import com.twitter.topiclisting.ProductId
+import com.twitter.topiclisting.TopicListingViewerContext
+import com.twitter.topiclisting.utt.UttLocalization
+import com.twitter.util.logging.Logging
+import javax.inject.Inject
+import javax.inject.Singleton
+
+/** *
+ * We configure Warmer to help warm up the cache hit rate under `CachedUttClient/get_utt_taxonomy/cache_hit_rate`
+ * In uttLocalization.getRecommendableTopics, we fetch all topics exist in UTT, and yet the process
+ * is in fact fetching the complete UTT tree struct (by calling getUttChildren recursively), which could take 1 sec
+ * Once we have the topics, we stored them in in-memory cache, and the cache hit rate is > 99%
+ *
+ */
+@Singleton
+class UttChildrenWarmupHandler @Inject() (uttLocalization: UttLocalization)
+    extends Handler
+    with Logging {
+
+  /** Executes the function of this handler. *   */
+  override def handle(): Unit = {
+    uttLocalization
+      .getRecommendableTopics(
+        productId = ProductId.Followable,
+        viewerContext = TopicListingViewerContext(languageCode = Some("en")),
+        enableInternationalTopics = true,
+        followableTopicProductId = FollowableTopicProductId.AllFollowable
+      )
+      .onSuccess { result =>
+        logger.info(s"successfully warmed up UttChildren. TopicId length = ${result.size}")
+      }
+      .onFailure { throwable =>
+        logger.info(s"failed to warm up UttChildren. Throwable = ${throwable}")
+      }
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/BUILD

@@ -0,0 +1,30 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "3rdparty/jvm/com/twitter/bijection:scrooge",
+        "3rdparty/jvm/com/twitter/storehaus:memcache",
+        "escherbird/src/scala/com/twitter/escherbird/util/uttclient",
+        "escherbird/src/thrift/com/twitter/escherbird/utt:strato-columns-scala",
+        "finagle-internal/mtls/src/main/scala/com/twitter/finagle/mtls/authentication",
+        "finatra-internal/mtls-thriftmux/src/main/scala",
+        "finatra/inject/inject-core/src/main/scala",
+        "finatra/inject/inject-thrift-client",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store/strato",
+        "hermit/hermit-core/src/main/scala/com/twitter/hermit/store/common",
+        "src/scala/com/twitter/storehaus_internal/memcache",
+        "src/scala/com/twitter/storehaus_internal/util",
+        "src/thrift/com/twitter/gizmoduck:thrift-scala",
+        "src/thrift/com/twitter/gizmoduck:user-thrift-scala",
+        "stitch/stitch-storehaus",
+        "stitch/stitch-tweetypie/src/main/scala",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/common",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/stores",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/utils",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+        "topiclisting/common/src/main/scala/com/twitter/topiclisting/clients",
+        "topiclisting/topiclisting-utt/src/main/scala/com/twitter/topiclisting/utt",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/GizmoduckUserModule.scala

@@ -0,0 +1,35 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Module
+import com.twitter.finagle.ThriftMux
+import com.twitter.finagle.mtls.authentication.ServiceIdentifier
+import com.twitter.finagle.mtls.client.MtlsStackClient._
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.finagle.thrift.ClientId
+import com.twitter.finatra.mtls.thriftmux.modules.MtlsClient
+import com.twitter.gizmoduck.thriftscala.UserService
+import com.twitter.inject.Injector
+import com.twitter.inject.thrift.modules.ThriftMethodBuilderClientModule
+
+object GizmoduckUserModule
+    extends ThriftMethodBuilderClientModule[
+      UserService.ServicePerEndpoint,
+      UserService.MethodPerEndpoint
+    ]
+    with MtlsClient {
+
+  override val label: String = "gizmoduck"
+  override val dest: String = "/s/gizmoduck/gizmoduck"
+  override val modules: Seq[Module] = Seq(TSPClientIdModule)
+
+  override def configureThriftMuxClient(
+    injector: Injector,
+    client: ThriftMux.Client
+  ): ThriftMux.Client = {
+    super
+      .configureThriftMuxClient(injector, client)
+      .withMutualTls(injector.instance[ServiceIdentifier])
+      .withClientId(injector.instance[ClientId])
+      .withStatsReceiver(injector.instance[StatsReceiver].scope("giz"))
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/RepresentationScorerStoreModule.scala

@@ -0,0 +1,47 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Module
+import com.google.inject.Provides
+import com.google.inject.Singleton
+import com.twitter.app.Flag
+import com.twitter.bijection.scrooge.BinaryScalaCodec
+import com.twitter.conversions.DurationOps._
+import com.twitter.finagle.memcached.{Client => MemClient}
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.hermit.store.common.ObservedMemcachedReadableStore
+import com.twitter.inject.TwitterModule
+import com.twitter.simclusters_v2.thriftscala.Score
+import com.twitter.simclusters_v2.thriftscala.ScoreId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.{Client => StratoClient}
+import com.twitter.tsp.stores.RepresentationScorerStore
+
+object RepresentationScorerStoreModule extends TwitterModule {
+  override def modules: Seq[Module] = Seq(UnifiedCacheClient)
+
+  private val tspRepresentationScoringColumnPath: Flag[String] = flag[String](
+    name = "tsp.representationScoringColumnPath",
+    default = "recommendations/representation_scorer/score",
+    help = "Strato column path for Representation Scorer Store"
+  )
+
+  @Provides
+  @Singleton
+  def providesRepresentationScorerStore(
+    statsReceiver: StatsReceiver,
+    stratoClient: StratoClient,
+    tspUnifiedCacheClient: MemClient
+  ): ReadableStore[ScoreId, Score] = {
+    val underlyingStore =
+      RepresentationScorerStore(stratoClient, tspRepresentationScoringColumnPath(), statsReceiver)
+    ObservedMemcachedReadableStore.fromCacheClient(
+      backingStore = underlyingStore,
+      cacheClient = tspUnifiedCacheClient,
+      ttl = 2.hours
+    )(
+      valueInjection = BinaryScalaCodec(Score),
+      statsReceiver = statsReceiver.scope("RepresentationScorerStore"),
+      keyToString = { k: ScoreId => s"rsx/$k" }
+    )
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TSPClientIdModule.scala

@@ -0,0 +1,14 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.twitter.finagle.thrift.ClientId
+import com.twitter.inject.TwitterModule
+import javax.inject.Singleton
+
+object TSPClientIdModule extends TwitterModule {
+  private val clientIdFlag = flag("thrift.clientId", "topic-social-proof.prod", "Thrift client id")
+
+  @Provides
+  @Singleton
+  def providesClientId: ClientId = ClientId(clientIdFlag())
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TopicListingModule.scala

@@ -0,0 +1,17 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.inject.TwitterModule
+import com.twitter.topiclisting.TopicListing
+import com.twitter.topiclisting.TopicListingBuilder
+import javax.inject.Singleton
+
+object TopicListingModule extends TwitterModule {
+
+  @Provides
+  @Singleton
+  def providesTopicListing(statsReceiver: StatsReceiver): TopicListing = {
+    new TopicListingBuilder(statsReceiver.scope(namespace = "TopicListingBuilder")).build
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TopicSocialProofStoreModule.scala

@@ -0,0 +1,68 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Module
+import com.google.inject.Provides
+import com.google.inject.Singleton
+import com.twitter.conversions.DurationOps._
+import com.twitter.finagle.memcached.{Client => MemClient}
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.hermit.store.common.ObservedCachedReadableStore
+import com.twitter.hermit.store.common.ObservedMemcachedReadableStore
+import com.twitter.hermit.store.common.ObservedReadableStore
+import com.twitter.inject.TwitterModule
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.thriftscala.Score
+import com.twitter.simclusters_v2.thriftscala.ScoreId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.{Client => StratoClient}
+import com.twitter.tsp.stores.SemanticCoreAnnotationStore
+import com.twitter.tsp.stores.TopicSocialProofStore
+import com.twitter.tsp.stores.TopicSocialProofStore.TopicSocialProof
+import com.twitter.tsp.utils.LZ4Injection
+import com.twitter.tsp.utils.SeqObjectInjection
+
+object TopicSocialProofStoreModule extends TwitterModule {
+  override def modules: Seq[Module] = Seq(UnifiedCacheClient)
+
+  @Provides
+  @Singleton
+  def providesTopicSocialProofStore(
+    representationScorerStore: ReadableStore[ScoreId, Score],
+    statsReceiver: StatsReceiver,
+    stratoClient: StratoClient,
+    tspUnifiedCacheClient: MemClient,
+  ): ReadableStore[TopicSocialProofStore.Query, Seq[TopicSocialProof]] = {
+    val semanticCoreAnnotationStore: ReadableStore[TweetId, Seq[
+      SemanticCoreAnnotationStore.TopicAnnotation
+    ]] = ObservedReadableStore(
+      SemanticCoreAnnotationStore(SemanticCoreAnnotationStore.getStratoStore(stratoClient))
+    )(statsReceiver.scope("SemanticCoreAnnotationStore"))
+
+    val underlyingStore = TopicSocialProofStore(
+      representationScorerStore,
+      semanticCoreAnnotationStore
+    )(statsReceiver.scope("TopicSocialProofStore"))
+
+    val memcachedStore = ObservedMemcachedReadableStore.fromCacheClient(
+      backingStore = underlyingStore,
+      cacheClient = tspUnifiedCacheClient,
+      ttl = 15.minutes,
+      asyncUpdate = true
+    )(
+      valueInjection = LZ4Injection.compose(SeqObjectInjection[TopicSocialProof]()),
+      statsReceiver = statsReceiver.scope("memCachedTopicSocialProofStore"),
+      keyToString = { k: TopicSocialProofStore.Query => s"tsps/${k.cacheableQuery}" }
+    )
+
+    val inMemoryCachedStore =
+      ObservedCachedReadableStore.from[TopicSocialProofStore.Query, Seq[TopicSocialProof]](
+        memcachedStore,
+        ttl = 10.minutes,
+        maxKeys = 16777215, // ~ avg 160B, < 3000MB
+        cacheName = "topic_social_proof_cache",
+        windowSize = 10000L
+      )(statsReceiver.scope("InMemoryCachedTopicSocialProofStore"))
+
+    inMemoryCachedStore
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TopicTweetCosineSimilarityAggregateStoreModule.scala

@@ -0,0 +1,26 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.google.inject.Singleton
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.inject.TwitterModule
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.thriftscala.Score
+import com.twitter.simclusters_v2.thriftscala.ScoreId
+import com.twitter.simclusters_v2.thriftscala.TopicId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.tsp.stores.TopicTweetsCosineSimilarityAggregateStore
+import com.twitter.tsp.stores.TopicTweetsCosineSimilarityAggregateStore.ScoreKey
+
+object TopicTweetCosineSimilarityAggregateStoreModule extends TwitterModule {
+
+  @Provides
+  @Singleton
+  def providesTopicTweetCosineSimilarityAggregateStore(
+    representationScorerStore: ReadableStore[ScoreId, Score],
+    statsReceiver: StatsReceiver,
+  ): ReadableStore[(TopicId, TweetId, Seq[ScoreKey]), Map[ScoreKey, Double]] = {
+    TopicTweetsCosineSimilarityAggregateStore(representationScorerStore)(
+      statsReceiver.scope("topicTweetsCosineSimilarityAggregateStore"))
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TweetInfoStoreModule.scala

@@ -0,0 +1,130 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Module
+import com.google.inject.Provides
+import com.google.inject.Singleton
+import com.twitter.bijection.scrooge.BinaryScalaCodec
+import com.twitter.conversions.DurationOps._
+import com.twitter.finagle.memcached.{Client => MemClient}
+import com.twitter.finagle.mtls.authentication.ServiceIdentifier
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.store.health.TweetHealthModelStore
+import com.twitter.frigate.common.store.health.TweetHealthModelStore.TweetHealthModelStoreConfig
+import com.twitter.frigate.common.store.health.UserHealthModelStore
+import com.twitter.frigate.common.store.interests.UserId
+import com.twitter.frigate.thriftscala.TweetHealthScores
+import com.twitter.frigate.thriftscala.UserAgathaScores
+import com.twitter.hermit.store.common.DeciderableReadableStore
+import com.twitter.hermit.store.common.ObservedCachedReadableStore
+import com.twitter.hermit.store.common.ObservedMemcachedReadableStore
+import com.twitter.inject.TwitterModule
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.stitch.tweetypie.TweetyPie
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.{Client => StratoClient}
+import com.twitter.tsp.common.DeciderKey
+import com.twitter.tsp.common.TopicSocialProofDecider
+import com.twitter.tsp.stores.TweetInfoStore
+import com.twitter.tsp.stores.TweetyPieFieldsStore
+import com.twitter.tweetypie.thriftscala.TweetService
+import com.twitter.tsp.thriftscala.TspTweetInfo
+import com.twitter.util.JavaTimer
+import com.twitter.util.Timer
+
+object TweetInfoStoreModule extends TwitterModule {
+  override def modules: Seq[Module] = Seq(UnifiedCacheClient)
+  implicit val timer: Timer = new JavaTimer(true)
+
+  @Provides
+  @Singleton
+  def providesTweetInfoStore(
+    decider: TopicSocialProofDecider,
+    serviceIdentifier: ServiceIdentifier,
+    statsReceiver: StatsReceiver,
+    stratoClient: StratoClient,
+    tspUnifiedCacheClient: MemClient,
+    tweetyPieService: TweetService.MethodPerEndpoint
+  ): ReadableStore[TweetId, TspTweetInfo] = {
+    val tweetHealthModelStore: ReadableStore[TweetId, TweetHealthScores] = {
+      val underlyingStore = TweetHealthModelStore.buildReadableStore(
+        stratoClient,
+        Some(
+          TweetHealthModelStoreConfig(
+            enablePBlock = true,
+            enableToxicity = true,
+            enablePSpammy = true,
+            enablePReported = true,
+            enableSpammyTweetContent = true,
+            enablePNegMultimodal = false))
+      )(statsReceiver.scope("UnderlyingTweetHealthModelStore"))
+
+      DeciderableReadableStore(
+        ObservedMemcachedReadableStore.fromCacheClient(
+          backingStore = underlyingStore,
+          cacheClient = tspUnifiedCacheClient,
+          ttl = 2.hours
+        )(
+          valueInjection = BinaryScalaCodec(TweetHealthScores),
+          statsReceiver = statsReceiver.scope("TweetHealthModelStore"),
+          keyToString = { k: TweetId => s"tHMS/$k" }
+        ),
+        decider.deciderGateBuilder.idGate(DeciderKey.enableHealthSignalsScoreDeciderKey),
+        statsReceiver.scope("TweetHealthModelStore")
+      )
+    }
+
+    val userHealthModelStore: ReadableStore[UserId, UserAgathaScores] = {
+      val underlyingStore =
+        UserHealthModelStore.buildReadableStore(stratoClient)(
+          statsReceiver.scope("UnderlyingUserHealthModelStore"))
+
+      DeciderableReadableStore(
+        ObservedMemcachedReadableStore.fromCacheClient(
+          backingStore = underlyingStore,
+          cacheClient = tspUnifiedCacheClient,
+          ttl = 18.hours
+        )(
+          valueInjection = BinaryScalaCodec(UserAgathaScores),
+          statsReceiver = statsReceiver.scope("UserHealthModelStore"),
+          keyToString = { k: UserId => s"uHMS/$k" }
+        ),
+        decider.deciderGateBuilder.idGate(DeciderKey.enableUserAgathaScoreDeciderKey),
+        statsReceiver.scope("UserHealthModelStore")
+      )
+    }
+
+    val tweetInfoStore: ReadableStore[TweetId, TspTweetInfo] = {
+      val underlyingStore = TweetInfoStore(
+        TweetyPieFieldsStore.getStoreFromTweetyPie(TweetyPie(tweetyPieService, statsReceiver)),
+        tweetHealthModelStore: ReadableStore[TweetId, TweetHealthScores],
+        userHealthModelStore: ReadableStore[UserId, UserAgathaScores],
+        timer: Timer
+      )(statsReceiver.scope("tweetInfoStore"))
+
+      val memcachedStore = ObservedMemcachedReadableStore.fromCacheClient(
+        backingStore = underlyingStore,
+        cacheClient = tspUnifiedCacheClient,
+        ttl = 15.minutes,
+        // Hydrating tweetInfo is now a required step for all candidates,
+        // hence we needed to tune these thresholds.
+        asyncUpdate = serviceIdentifier.environment == "prod"
+      )(
+        valueInjection = BinaryScalaCodec(TspTweetInfo),
+        statsReceiver = statsReceiver.scope("memCachedTweetInfoStore"),
+        keyToString = { k: TweetId => s"tIS/$k" }
+      )
+
+      val inMemoryStore = ObservedCachedReadableStore.from(
+        memcachedStore,
+        ttl = 15.minutes,
+        maxKeys = 8388607, // Check TweetInfo definition. size~92b. Around 736 MB
+        windowSize = 10000L,
+        cacheName = "tweet_info_cache",
+        maxMultiGetSize = 20
+      )(statsReceiver.scope("inMemoryCachedTweetInfoStore"))
+
+      inMemoryStore
+    }
+    tweetInfoStore
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/TweetyPieClientModule.scala

@@ -0,0 +1,63 @@
+package com.twitter.tsp
+package modules
+
+import com.google.inject.Module
+import com.google.inject.Provides
+import com.twitter.conversions.DurationOps.richDurationFromInt
+import com.twitter.finagle.ThriftMux
+import com.twitter.finagle.mtls.authentication.ServiceIdentifier
+import com.twitter.finagle.mtls.client.MtlsStackClient.MtlsThriftMuxClientSyntax
+import com.twitter.finagle.mux.ClientDiscardedRequestException
+import com.twitter.finagle.service.ReqRep
+import com.twitter.finagle.service.ResponseClass
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.finagle.thrift.ClientId
+import com.twitter.inject.Injector
+import com.twitter.inject.thrift.modules.ThriftMethodBuilderClientModule
+import com.twitter.tweetypie.thriftscala.TweetService
+import com.twitter.util.Duration
+import com.twitter.util.Throw
+import com.twitter.stitch.tweetypie.{TweetyPie => STweetyPie}
+import com.twitter.finatra.mtls.thriftmux.modules.MtlsClient
+import javax.inject.Singleton
+
+object TweetyPieClientModule
+    extends ThriftMethodBuilderClientModule[
+      TweetService.ServicePerEndpoint,
+      TweetService.MethodPerEndpoint
+    ]
+    with MtlsClient {
+  override val label = "tweetypie"
+  override val dest = "/s/tweetypie/tweetypie"
+  override val requestTimeout: Duration = 450.milliseconds
+
+  override val modules: Seq[Module] = Seq(TSPClientIdModule)
+
+  // We bump the success rate from the default of 0.8 to 0.9 since we're dropping the
+  // consecutive failures part of the default policy.
+  override def configureThriftMuxClient(
+    injector: Injector,
+    client: ThriftMux.Client
+  ): ThriftMux.Client =
+    super
+      .configureThriftMuxClient(injector, client)
+      .withMutualTls(injector.instance[ServiceIdentifier])
+      .withStatsReceiver(injector.instance[StatsReceiver].scope("clnt"))
+      .withClientId(injector.instance[ClientId])
+      .withResponseClassifier {
+        case ReqRep(_, Throw(_: ClientDiscardedRequestException)) => ResponseClass.Ignorable
+      }
+      .withSessionQualifier
+      .successRateFailureAccrual(successRate = 0.9, window = 30.seconds)
+      .withResponseClassifier {
+        case ReqRep(_, Throw(_: ClientDiscardedRequestException)) => ResponseClass.Ignorable
+      }
+
+  @Provides
+  @Singleton
+  def providesTweetyPie(
+    tweetyPieService: TweetService.MethodPerEndpoint
+  ): STweetyPie = {
+    STweetyPie(tweetyPieService)
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/UnifiedCacheClient.scala

@@ -0,0 +1,33 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.google.inject.Singleton
+import com.twitter.app.Flag
+import com.twitter.finagle.memcached.Client
+import com.twitter.finagle.mtls.authentication.ServiceIdentifier
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.inject.TwitterModule
+import com.twitter.storehaus_internal.memcache.MemcacheStore
+import com.twitter.storehaus_internal.util.ClientName
+import com.twitter.storehaus_internal.util.ZkEndPoint
+
+object UnifiedCacheClient extends TwitterModule {
+  val tspUnifiedCacheDest: Flag[String] = flag[String](
+    name = "tsp.unifiedCacheDest",
+    default = "/srv#/prod/local/cache/topic_social_proof_unified",
+    help = "Wily path to topic social proof unified cache"
+  )
+
+  @Provides
+  @Singleton
+  def provideUnifiedCacheClient(
+    serviceIdentifier: ServiceIdentifier,
+    statsReceiver: StatsReceiver,
+  ): Client =
+    MemcacheStore.memcachedClient(
+      name = ClientName("topic-social-proof-unified-memcache"),
+      dest = ZkEndPoint(tspUnifiedCacheDest()),
+      statsReceiver = statsReceiver.scope("cache_client"),
+      serviceIdentifier = serviceIdentifier
+    )
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/UttClientModule.scala

@@ -0,0 +1,41 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.twitter.escherbird.util.uttclient.CacheConfigV2
+import com.twitter.escherbird.util.uttclient.CachedUttClientV2
+import com.twitter.escherbird.util.uttclient.UttClientCacheConfigsV2
+import com.twitter.escherbird.utt.strato.thriftscala.Environment
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.inject.TwitterModule
+import com.twitter.strato.client.Client
+import com.twitter.topiclisting.clients.utt.UttClient
+import javax.inject.Singleton
+
+object UttClientModule extends TwitterModule {
+
+  @Provides
+  @Singleton
+  def providesUttClient(
+    stratoClient: Client,
+    statsReceiver: StatsReceiver
+  ): UttClient = {
+
+    // Save 2 ^ 18 UTTs. Promising 100% cache rate
+    lazy val defaultCacheConfigV2: CacheConfigV2 = CacheConfigV2(262143)
+    lazy val uttClientCacheConfigsV2: UttClientCacheConfigsV2 = UttClientCacheConfigsV2(
+      getTaxonomyConfig = defaultCacheConfigV2,
+      getUttTaxonomyConfig = defaultCacheConfigV2,
+      getLeafIds = defaultCacheConfigV2,
+      getLeafUttEntities = defaultCacheConfigV2
+    )
+
+    // CachedUttClient to use StratoClient
+    lazy val cachedUttClientV2: CachedUttClientV2 = new CachedUttClientV2(
+      stratoClient = stratoClient,
+      env = Environment.Prod,
+      cacheConfigs = uttClientCacheConfigsV2,
+      statsReceiver = statsReceiver.scope("CachedUttClient")
+    )
+    new UttClient(cachedUttClientV2, statsReceiver)
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/modules/UttLocalizationModule.scala

@@ -0,0 +1,27 @@
+package com.twitter.tsp.modules
+
+import com.google.inject.Provides
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.inject.TwitterModule
+import com.twitter.topiclisting.TopicListing
+import com.twitter.topiclisting.clients.utt.UttClient
+import com.twitter.topiclisting.utt.UttLocalization
+import com.twitter.topiclisting.utt.UttLocalizationImpl
+import javax.inject.Singleton
+
+object UttLocalizationModule extends TwitterModule {
+
+  @Provides
+  @Singleton
+  def providesUttLocalization(
+    topicListing: TopicListing,
+    uttClient: UttClient,
+    statsReceiver: StatsReceiver
+  ): UttLocalization = {
+    new UttLocalizationImpl(
+      topicListing,
+      uttClient,
+      statsReceiver
+    )
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/service/BUILD

@@ -0,0 +1,23 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "3rdparty/jvm/javax/inject:javax.inject",
+        "abdecider/src/main/scala",
+        "content-recommender/thrift/src/main/thrift:thrift-scala",
+        "hermit/hermit-core/src/main/scala/com/twitter/hermit/store/common",
+        "hermit/hermit-core/src/main/scala/com/twitter/hermit/store/gizmoduck",
+        "src/scala/com/twitter/topic_recos/stores",
+        "src/thrift/com/twitter/gizmoduck:thrift-scala",
+        "src/thrift/com/twitter/gizmoduck:user-thrift-scala",
+        "src/thrift/com/twitter/simclusters_v2:simclusters_v2-thrift-scala",
+        "stitch/stitch-storehaus",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/common",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/handlers",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/modules",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/stores",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/service/TopicSocialProofService.scala

@@ -0,0 +1,182 @@
+package com.twitter.tsp.service
+
+import com.twitter.abdecider.ABDeciderFactory
+import com.twitter.abdecider.LoggingABDecider
+import com.twitter.tsp.thriftscala.TspTweetInfo
+import com.twitter.discovery.common.configapi.FeatureContextBuilder
+import com.twitter.finagle.mtls.authentication.ServiceIdentifier
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.gizmoduck.thriftscala.LookupContext
+import com.twitter.gizmoduck.thriftscala.QueryFields
+import com.twitter.gizmoduck.thriftscala.User
+import com.twitter.gizmoduck.thriftscala.UserService
+import com.twitter.hermit.store.gizmoduck.GizmoduckUserStore
+import com.twitter.logging.Logger
+import com.twitter.simclusters_v2.common.SemanticCoreEntityId
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.common.UserId
+import com.twitter.spam.rtf.thriftscala.SafetyLevel
+import com.twitter.stitch.storehaus.StitchOfReadableStore
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.{Client => StratoClient}
+import com.twitter.timelines.configapi
+import com.twitter.timelines.configapi.CompositeConfig
+import com.twitter.tsp.common.FeatureSwitchConfig
+import com.twitter.tsp.common.FeatureSwitchesBuilder
+import com.twitter.tsp.common.LoadShedder
+import com.twitter.tsp.common.ParamsBuilder
+import com.twitter.tsp.common.RecTargetFactory
+import com.twitter.tsp.common.TopicSocialProofDecider
+import com.twitter.tsp.handlers.TopicSocialProofHandler
+import com.twitter.tsp.stores.LocalizedUttRecommendableTopicsStore
+import com.twitter.tsp.stores.LocalizedUttTopicNameRequest
+import com.twitter.tsp.stores.TopicResponses
+import com.twitter.tsp.stores.TopicSocialProofStore
+import com.twitter.tsp.stores.TopicSocialProofStore.TopicSocialProof
+import com.twitter.tsp.stores.TopicStore
+import com.twitter.tsp.stores.UttTopicFilterStore
+import com.twitter.tsp.thriftscala.TopicSocialProofRequest
+import com.twitter.tsp.thriftscala.TopicSocialProofResponse
+import com.twitter.util.JavaTimer
+import com.twitter.util.Timer
+import javax.inject.Inject
+import javax.inject.Singleton
+import com.twitter.topiclisting.TopicListing
+import com.twitter.topiclisting.utt.UttLocalization
+
+@Singleton
+class TopicSocialProofService @Inject() (
+  topicSocialProofStore: ReadableStore[TopicSocialProofStore.Query, Seq[TopicSocialProof]],
+  tweetInfoStore: ReadableStore[TweetId, TspTweetInfo],
+  serviceIdentifier: ServiceIdentifier,
+  stratoClient: StratoClient,
+  gizmoduck: UserService.MethodPerEndpoint,
+  topicListing: TopicListing,
+  uttLocalization: UttLocalization,
+  decider: TopicSocialProofDecider,
+  loadShedder: LoadShedder,
+  stats: StatsReceiver) {
+
+  import TopicSocialProofService._
+
+  private val statsReceiver = stats.scope("topic-social-proof-management")
+
+  private val isProd: Boolean = serviceIdentifier.environment == "prod"
+
+  private val optOutStratoStorePath: String =
+    if (isProd) "interests/optOutInterests" else "interests/staging/optOutInterests"
+
+  private val notInterestedInStorePath: String =
+    if (isProd) "interests/notInterestedTopicsGetter"
+    else "interests/staging/notInterestedTopicsGetter"
+
+  private val userOptOutTopicsStore: ReadableStore[UserId, TopicResponses] =
+    TopicStore.userOptOutTopicStore(stratoClient, optOutStratoStorePath)(
+      statsReceiver.scope("ints_interests_opt_out_store"))
+  private val explicitFollowingTopicsStore: ReadableStore[UserId, TopicResponses] =
+    TopicStore.explicitFollowingTopicStore(stratoClient)(
+      statsReceiver.scope("ints_explicit_following_interests_store"))
+  private val userNotInterestedInTopicsStore: ReadableStore[UserId, TopicResponses] =
+    TopicStore.notInterestedInTopicsStore(stratoClient, notInterestedInStorePath)(
+      statsReceiver.scope("ints_not_interested_in_store"))
+
+  private lazy val localizedUttRecommendableTopicsStore: ReadableStore[
+    LocalizedUttTopicNameRequest,
+    Set[
+      SemanticCoreEntityId
+    ]
+  ] = new LocalizedUttRecommendableTopicsStore(uttLocalization)
+
+  implicit val timer: Timer = new JavaTimer(true)
+
+  private lazy val uttTopicFilterStore = new UttTopicFilterStore(
+    topicListing = topicListing,
+    userOptOutTopicsStore = userOptOutTopicsStore,
+    explicitFollowingTopicsStore = explicitFollowingTopicsStore,
+    notInterestedTopicsStore = userNotInterestedInTopicsStore,
+    localizedUttRecommendableTopicsStore = localizedUttRecommendableTopicsStore,
+    timer = timer,
+    stats = statsReceiver.scope("UttTopicFilterStore")
+  )
+
+  private lazy val scribeLogger: Option[Logger] = Some(Logger.get("client_event"))
+
+  private lazy val abDecider: LoggingABDecider =
+    ABDeciderFactory(
+      abDeciderYmlPath = configRepoDirectory + "/abdecider/abdecider.yml",
+      scribeLogger = scribeLogger,
+      decider = None,
+      environment = Some("production"),
+    ).buildWithLogging()
+
+  private val builder: FeatureSwitchesBuilder = FeatureSwitchesBuilder(
+    statsReceiver = statsReceiver.scope("featureswitches-v2"),
+    abDecider = abDecider,
+    featuresDirectory = "features/topic-social-proof/main",
+    configRepoDirectory = configRepoDirectory,
+    addServiceDetailsFromAurora = !serviceIdentifier.isLocal,
+    fastRefresh = !isProd
+  )
+
+  private lazy val overridesConfig: configapi.Config = {
+    new CompositeConfig(
+      Seq(
+        FeatureSwitchConfig.config
+      )
+    )
+  }
+
+  private val featureContextBuilder: FeatureContextBuilder = FeatureContextBuilder(builder.build())
+
+  private val paramsBuilder: ParamsBuilder = ParamsBuilder(
+    featureContextBuilder,
+    abDecider,
+    overridesConfig,
+    statsReceiver.scope("params")
+  )
+
+  private val userStore: ReadableStore[UserId, User] = {
+    val queryFields: Set[QueryFields] = Set(
+      QueryFields.Profile,
+      QueryFields.Account,
+      QueryFields.Roles,
+      QueryFields.Discoverability,
+      QueryFields.Safety,
+      QueryFields.Takedowns
+    )
+    val context: LookupContext = LookupContext(safetyLevel = Some(SafetyLevel.Recommendations))
+
+    GizmoduckUserStore(
+      client = gizmoduck,
+      queryFields = queryFields,
+      context = context,
+      statsReceiver = statsReceiver.scope("gizmoduck")
+    )
+  }
+
+  private val recTargetFactory: RecTargetFactory = RecTargetFactory(
+    abDecider,
+    userStore,
+    paramsBuilder,
+    statsReceiver
+  )
+
+  private val topicSocialProofHandler =
+    new TopicSocialProofHandler(
+      topicSocialProofStore,
+      tweetInfoStore,
+      uttTopicFilterStore,
+      recTargetFactory,
+      decider,
+      statsReceiver.scope("TopicSocialProofHandler"),
+      loadShedder,
+      timer)
+
+  val topicSocialProofHandlerStoreStitch: TopicSocialProofRequest => com.twitter.stitch.Stitch[
+    TopicSocialProofResponse
+  ] = StitchOfReadableStore(topicSocialProofHandler.toReadableStore)
+}
+
+object TopicSocialProofService {
+  private val configRepoDirectory = "/usr/local/config"
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/BUILD

@@ -0,0 +1,32 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "3rdparty/jvm/com/twitter/storehaus:core",
+        "content-recommender/thrift/src/main/thrift:thrift-scala",
+        "escherbird/src/thrift/com/twitter/escherbird/topicannotation:topicannotation-thrift-scala",
+        "frigate/frigate-common:util",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store/health",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store/interests",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store/strato",
+        "hermit/hermit-core/src/main/scala/com/twitter/hermit/store/common",
+        "mediaservices/commons/src/main/thrift:thrift-scala",
+        "src/scala/com/twitter/simclusters_v2/common",
+        "src/scala/com/twitter/simclusters_v2/score",
+        "src/scala/com/twitter/topic_recos/common",
+        "src/scala/com/twitter/topic_recos/stores",
+        "src/thrift/com/twitter/frigate:frigate-common-thrift-scala",
+        "src/thrift/com/twitter/simclusters_v2:simclusters_v2-thrift-scala",
+        "src/thrift/com/twitter/spam/rtf:safety-level-scala",
+        "src/thrift/com/twitter/tweetypie:service-scala",
+        "src/thrift/com/twitter/tweetypie:tweet-scala",
+        "stitch/stitch-storehaus",
+        "stitch/stitch-tweetypie/src/main/scala",
+        "strato/src/main/scala/com/twitter/strato/client",
+        "topic-social-proof/server/src/main/scala/com/twitter/tsp/utils",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+        "topiclisting/topiclisting-core/src/main/scala/com/twitter/topiclisting",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/LocalizedUttRecommendableTopicsStore.scala

@@ -0,0 +1,30 @@
+package com.twitter.tsp.stores
+
+import com.twitter.storehaus.ReadableStore
+import com.twitter.topiclisting.FollowableTopicProductId
+import com.twitter.topiclisting.ProductId
+import com.twitter.topiclisting.SemanticCoreEntityId
+import com.twitter.topiclisting.TopicListingViewerContext
+import com.twitter.topiclisting.utt.UttLocalization
+import com.twitter.util.Future
+
+case class LocalizedUttTopicNameRequest(
+  productId: ProductId.Value,
+  viewerContext: TopicListingViewerContext,
+  enableInternationalTopics: Boolean)
+
+class LocalizedUttRecommendableTopicsStore(uttLocalization: UttLocalization)
+    extends ReadableStore[LocalizedUttTopicNameRequest, Set[SemanticCoreEntityId]] {
+
+  override def get(
+    request: LocalizedUttTopicNameRequest
+  ): Future[Option[Set[SemanticCoreEntityId]]] = {
+    uttLocalization
+      .getRecommendableTopics(
+        productId = request.productId,
+        viewerContext = request.viewerContext,
+        enableInternationalTopics = request.enableInternationalTopics,
+        followableTopicProductId = FollowableTopicProductId.AllFollowable
+      ).map { response => Some(response) }
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/RepresentationScorerStore.scala

@@ -0,0 +1,31 @@
+package com.twitter.tsp.stores
+
+import com.twitter.contentrecommender.thriftscala.ScoringResponse
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.store.strato.StratoFetchableStore
+import com.twitter.hermit.store.common.ObservedReadableStore
+import com.twitter.simclusters_v2.thriftscala.Score
+import com.twitter.simclusters_v2.thriftscala.ScoreId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.Client
+import com.twitter.strato.thrift.ScroogeConvImplicits._
+import com.twitter.tsp.utils.ReadableStoreWithMapOptionValues
+
+object RepresentationScorerStore {
+
+  def apply(
+    stratoClient: Client,
+    scoringColumnPath: String,
+    stats: StatsReceiver
+  ): ReadableStore[ScoreId, Score] = {
+    val stratoFetchableStore = StratoFetchableStore
+      .withUnitView[ScoreId, ScoringResponse](stratoClient, scoringColumnPath)
+
+    val enrichedStore = new ReadableStoreWithMapOptionValues[ScoreId, ScoringResponse, Score](
+      stratoFetchableStore).mapOptionValues(_.score)
+
+    ObservedReadableStore(
+      enrichedStore
+    )(stats.scope("representation_scorer_store"))
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/SemanticCoreAnnotationStore.scala

@@ -0,0 +1,64 @@
+package com.twitter.tsp.stores
+
+import com.twitter.escherbird.topicannotation.strato.thriftscala.TopicAnnotationValue
+import com.twitter.escherbird.topicannotation.strato.thriftscala.TopicAnnotationView
+import com.twitter.frigate.common.store.strato.StratoFetchableStore
+import com.twitter.simclusters_v2.common.TopicId
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.Client
+import com.twitter.strato.thrift.ScroogeConvImplicits._
+import com.twitter.util.Future
+
+/**
+ * This is copied from `src/scala/com/twitter/topic_recos/stores/SemanticCoreAnnotationStore.scala`
+ * Unfortunately their version assumes (incorrectly) that there is no View which causes warnings.
+ * While these warnings may not cause any problems in practice, better safe than sorry.
+ */
+object SemanticCoreAnnotationStore {
+  private val column = "semanticCore/topicannotation/topicAnnotation.Tweet"
+
+  def getStratoStore(stratoClient: Client): ReadableStore[TweetId, TopicAnnotationValue] = {
+    StratoFetchableStore
+      .withView[TweetId, TopicAnnotationView, TopicAnnotationValue](
+        stratoClient,
+        column,
+        TopicAnnotationView())
+  }
+
+  case class TopicAnnotation(
+    topicId: TopicId,
+    ignoreSimClustersFilter: Boolean,
+    modelVersionId: Long)
+}
+
+/**
+ * Given a tweet Id, return the list of annotations defined by the TSIG team.
+ */
+case class SemanticCoreAnnotationStore(stratoStore: ReadableStore[TweetId, TopicAnnotationValue])
+    extends ReadableStore[TweetId, Seq[SemanticCoreAnnotationStore.TopicAnnotation]] {
+  import SemanticCoreAnnotationStore._
+
+  override def multiGet[K1 <: TweetId](
+    ks: Set[K1]
+  ): Map[K1, Future[Option[Seq[TopicAnnotation]]]] = {
+    stratoStore
+      .multiGet(ks)
+      .mapValues(_.map(_.map { topicAnnotationValue =>
+        topicAnnotationValue.annotationsPerModel match {
+          case Some(annotationWithVersions) =>
+            annotationWithVersions.flatMap { annotations =>
+              annotations.annotations.map { annotation =>
+                TopicAnnotation(
+                  annotation.entityId,
+                  annotation.ignoreQualityFilter.getOrElse(false),
+                  annotations.modelVersionId
+                )
+              }
+            }
+          case _ =>
+            Nil
+        }
+      }))
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/TopicSocialProofStore.scala

@@ -0,0 +1,127 @@
+package com.twitter.tsp.stores
+
+import com.twitter.tsp.stores.TopicTweetsCosineSimilarityAggregateStore.ScoreKey
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.util.StatsUtil
+import com.twitter.simclusters_v2.thriftscala._
+import com.twitter.storehaus.ReadableStore
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.tsp.stores.SemanticCoreAnnotationStore._
+import com.twitter.tsp.stores.TopicSocialProofStore.TopicSocialProof
+import com.twitter.util.Future
+
+/**
+ * Provides a session-less Topic Social Proof information which doesn't rely on any User Info.
+ * This store is used by MemCache and In-Memory cache to achieve a higher performance.
+ * One Consumer embedding and Producer embedding are used to calculate raw score.
+ */
+case class TopicSocialProofStore(
+  representationScorerStore: ReadableStore[ScoreId, Score],
+  semanticCoreAnnotationStore: ReadableStore[TweetId, Seq[TopicAnnotation]]
+)(
+  statsReceiver: StatsReceiver)
+    extends ReadableStore[TopicSocialProofStore.Query, Seq[TopicSocialProof]] {
+  import TopicSocialProofStore._
+
+  // Fetches the tweet's topic annotations from SemanticCore's Annotation API
+  override def get(query: TopicSocialProofStore.Query): Future[Option[Seq[TopicSocialProof]]] = {
+    StatsUtil.trackOptionStats(statsReceiver) {
+      for {
+        annotations <-
+          StatsUtil.trackItemsStats(statsReceiver.scope("semanticCoreAnnotationStore")) {
+            semanticCoreAnnotationStore.get(query.cacheableQuery.tweetId).map(_.getOrElse(Nil))
+          }
+
+        filteredAnnotations = filterAnnotationsByAllowList(annotations, query)
+
+        scoredTopics <-
+          StatsUtil.trackItemMapStats(statsReceiver.scope("scoreTopicTweetsTweetLanguage")) {
+            // de-dup identical topicIds
+            val uniqueTopicIds = filteredAnnotations.map { annotation =>
+              TopicId(annotation.topicId, Some(query.cacheableQuery.tweetLanguage), country = None)
+            }.toSet
+
+            if (query.cacheableQuery.enableCosineSimilarityScoreCalculation) {
+              scoreTopicTweets(query.cacheableQuery.tweetId, uniqueTopicIds)
+            } else {
+              Future.value(uniqueTopicIds.map(id => id -> Map.empty[ScoreKey, Double]).toMap)
+            }
+          }
+
+      } yield {
+        if (scoredTopics.nonEmpty) {
+          val versionedTopicProofs = filteredAnnotations.map { annotation =>
+            val topicId =
+              TopicId(annotation.topicId, Some(query.cacheableQuery.tweetLanguage), country = None)
+
+            TopicSocialProof(
+              topicId,
+              scores = scoredTopics.getOrElse(topicId, Map.empty),
+              annotation.ignoreSimClustersFilter,
+              annotation.modelVersionId
+            )
+          }
+          Some(versionedTopicProofs)
+        } else {
+          None
+        }
+      }
+    }
+  }
+
+  /***
+   * When the allowList is not empty (e.g., TSP handler call, CrTopic handler call),
+   * the filter will be enabled and we will only keep annotations that have versionIds existing
+   * in the input allowedSemanticCoreVersionIds set.
+   * But when the allowList is empty (e.g., some debugger calls),
+   * we will not filter anything and pass.
+   * We limit the number of versionIds to be K = MaxNumberVersionIds
+   */
+  private def filterAnnotationsByAllowList(
+    annotations: Seq[TopicAnnotation],
+    query: TopicSocialProofStore.Query
+  ): Seq[TopicAnnotation] = {
+
+    val trimmedVersionIds = query.allowedSemanticCoreVersionIds.take(MaxNumberVersionIds)
+    annotations.filter { annotation =>
+      trimmedVersionIds.isEmpty || trimmedVersionIds.contains(annotation.modelVersionId)
+    }
+  }
+
+  private def scoreTopicTweets(
+    tweetId: TweetId,
+    topicIds: Set[TopicId]
+  ): Future[Map[TopicId, Map[ScoreKey, Double]]] = {
+    Future.collect {
+      topicIds.map { topicId =>
+        val scoresFut = TopicTweetsCosineSimilarityAggregateStore.getRawScoresMap(
+          topicId,
+          tweetId,
+          TopicTweetsCosineSimilarityAggregateStore.DefaultScoreKeys,
+          representationScorerStore
+        )
+        topicId -> scoresFut
+      }.toMap
+    }
+  }
+}
+
+object TopicSocialProofStore {
+
+  private val MaxNumberVersionIds = 9
+
+  case class Query(
+    cacheableQuery: CacheableQuery,
+    allowedSemanticCoreVersionIds: Set[Long] = Set.empty) // overridden by FS
+
+  case class CacheableQuery(
+    tweetId: TweetId,
+    tweetLanguage: String,
+    enableCosineSimilarityScoreCalculation: Boolean = true)
+
+  case class TopicSocialProof(
+    topicId: TopicId,
+    scores: Map[ScoreKey, Double],
+    ignoreSimClusterFiltering: Boolean,
+    semanticCoreVersionId: Long)
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/TopicStore.scala

@@ -0,0 +1,135 @@
+package com.twitter.tsp.stores
+
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.store.InterestedInInterestsFetchKey
+import com.twitter.frigate.common.store.strato.StratoFetchableStore
+import com.twitter.hermit.store.common.ObservedReadableStore
+import com.twitter.interests.thriftscala.InterestId
+import com.twitter.interests.thriftscala.InterestLabel
+import com.twitter.interests.thriftscala.InterestRelationship
+import com.twitter.interests.thriftscala.InterestRelationshipV1
+import com.twitter.interests.thriftscala.InterestedInInterestLookupContext
+import com.twitter.interests.thriftscala.InterestedInInterestModel
+import com.twitter.interests.thriftscala.OptOutInterestLookupContext
+import com.twitter.interests.thriftscala.UserInterest
+import com.twitter.interests.thriftscala.UserInterestData
+import com.twitter.interests.thriftscala.UserInterestsResponse
+import com.twitter.simclusters_v2.common.UserId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.strato.client.Client
+import com.twitter.strato.thrift.ScroogeConvImplicits._
+
+case class TopicResponse(
+  entityId: Long,
+  interestedInData: Seq[InterestedInInterestModel],
+  scoreOverride: Option[Double] = None,
+  notInterestedInTimestamp: Option[Long] = None,
+  topicFollowTimestamp: Option[Long] = None)
+
+case class TopicResponses(responses: Seq[TopicResponse])
+
+object TopicStore {
+
+  private val InterestedInInterestsColumn = "interests/interestedInInterests"
+  private lazy val ExplicitInterestsContext: InterestedInInterestLookupContext =
+    InterestedInInterestLookupContext(
+      explicitContext = None,
+      inferredContext = None,
+      disableImplicit = Some(true)
+    )
+
+  private def userInterestsResponseToTopicResponse(
+    userInterestsResponse: UserInterestsResponse
+  ): TopicResponses = {
+    val responses = userInterestsResponse.interests.interests.toSeq.flatMap { userInterests =>
+      userInterests.collect {
+        case UserInterest(
+              InterestId.SemanticCore(semanticCoreEntity),
+              Some(UserInterestData.InterestedIn(data))) =>
+          val topicFollowingTimestampOpt = data.collect {
+            case InterestedInInterestModel.ExplicitModel(
+                  InterestRelationship.V1(interestRelationshipV1)) =>
+              interestRelationshipV1.timestampMs
+          }.lastOption
+
+          TopicResponse(semanticCoreEntity.id, data, None, None, topicFollowingTimestampOpt)
+      }
+    }
+    TopicResponses(responses)
+  }
+
+  def explicitFollowingTopicStore(
+    stratoClient: Client
+  )(
+    implicit statsReceiver: StatsReceiver
+  ): ReadableStore[UserId, TopicResponses] = {
+    val stratoStore =
+      StratoFetchableStore
+        .withUnitView[InterestedInInterestsFetchKey, UserInterestsResponse](
+          stratoClient,
+          InterestedInInterestsColumn)
+        .composeKeyMapping[UserId](uid =>
+          InterestedInInterestsFetchKey(
+            userId = uid,
+            labels = None,
+            lookupContext = Some(ExplicitInterestsContext)
+          ))
+        .mapValues(userInterestsResponseToTopicResponse)
+
+    ObservedReadableStore(stratoStore)
+  }
+
+  def userOptOutTopicStore(
+    stratoClient: Client,
+    optOutStratoStorePath: String
+  )(
+    implicit statsReceiver: StatsReceiver
+  ): ReadableStore[UserId, TopicResponses] = {
+    val stratoStore =
+      StratoFetchableStore
+        .withUnitView[
+          (Long, Option[Seq[InterestLabel]], Option[OptOutInterestLookupContext]),
+          UserInterestsResponse](stratoClient, optOutStratoStorePath)
+        .composeKeyMapping[UserId](uid => (uid, None, None))
+        .mapValues { userInterestsResponse =>
+          val responses = userInterestsResponse.interests.interests.toSeq.flatMap { userInterests =>
+            userInterests.collect {
+              case UserInterest(
+                    InterestId.SemanticCore(semanticCoreEntity),
+                    Some(UserInterestData.InterestedIn(data))) =>
+                TopicResponse(semanticCoreEntity.id, data, None)
+            }
+          }
+          TopicResponses(responses)
+        }
+    ObservedReadableStore(stratoStore)
+  }
+
+  def notInterestedInTopicsStore(
+    stratoClient: Client,
+    notInterestedInStorePath: String
+  )(
+    implicit statsReceiver: StatsReceiver
+  ): ReadableStore[UserId, TopicResponses] = {
+    val stratoStore =
+      StratoFetchableStore
+        .withUnitView[Long, Seq[UserInterest]](stratoClient, notInterestedInStorePath)
+        .composeKeyMapping[UserId](identity)
+        .mapValues { notInterestedInInterests =>
+          val responses = notInterestedInInterests.collect {
+            case UserInterest(
+                  InterestId.SemanticCore(semanticCoreEntity),
+                  Some(UserInterestData.NotInterested(notInterestedInData))) =>
+              val notInterestedInTimestampOpt = notInterestedInData.collect {
+                case InterestRelationship.V1(interestRelationshipV1: InterestRelationshipV1) =>
+                  interestRelationshipV1.timestampMs
+              }.lastOption
+
+              TopicResponse(semanticCoreEntity.id, Seq.empty, None, notInterestedInTimestampOpt)
+          }
+          TopicResponses(responses)
+        }
+    ObservedReadableStore(stratoStore)
+  }
+
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/TopicTweetsCosineSimilarityAggregateStore.scala

@@ -0,0 +1,99 @@
+package com.twitter.tsp.stores
+
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.thriftscala.EmbeddingType
+import com.twitter.simclusters_v2.thriftscala.InternalId
+import com.twitter.simclusters_v2.thriftscala.ModelVersion
+import com.twitter.simclusters_v2.thriftscala.ScoreInternalId
+import com.twitter.simclusters_v2.thriftscala.ScoringAlgorithm
+import com.twitter.simclusters_v2.thriftscala.SimClustersEmbeddingId
+import com.twitter.simclusters_v2.thriftscala.{
+  SimClustersEmbeddingPairScoreId => ThriftSimClustersEmbeddingPairScoreId
+}
+import com.twitter.simclusters_v2.thriftscala.TopicId
+import com.twitter.simclusters_v2.thriftscala.{Score => ThriftScore}
+import com.twitter.simclusters_v2.thriftscala.{ScoreId => ThriftScoreId}
+import com.twitter.storehaus.ReadableStore
+import com.twitter.topic_recos.common._
+import com.twitter.topic_recos.common.Configs.DefaultModelVersion
+import com.twitter.tsp.stores.TopicTweetsCosineSimilarityAggregateStore.ScoreKey
+import com.twitter.util.Future
+
+object TopicTweetsCosineSimilarityAggregateStore {
+
+  val TopicEmbeddingTypes: Seq[EmbeddingType] =
+    Seq(
+      EmbeddingType.FavTfgTopic,
+      EmbeddingType.LogFavBasedKgoApeTopic
+    )
+
+  // Add the new embedding types if want to test the new Tweet embedding performance.
+  val TweetEmbeddingTypes: Seq[EmbeddingType] = Seq(EmbeddingType.LogFavBasedTweet)
+
+  val ModelVersions: Seq[ModelVersion] =
+    Seq(DefaultModelVersion)
+
+  val DefaultScoreKeys: Seq[ScoreKey] = {
+    for {
+      modelVersion <- ModelVersions
+      topicEmbeddingType <- TopicEmbeddingTypes
+      tweetEmbeddingType <- TweetEmbeddingTypes
+    } yield {
+      ScoreKey(
+        topicEmbeddingType = topicEmbeddingType,
+        tweetEmbeddingType = tweetEmbeddingType,
+        modelVersion = modelVersion
+      )
+    }
+  }
+
+  case class ScoreKey(
+    topicEmbeddingType: EmbeddingType,
+    tweetEmbeddingType: EmbeddingType,
+    modelVersion: ModelVersion)
+
+  def getRawScoresMap(
+    topicId: TopicId,
+    tweetId: TweetId,
+    scoreKeys: Seq[ScoreKey],
+    representationScorerStore: ReadableStore[ThriftScoreId, ThriftScore]
+  ): Future[Map[ScoreKey, Double]] = {
+    val scoresMapFut = scoreKeys.map { key =>
+      val scoreInternalId = ScoreInternalId.SimClustersEmbeddingPairScoreId(
+        ThriftSimClustersEmbeddingPairScoreId(
+          buildTopicEmbedding(topicId, key.topicEmbeddingType, key.modelVersion),
+          SimClustersEmbeddingId(
+            key.tweetEmbeddingType,
+            key.modelVersion,
+            InternalId.TweetId(tweetId))
+        ))
+      val scoreFut = representationScorerStore
+        .get(
+          ThriftScoreId(
+            algorithm = ScoringAlgorithm.PairEmbeddingCosineSimilarity, // Hard code as cosine sim
+            internalId = scoreInternalId
+          ))
+      key -> scoreFut
+    }.toMap
+
+    Future
+      .collect(scoresMapFut).map(_.collect {
+        case (key, Some(ThriftScore(score))) =>
+          (key, score)
+      })
+  }
+}
+
+case class TopicTweetsCosineSimilarityAggregateStore(
+  representationScorerStore: ReadableStore[ThriftScoreId, ThriftScore]
+)(
+  statsReceiver: StatsReceiver)
+    extends ReadableStore[(TopicId, TweetId, Seq[ScoreKey]), Map[ScoreKey, Double]] {
+  import TopicTweetsCosineSimilarityAggregateStore._
+
+  override def get(k: (TopicId, TweetId, Seq[ScoreKey])): Future[Option[Map[ScoreKey, Double]]] = {
+    statsReceiver.counter("topicTweetsCosineSimilariltyAggregateStore").incr()
+    getRawScoresMap(k._1, k._2, k._3, representationScorerStore).map(Some(_))
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/TweetInfoStore.scala

@@ -0,0 +1,230 @@
+package com.twitter.tsp.stores
+
+import com.twitter.conversions.DurationOps._
+import com.twitter.tsp.thriftscala.TspTweetInfo
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.thriftscala.TweetHealthScores
+import com.twitter.frigate.thriftscala.UserAgathaScores
+import com.twitter.logging.Logger
+import com.twitter.mediaservices.commons.thriftscala.MediaCategory
+import com.twitter.mediaservices.commons.tweetmedia.thriftscala.MediaInfo
+import com.twitter.mediaservices.commons.tweetmedia.thriftscala.MediaSizeType
+import com.twitter.simclusters_v2.common.TweetId
+import com.twitter.simclusters_v2.common.UserId
+import com.twitter.spam.rtf.thriftscala.SafetyLevel
+import com.twitter.stitch.Stitch
+import com.twitter.stitch.storehaus.ReadableStoreOfStitch
+import com.twitter.stitch.tweetypie.TweetyPie
+import com.twitter.stitch.tweetypie.TweetyPie.TweetyPieException
+import com.twitter.storehaus.ReadableStore
+import com.twitter.topiclisting.AnnotationRuleProvider
+import com.twitter.tsp.utils.HealthSignalsUtils
+import com.twitter.tweetypie.thriftscala.TweetInclude
+import com.twitter.tweetypie.thriftscala.{Tweet => TTweet}
+import com.twitter.tweetypie.thriftscala._
+import com.twitter.util.Duration
+import com.twitter.util.Future
+import com.twitter.util.TimeoutException
+import com.twitter.util.Timer
+
+object TweetyPieFieldsStore {
+
+  // Tweet fields options. Only fields specified here will be hydrated in the tweet
+  private val CoreTweetFields: Set[TweetInclude] = Set[TweetInclude](
+    TweetInclude.TweetFieldId(TTweet.IdField.id),
+    TweetInclude.TweetFieldId(TTweet.CoreDataField.id), // needed for the authorId
+    TweetInclude.TweetFieldId(TTweet.LanguageField.id),
+    TweetInclude.CountsFieldId(StatusCounts.FavoriteCountField.id),
+    TweetInclude.CountsFieldId(StatusCounts.RetweetCountField.id),
+    TweetInclude.TweetFieldId(TTweet.QuotedTweetField.id),
+    TweetInclude.TweetFieldId(TTweet.MediaKeysField.id),
+    TweetInclude.TweetFieldId(TTweet.EscherbirdEntityAnnotationsField.id),
+    TweetInclude.TweetFieldId(TTweet.MediaField.id),
+    TweetInclude.TweetFieldId(TTweet.UrlsField.id)
+  )
+
+  private val gtfo: GetTweetFieldsOptions = GetTweetFieldsOptions(
+    tweetIncludes = CoreTweetFields,
+    safetyLevel = Some(SafetyLevel.Recommendations)
+  )
+
+  def getStoreFromTweetyPie(
+    tweetyPie: TweetyPie,
+    convertExceptionsToNotFound: Boolean = true
+  ): ReadableStore[Long, GetTweetFieldsResult] = {
+    val log = Logger("TweetyPieFieldsStore")
+
+    ReadableStoreOfStitch { tweetId: Long =>
+      tweetyPie
+        .getTweetFields(tweetId, options = gtfo)
+        .rescue {
+          case ex: TweetyPieException if convertExceptionsToNotFound =>
+            log.error(ex, s"Error while hitting tweetypie ${ex.result}")
+            Stitch.NotFound
+        }
+    }
+  }
+}
+
+object TweetInfoStore {
+
+  case class IsPassTweetHealthFilters(tweetStrictest: Option[Boolean])
+
+  case class IsPassAgathaHealthFilters(agathaStrictest: Option[Boolean])
+
+  private val HealthStoreTimeout: Duration = 40.milliseconds
+  private val isPassTweetHealthFilters: IsPassTweetHealthFilters = IsPassTweetHealthFilters(None)
+  private val isPassAgathaHealthFilters: IsPassAgathaHealthFilters = IsPassAgathaHealthFilters(None)
+}
+
+case class TweetInfoStore(
+  tweetFieldsStore: ReadableStore[TweetId, GetTweetFieldsResult],
+  tweetHealthModelStore: ReadableStore[TweetId, TweetHealthScores],
+  userHealthModelStore: ReadableStore[UserId, UserAgathaScores],
+  timer: Timer
+)(
+  statsReceiver: StatsReceiver)
+    extends ReadableStore[TweetId, TspTweetInfo] {
+
+  import TweetInfoStore._
+
+  private[this] def toTweetInfo(
+    tweetFieldsResult: GetTweetFieldsResult
+  ): Future[Option[TspTweetInfo]] = {
+    tweetFieldsResult.tweetResult match {
+      case result: TweetFieldsResultState.Found if result.found.suppressReason.isEmpty =>
+        val tweet = result.found.tweet
+
+        val authorIdOpt = tweet.coreData.map(_.userId)
+        val favCountOpt = tweet.counts.flatMap(_.favoriteCount)
+
+        val languageOpt = tweet.language.map(_.language)
+        val hasImageOpt =
+          tweet.mediaKeys.map(_.map(_.mediaCategory).exists(_ == MediaCategory.TweetImage))
+        val hasGifOpt =
+          tweet.mediaKeys.map(_.map(_.mediaCategory).exists(_ == MediaCategory.TweetGif))
+        val isNsfwAuthorOpt = Some(
+          tweet.coreData.exists(_.nsfwUser) || tweet.coreData.exists(_.nsfwAdmin))
+        val isTweetReplyOpt = tweet.coreData.map(_.reply.isDefined)
+        val hasMultipleMediaOpt =
+          tweet.mediaKeys.map(_.map(_.mediaCategory).size > 1)
+
+        val isKGODenylist = Some(
+          tweet.escherbirdEntityAnnotations
+            .exists(_.entityAnnotations.exists(AnnotationRuleProvider.isSuppressedTopicsDenylist)))
+
+        val isNullcastOpt = tweet.coreData.map(_.nullcast) // These are Ads. go/nullcast
+
+        val videoDurationOpt = tweet.media.flatMap(_.flatMap {
+          _.mediaInfo match {
+            case Some(MediaInfo.VideoInfo(info)) =>
+              Some((info.durationMillis + 999) / 1000) // video playtime always round up
+            case _ => None
+          }
+        }.headOption)
+
+        // There many different types of videos. To be robust to new types being added, we just use
+        // the videoDurationOpt to keep track of whether the item has a video or not.
+        val hasVideo = videoDurationOpt.isDefined
+
+        val mediaDimensionsOpt =
+          tweet.media.flatMap(_.headOption.flatMap(
+            _.sizes.find(_.sizeType == MediaSizeType.Orig).map(size => (size.width, size.height))))
+
+        val mediaWidth = mediaDimensionsOpt.map(_._1).getOrElse(1)
+        val mediaHeight = mediaDimensionsOpt.map(_._2).getOrElse(1)
+        // high resolution media's width is always greater than 480px and height is always greater than 480px
+        val isHighMediaResolution = mediaHeight > 480 && mediaWidth > 480
+        val isVerticalAspectRatio = mediaHeight >= mediaWidth && mediaWidth > 1
+        val hasUrlOpt = tweet.urls.map(_.nonEmpty)
+
+        (authorIdOpt, favCountOpt) match {
+          case (Some(authorId), Some(favCount)) =>
+            hydrateHealthScores(tweet.id, authorId).map {
+              case (isPassAgathaHealthFilters, isPassTweetHealthFilters) =>
+                Some(
+                  TspTweetInfo(
+                    authorId = authorId,
+                    favCount = favCount,
+                    language = languageOpt,
+                    hasImage = hasImageOpt,
+                    hasVideo = Some(hasVideo),
+                    hasGif = hasGifOpt,
+                    isNsfwAuthor = isNsfwAuthorOpt,
+                    isKGODenylist = isKGODenylist,
+                    isNullcast = isNullcastOpt,
+                    videoDurationSeconds = videoDurationOpt,
+                    isHighMediaResolution = Some(isHighMediaResolution),
+                    isVerticalAspectRatio = Some(isVerticalAspectRatio),
+                    isPassAgathaHealthFilterStrictest = isPassAgathaHealthFilters.agathaStrictest,
+                    isPassTweetHealthFilterStrictest = isPassTweetHealthFilters.tweetStrictest,
+                    isReply = isTweetReplyOpt,
+                    hasMultipleMedia = hasMultipleMediaOpt,
+                    hasUrl = hasUrlOpt
+                  ))
+            }
+          case _ =>
+            statsReceiver.counter("missingFields").incr()
+            Future.None // These values should always exist.
+        }
+      case _: TweetFieldsResultState.NotFound =>
+        statsReceiver.counter("notFound").incr()
+        Future.None
+      case _: TweetFieldsResultState.Failed =>
+        statsReceiver.counter("failed").incr()
+        Future.None
+      case _: TweetFieldsResultState.Filtered =>
+        statsReceiver.counter("filtered").incr()
+        Future.None
+      case _ =>
+        statsReceiver.counter("unknown").incr()
+        Future.None
+    }
+  }
+
+  private[this] def hydrateHealthScores(
+    tweetId: TweetId,
+    authorId: Long
+  ): Future[(IsPassAgathaHealthFilters, IsPassTweetHealthFilters)] = {
+    Future
+      .join(
+        tweetHealthModelStore
+          .multiGet(Set(tweetId))(tweetId),
+        userHealthModelStore
+          .multiGet(Set(authorId))(authorId)
+      ).map {
+        case (tweetHealthScoresOpt, userAgathaScoresOpt) =>
+          // This stats help us understand empty rate for AgathaCalibratedNsfw / NsfwTextUserScore
+          statsReceiver.counter("totalCountAgathaScore").incr()
+          if (userAgathaScoresOpt.getOrElse(UserAgathaScores()).agathaCalibratedNsfw.isEmpty)
+            statsReceiver.counter("emptyCountAgathaCalibratedNsfw").incr()
+          if (userAgathaScoresOpt.getOrElse(UserAgathaScores()).nsfwTextUserScore.isEmpty)
+            statsReceiver.counter("emptyCountNsfwTextUserScore").incr()
+
+          val isPassAgathaHealthFilters = IsPassAgathaHealthFilters(
+            agathaStrictest =
+              Some(HealthSignalsUtils.isTweetAgathaModelQualified(userAgathaScoresOpt)),
+          )
+
+          val isPassTweetHealthFilters = IsPassTweetHealthFilters(
+            tweetStrictest =
+              Some(HealthSignalsUtils.isTweetHealthModelQualified(tweetHealthScoresOpt))
+          )
+
+          (isPassAgathaHealthFilters, isPassTweetHealthFilters)
+      }.raiseWithin(HealthStoreTimeout)(timer).rescue {
+        case _: TimeoutException =>
+          statsReceiver.counter("hydrateHealthScoreTimeout").incr()
+          Future.value((isPassAgathaHealthFilters, isPassTweetHealthFilters))
+        case _ =>
+          statsReceiver.counter("hydrateHealthScoreFailure").incr()
+          Future.value((isPassAgathaHealthFilters, isPassTweetHealthFilters))
+      }
+  }
+
+  override def multiGet[K1 <: TweetId](ks: Set[K1]): Map[K1, Future[Option[TspTweetInfo]]] = {
+    statsReceiver.counter("tweetFieldsStore").incr(ks.size)
+    tweetFieldsStore
+      .multiGet(ks).mapValues(_.flatMap { _.map { v => toTweetInfo(v) }.getOrElse(Future.None) })
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/stores/UttTopicFilterStore.scala

@@ -0,0 +1,248 @@
+package com.twitter.tsp.stores
+
+import com.twitter.conversions.DurationOps._
+import com.twitter.finagle.FailureFlags.flagsOf
+import com.twitter.finagle.mux.ClientDiscardedRequestException
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.frigate.common.store.interests
+import com.twitter.simclusters_v2.common.UserId
+import com.twitter.storehaus.ReadableStore
+import com.twitter.topiclisting.ProductId
+import com.twitter.topiclisting.TopicListing
+import com.twitter.topiclisting.TopicListingViewerContext
+import com.twitter.topiclisting.{SemanticCoreEntityId => ScEntityId}
+import com.twitter.tsp.thriftscala.TopicFollowType
+import com.twitter.tsp.thriftscala.TopicListingSetting
+import com.twitter.tsp.thriftscala.TopicSocialProofFilteringBypassMode
+import com.twitter.util.Duration
+import com.twitter.util.Future
+import com.twitter.util.TimeoutException
+import com.twitter.util.Timer
+
+class UttTopicFilterStore(
+  topicListing: TopicListing,
+  userOptOutTopicsStore: ReadableStore[interests.UserId, TopicResponses],
+  explicitFollowingTopicsStore: ReadableStore[interests.UserId, TopicResponses],
+  notInterestedTopicsStore: ReadableStore[interests.UserId, TopicResponses],
+  localizedUttRecommendableTopicsStore: ReadableStore[LocalizedUttTopicNameRequest, Set[Long]],
+  timer: Timer,
+  stats: StatsReceiver) {
+  import UttTopicFilterStore._
+
+  // Set of blacklisted SemanticCore IDs that are paused.
+  private[this] def getPausedTopics(topicCtx: TopicListingViewerContext): Set[ScEntityId] = {
+    topicListing.getPausedTopics(topicCtx)
+  }
+
+  private[this] def getOptOutTopics(userId: Long): Future[Set[ScEntityId]] = {
+    stats.counter("getOptOutTopicsCount").incr()
+    userOptOutTopicsStore
+      .get(userId).map { responseOpt =>
+        responseOpt
+          .map { responses => responses.responses.map(_.entityId) }.getOrElse(Seq.empty).toSet
+      }.raiseWithin(DefaultOptOutTimeout)(timer).rescue {
+        case err: TimeoutException =>
+          stats.counter("getOptOutTopicsTimeout").incr()
+          Future.exception(err)
+        case err: ClientDiscardedRequestException
+            if flagsOf(err).contains("interrupted") && flagsOf(err)
+              .contains("ignorable") =>
+          stats.counter("getOptOutTopicsDiscardedBackupRequest").incr()
+          Future.exception(err)
+        case err =>
+          stats.counter("getOptOutTopicsFailure").incr()
+          Future.exception(err)
+      }
+  }
+
+  private[this] def getNotInterestedIn(userId: Long): Future[Set[ScEntityId]] = {
+    stats.counter("getNotInterestedInCount").incr()
+    notInterestedTopicsStore
+      .get(userId).map { responseOpt =>
+        responseOpt
+          .map { responses => responses.responses.map(_.entityId) }.getOrElse(Seq.empty).toSet
+      }.raiseWithin(DefaultNotInterestedInTimeout)(timer).rescue {
+        case err: TimeoutException =>
+          stats.counter("getNotInterestedInTimeout").incr()
+          Future.exception(err)
+        case err: ClientDiscardedRequestException
+            if flagsOf(err).contains("interrupted") && flagsOf(err)
+              .contains("ignorable") =>
+          stats.counter("getNotInterestedInDiscardedBackupRequest").incr()
+          Future.exception(err)
+        case err =>
+          stats.counter("getNotInterestedInFailure").incr()
+          Future.exception(err)
+      }
+  }
+
+  private[this] def getFollowedTopics(userId: Long): Future[Set[TopicResponse]] = {
+    stats.counter("getFollowedTopicsCount").incr()
+
+    explicitFollowingTopicsStore
+      .get(userId).map { responseOpt =>
+        responseOpt.map(_.responses.toSet).getOrElse(Set.empty)
+      }.raiseWithin(DefaultInterestedInTimeout)(timer).rescue {
+        case _: TimeoutException =>
+          stats.counter("getFollowedTopicsTimeout").incr()
+          Future(Set.empty)
+        case _ =>
+          stats.counter("getFollowedTopicsFailure").incr()
+          Future(Set.empty)
+      }
+  }
+
+  private[this] def getFollowedTopicIds(userId: Long): Future[Set[ScEntityId]] = {
+    getFollowedTopics(userId: Long).map(_.map(_.entityId))
+  }
+
+  private[this] def getWhitelistTopicIds(
+    normalizedContext: TopicListingViewerContext,
+    enableInternationalTopics: Boolean
+  ): Future[Set[ScEntityId]] = {
+    stats.counter("getWhitelistTopicIdsCount").incr()
+
+    val uttRequest = LocalizedUttTopicNameRequest(
+      productId = ProductId.Followable,
+      viewerContext = normalizedContext,
+      enableInternationalTopics = enableInternationalTopics
+    )
+    localizedUttRecommendableTopicsStore
+      .get(uttRequest).map { response =>
+        response.getOrElse(Set.empty)
+      }.rescue {
+        case _ =>
+          stats.counter("getWhitelistTopicIdsFailure").incr()
+          Future(Set.empty)
+      }
+  }
+
+  private[this] def getDenyListTopicIdsForUser(
+    userId: UserId,
+    topicListingSetting: TopicListingSetting,
+    context: TopicListingViewerContext,
+    bypassModes: Option[Set[TopicSocialProofFilteringBypassMode]]
+  ): Future[Set[ScEntityId]] = {
+
+    val denyListTopicIdsFuture = topicListingSetting match {
+      case TopicListingSetting.ImplicitFollow =>
+        getFollowedTopicIds(userId)
+      case _ =>
+        Future(Set.empty[ScEntityId])
+    }
+
+    // we don't filter opt-out topics for implicit follow topic listing setting
+    val optOutTopicIdsFuture = topicListingSetting match {
+      case TopicListingSetting.ImplicitFollow => Future(Set.empty[ScEntityId])
+      case _ => getOptOutTopics(userId)
+    }
+
+    val notInterestedTopicIdsFuture =
+      if (bypassModes.exists(_.contains(TopicSocialProofFilteringBypassMode.NotInterested))) {
+        Future(Set.empty[ScEntityId])
+      } else {
+        getNotInterestedIn(userId)
+      }
+    val pausedTopicIdsFuture = Future.value(getPausedTopics(context))
+
+    Future
+      .collect(
+        List(
+          denyListTopicIdsFuture,
+          optOutTopicIdsFuture,
+          notInterestedTopicIdsFuture,
+          pausedTopicIdsFuture)).map { list => list.reduce(_ ++ _) }
+  }
+
+  private[this] def getDiff(
+    aFut: Future[Set[ScEntityId]],
+    bFut: Future[Set[ScEntityId]]
+  ): Future[Set[ScEntityId]] = {
+    Future.join(aFut, bFut).map {
+      case (a, b) => a.diff(b)
+    }
+  }
+
+  /**
+   * calculates the diff of all the whitelisted IDs with blacklisted IDs and returns the set of IDs
+   * that we will be recommending from or followed topics by the user by client setting.
+   */
+  def getAllowListTopicsForUser(
+    userId: UserId,
+    topicListingSetting: TopicListingSetting,
+    context: TopicListingViewerContext,
+    bypassModes: Option[Set[TopicSocialProofFilteringBypassMode]]
+  ): Future[Map[ScEntityId, Option[TopicFollowType]]] = {
+
+    /**
+     * Title: an illustrative table to explain how allow list is composed
+     * AllowList = WhiteList - DenyList - OptOutTopics - PausedTopics - NotInterestedInTopics
+     *
+     * TopicListingSetting: Following                 ImplicitFollow                       All                       Followable
+     * Whitelist:          FollowedTopics(user)      AllWhitelistedTopics                 Nil                       AllWhitelistedTopics
+     * DenyList:           Nil                       FollowedTopics(user)                 Nil                       Nil
+     *
+     * ps. for TopicListingSetting.All, the returned allow list is Nil. Why?
+     * It's because that allowList is not required given the TopicListingSetting == 'All'.
+     * See TopicSocialProofHandler.filterByAllowedList() for more details.
+     */
+
+    topicListingSetting match {
+      // "All" means all the UTT entity is qualified. So don't need to fetch the Whitelist anymore.
+      case TopicListingSetting.All => Future.value(Map.empty)
+      case TopicListingSetting.Following =>
+        getFollowingTopicsForUserWithTimestamp(userId, context, bypassModes).map {
+          _.mapValues(_ => Some(TopicFollowType.Following))
+        }
+      case TopicListingSetting.ImplicitFollow =>
+        getDiff(
+          getWhitelistTopicIds(context, enableInternationalTopics = true),
+          getDenyListTopicIdsForUser(userId, topicListingSetting, context, bypassModes)).map {
+          _.map { scEntityId =>
+            scEntityId -> Some(TopicFollowType.ImplicitFollow)
+          }.toMap
+        }
+      case _ =>
+        val followedTopicIdsFut = getFollowedTopicIds(userId)
+        val allowListTopicIdsFut = getDiff(
+          getWhitelistTopicIds(context, enableInternationalTopics = true),
+          getDenyListTopicIdsForUser(userId, topicListingSetting, context, bypassModes))
+        Future.join(allowListTopicIdsFut, followedTopicIdsFut).map {
+          case (allowListTopicId, followedTopicIds) =>
+            allowListTopicId.map { scEntityId =>
+              if (followedTopicIds.contains(scEntityId))
+                scEntityId -> Some(TopicFollowType.Following)
+              else scEntityId -> Some(TopicFollowType.ImplicitFollow)
+            }.toMap
+        }
+    }
+  }
+
+  private[this] def getFollowingTopicsForUserWithTimestamp(
+    userId: UserId,
+    context: TopicListingViewerContext,
+    bypassModes: Option[Set[TopicSocialProofFilteringBypassMode]]
+  ): Future[Map[ScEntityId, Option[Long]]] = {
+
+    val followedTopicIdToTimestampFut = getFollowedTopics(userId).map(_.map { followedTopic =>
+      followedTopic.entityId -> followedTopic.topicFollowTimestamp
+    }.toMap)
+
+    followedTopicIdToTimestampFut.flatMap { followedTopicIdToTimestamp =>
+      getDiff(
+        Future(followedTopicIdToTimestamp.keySet),
+        getDenyListTopicIdsForUser(userId, TopicListingSetting.Following, context, bypassModes)
+      ).map {
+        _.map { scEntityId =>
+          scEntityId -> followedTopicIdToTimestamp.get(scEntityId).flatten
+        }.toMap
+      }
+    }
+  }
+}
+
+object UttTopicFilterStore {
+  val DefaultNotInterestedInTimeout: Duration = 60.milliseconds
+  val DefaultOptOutTimeout: Duration = 60.milliseconds
+  val DefaultInterestedInTimeout: Duration = 60.milliseconds
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/utils/BUILD

@@ -0,0 +1,14 @@
+scala_library(
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+    ],
+    dependencies = [
+        "3rdparty/jvm/org/lz4:lz4-java",
+        "content-recommender/thrift/src/main/thrift:thrift-scala",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store",
+        "frigate/frigate-common/src/main/scala/com/twitter/frigate/common/store/health",
+        "stitch/stitch-storehaus",
+        "topic-social-proof/server/src/main/thrift:thrift-scala",
+    ],
+)

topic-social-proof/server/src/main/scala/com/twitter/tsp/utils/LZ4Injection.scala

@@ -0,0 +1,19 @@
+package com.twitter.tsp.utils
+
+import com.twitter.bijection.Injection
+import scala.util.Try
+import net.jpountz.lz4.LZ4CompressorWithLength
+import net.jpountz.lz4.LZ4DecompressorWithLength
+import net.jpountz.lz4.LZ4Factory
+
+object LZ4Injection extends Injection[Array[Byte], Array[Byte]] {
+  private val lz4Factory = LZ4Factory.fastestInstance()
+  private val fastCompressor = new LZ4CompressorWithLength(lz4Factory.fastCompressor())
+  private val decompressor = new LZ4DecompressorWithLength(lz4Factory.fastDecompressor())
+
+  override def apply(a: Array[Byte]): Array[Byte] = LZ4Injection.fastCompressor.compress(a)
+
+  override def invert(b: Array[Byte]): Try[Array[Byte]] = Try {
+    LZ4Injection.decompressor.decompress(b)
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/utils/ReadableStoreWithMapOptionValues.scala

@@ -0,0 +1,20 @@
+package com.twitter.tsp.utils
+
+import com.twitter.storehaus.AbstractReadableStore
+import com.twitter.storehaus.ReadableStore
+import com.twitter.util.Future
+
+class ReadableStoreWithMapOptionValues[K, V1, V2](rs: ReadableStore[K, V1]) {
+
+  def mapOptionValues(
+    fn: V1 => Option[V2]
+  ): ReadableStore[K, V2] = {
+    val self = rs
+    new AbstractReadableStore[K, V2] {
+      override def get(k: K): Future[Option[V2]] = self.get(k).map(_.flatMap(fn))
+
+      override def multiGet[K1 <: K](ks: Set[K1]): Map[K1, Future[Option[V2]]] =
+        self.multiGet(ks).mapValues(_.map(_.flatMap(fn)))
+    }
+  }
+}

topic-social-proof/server/src/main/scala/com/twitter/tsp/utils/SeqObjectInjection.scala

@@ -0,0 +1,32 @@
+package com.twitter.tsp.utils
+
+import com.twitter.bijection.Injection
+import java.io.ByteArrayInputStream
+import java.io.ByteArrayOutputStream
+import java.io.ObjectInputStream
+import java.io.ObjectOutputStream
+import java.io.Serializable
+import scala.util.Try
+
+/**
+ * @tparam T must be a serializable class
+ */
+case class SeqObjectInjection[T <: Serializable]() extends Injection[Seq[T], Array[Byte]] {
+
+  override def apply(seq: Seq[T]): Array[Byte] = {
+    val byteStream = new ByteArrayOutputStream()
+    val outputStream = new ObjectOutputStream(byteStream)
+    outputStream.writeObject(seq)
+    outputStream.close()
+    byteStream.toByteArray
+  }
+
+  override def invert(bytes: Array[Byte]): Try[Seq[T]] = {
+    Try {
+      val inputStream = new ObjectInputStream(new ByteArrayInputStream(bytes))
+      val seq = inputStream.readObject().asInstanceOf[Seq[T]]
+      inputStream.close()
+      seq
+    }
+  }
+}

topic-social-proof/server/src/main/thrift/BUILD

@@ -0,0 +1,21 @@
+create_thrift_libraries(
+    base_name = "thrift",
+    sources = ["*.thrift"],
+    platform = "java8",
+    tags = [
+        "bazel-compatible",
+    ],
+    dependency_roots = [
+        "content-recommender/thrift/src/main/thrift",
+        "content-recommender/thrift/src/main/thrift:content-recommender-common",
+        "interests-service/thrift/src/main/thrift",
+        "src/thrift/com/twitter/simclusters_v2:simclusters_v2-thrift",
+    ],
+    generate_languages = [
+        "java",
+        "scala",
+        "strato",
+    ],
+    provides_java_name = "tsp-thrift-java",
+    provides_scala_name = "tsp-thrift-scala",
+)

topic-social-proof/server/src/main/thrift/service.thrift

@@ -0,0 +1,104 @@
+namespace java com.twitter.tsp.thriftjava
+namespace py gen.twitter.tsp
+#@namespace scala com.twitter.tsp.thriftscala
+#@namespace strato com.twitter.tsp.strato
+
+include "com/twitter/contentrecommender/common.thrift"
+include "com/twitter/simclusters_v2/identifier.thrift"
+include "com/twitter/simclusters_v2/online_store.thrift"
+include "topic_listing.thrift"
+
+enum TopicListingSetting {
+  All = 0 // All the existing Semantic Core Entity/Topics. ie., All topics on twitter, and may or may not have been launched yet.
+  Followable = 1 // All the topics which the user is allowed to follow. ie., topics that have shipped, and user may or may not be following it.
+  Following = 2 // Only topics the user is explicitly following
+  ImplicitFollow = 3 // The topics user has not followed but implicitly may follow. ie., Only topics that user has not followed.
+} (hasPersonalData='false')
+
+
+// used to tell Topic Social Proof endpoint which specific filtering can be bypassed
+enum TopicSocialProofFilteringBypassMode {
+  NotInterested = 0
+} (hasPersonalData='false')
+
+struct TopicSocialProofRequest {
+  1: required i64 userId(personalDataType = "UserId")
+  2: required set<i64> tweetIds(personalDataType = 'TweetId')
+  3: required common.DisplayLocation displayLocation
+  4: required TopicListingSetting topicListingSetting
+  5: required topic_listing.TopicListingViewerContext context
+  6: optional set<TopicSocialProofFilteringBypassMode> bypassModes
+  7: optional map<i64, set<MetricTag>> tags
+}
+
+struct TopicSocialProofOptions {
+  1: required i64 userId(personalDataType = "UserId")
+  2: required common.DisplayLocation displayLocation
+  3: required TopicListingSetting topicListingSetting
+  4: required topic_listing.TopicListingViewerContext context
+  5: optional set<TopicSocialProofFilteringBypassMode> bypassModes
+  6: optional map<i64, set<MetricTag>> tags
+}
+
+struct TopicSocialProofResponse {
+  1: required map<i64, list<TopicWithScore>> socialProofs
+}(hasPersonalData='false')
+
+// Distinguishes between how a topic tweet is generated. Useful for metric tracking and debugging
+enum TopicTweetType {
+  // CrOON candidates
+  UserInterestedIn        = 1
+  Twistly                 = 2
+  // crTopic candidates
+  SkitConsumerEmbeddings  = 100
+  SkitProducerEmbeddings  = 101
+  SkitHighPrecision       = 102
+  SkitInterestBrowser     = 103
+  Certo                   = 104
+}(persisted='true')
+
+struct TopicWithScore {
+  1: required i64 topicId
+  2: required double score // score used to rank topics relative to one another
+  3: optional TopicTweetType algorithmType // how the topic is generated
+  4: optional TopicFollowType topicFollowType // Whether the topic is being explicitly or implicily followed
+}(persisted='true', hasPersonalData='false')
+
+
+struct ScoreKey {
+  1: required identifier.EmbeddingType userEmbeddingType
+  2: required identifier.EmbeddingType topicEmbeddingType
+  3: required online_store.ModelVersion modelVersion
+}(persisted='true', hasPersonalData='false')
+
+struct UserTopicScore {
+  1: required map<ScoreKey, double> scores
+}(persisted='true', hasPersonalData='false')
+
+
+enum TopicFollowType {
+  Following = 1
+  ImplicitFollow = 2
+}(persisted='true')
+
+// Provide the Tags which provides the Recommended Tweets Source Signal and other context.
+// Warning: Please don't use this tag in any ML Features or business logic.
+enum MetricTag {
+  // Source Signal Tags
+  TweetFavorite         = 0
+  Retweet               = 1
+
+  UserFollow            = 101
+  PushOpenOrNtabClick   = 201
+
+  HomeTweetClick        = 301
+  HomeVideoView         = 302
+  HomeSongbirdShowMore  = 303
+
+
+  InterestsRankerRecentSearches = 401  // For Interests Candidate Expansion
+
+  UserInterestedIn      = 501
+  MBCG                = 503
+  // Other Metric Tags
+} (persisted='true', hasPersonalData='true')

topic-social-proof/server/src/main/thrift/tweet_info.thrift

@@ -0,0 +1,26 @@
+namespace java com.twitter.tsp.thriftjava
+namespace py gen.twitter.tsp
+#@namespace scala com.twitter.tsp.thriftscala
+#@namespace strato com.twitter.tsp.strato
+
+struct TspTweetInfo {
+  1: required i64 authorId
+  2: required i64 favCount
+  3: optional string language
+  6: optional bool hasImage
+  7: optional bool hasVideo
+  8: optional bool hasGif
+  9: optional bool isNsfwAuthor
+  10: optional bool isKGODenylist
+  11: optional bool isNullcast
+  // available if the tweet contains video
+  12: optional i32 videoDurationSeconds
+  13: optional bool isHighMediaResolution
+  14: optional bool isVerticalAspectRatio
+  // health signal scores
+  15: optional bool isPassAgathaHealthFilterStrictest
+  16: optional bool isPassTweetHealthFilterStrictest
+  17: optional bool isReply
+  18: optional bool hasMultipleMedia
+  23: optional bool hasUrl
+}(persisted='false', hasPersonalData='true')

unified_user_actions/.gitignore

@@ -0,0 +1,4 @@
+.DS_Store
+CONFIG.ini
+PROJECT
+docs

unified_user_actions/BUILD.bazel

@@ -0,0 +1 @@
+# This prevents SQ query from grabbing //:all since it traverses up once to find a BUILD

unified_user_actions/README.md

@@ -0,0 +1,10 @@
+# Unified User Actions (UUA)
+
+**Unified User Actions** (UUA) is a centralized, real-time stream of user actions on Twitter, consumed by various product, ML, and marketing teams. UUA reads client-side and server-side event streams that contain the user's actions and generates a unified real-time user actions Kafka stream. The Kafka stream is replicated to HDFS, GCP Pubsub, GCP GCS, GCP BigQuery.  The user actions include public actions such as favorites, retweets, replies and implicit actions like bookmark, impression, video view.
+
+## Components 
+
+- adapter: transform the raw inputs to UUA Thrift output
+- client: Kafka client related utils
+- kafka: more specific Kafka utils like customized serde
+- service: deployment, modules and services

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/AbstractAdapter.scala

@@ -0,0 +1,19 @@
+package com.twitter.unified_user_actions.adapter
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.finagle.stats.StatsReceiver
+
+trait AbstractAdapter[INPUT, OUTK, OUTV] extends Serializable {
+
+  /**
+   * The basic input -> seq[output] adapter which concrete adapters should extend from
+   * @param input a single INPUT
+   * @return A list of (OUTK, OUTV) tuple. The OUTK is the output key mainly for publishing to Kafka (or Pubsub).
+   *         If other processing, e.g. offline batch processing, doesn't require the output key then it can drop it
+   *         like source.adaptOneToKeyedMany.map(_._2)
+   */
+  def adaptOneToKeyedMany(
+    input: INPUT,
+    statsReceiver: StatsReceiver = NullStatsReceiver
+  ): Seq[(OUTK, OUTV)]
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/BUILD

@@ -0,0 +1,11 @@
+scala_library(
+    name = "base",
+    sources = [
+        "AbstractAdapter.scala",
+    ],
+    compiler_option_sets = ["fatal_warnings"],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "util/util-stats/src/main/scala/com/twitter/finagle/stats",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/AdsCallbackEngagement.scala

@@ -0,0 +1,125 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.thriftscala._
+
+object AdsCallbackEngagement {
+  object PromotedTweetFav extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetFav)
+
+  object PromotedTweetUnfav extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetUnfav)
+
+  object PromotedTweetReply extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetReply)
+
+  object PromotedTweetRetweet
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetRetweet)
+
+  object PromotedTweetBlockAuthor
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetBlockAuthor)
+
+  object PromotedTweetUnblockAuthor
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetUnblockAuthor)
+
+  object PromotedTweetComposeTweet
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetComposeTweet)
+
+  object PromotedTweetClick extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetClick)
+
+  object PromotedTweetReport extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetReport)
+
+  object PromotedProfileFollow
+      extends ProfileAdsCallbackEngagement(ActionType.ServerPromotedProfileFollow)
+
+  object PromotedProfileUnfollow
+      extends ProfileAdsCallbackEngagement(ActionType.ServerPromotedProfileUnfollow)
+
+  object PromotedTweetMuteAuthor
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetMuteAuthor)
+
+  object PromotedTweetClickProfile
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetClickProfile)
+
+  object PromotedTweetClickHashtag
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetClickHashtag)
+
+  object PromotedTweetOpenLink
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetOpenLink) {
+    override def getItem(input: SpendServerEvent): Option[Item] = {
+      input.engagementEvent.flatMap { e =>
+        e.impressionData.flatMap { i =>
+          getPromotedTweetInfo(
+            i.promotedTweetId,
+            i.advertiserId,
+            tweetActionInfoOpt = Some(
+              TweetActionInfo.ServerPromotedTweetOpenLink(
+                ServerPromotedTweetOpenLink(url = e.url))))
+        }
+      }
+    }
+  }
+
+  object PromotedTweetCarouselSwipeNext
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetCarouselSwipeNext)
+
+  object PromotedTweetCarouselSwipePrevious
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetCarouselSwipePrevious)
+
+  object PromotedTweetLingerImpressionShort
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetLingerImpressionShort)
+
+  object PromotedTweetLingerImpressionMedium
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetLingerImpressionMedium)
+
+  object PromotedTweetLingerImpressionLong
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetLingerImpressionLong)
+
+  object PromotedTweetClickSpotlight
+      extends BaseTrendAdsCallbackEngagement(ActionType.ServerPromotedTweetClickSpotlight)
+
+  object PromotedTweetViewSpotlight
+      extends BaseTrendAdsCallbackEngagement(ActionType.ServerPromotedTweetViewSpotlight)
+
+  object PromotedTrendView
+      extends BaseTrendAdsCallbackEngagement(ActionType.ServerPromotedTrendView)
+
+  object PromotedTrendClick
+      extends BaseTrendAdsCallbackEngagement(ActionType.ServerPromotedTrendClick)
+
+  object PromotedTweetVideoPlayback25
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoPlayback25)
+
+  object PromotedTweetVideoPlayback50
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoPlayback50)
+
+  object PromotedTweetVideoPlayback75
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoPlayback75)
+
+  object PromotedTweetVideoAdPlayback25
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoAdPlayback25)
+
+  object PromotedTweetVideoAdPlayback50
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoAdPlayback50)
+
+  object PromotedTweetVideoAdPlayback75
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerPromotedTweetVideoAdPlayback75)
+
+  object TweetVideoAdPlayback25
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerTweetVideoAdPlayback25)
+
+  object TweetVideoAdPlayback50
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerTweetVideoAdPlayback50)
+
+  object TweetVideoAdPlayback75
+      extends BaseVideoAdsCallbackEngagement(ActionType.ServerTweetVideoAdPlayback75)
+
+  object PromotedTweetDismissWithoutReason
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetDismissWithoutReason)
+
+  object PromotedTweetDismissUninteresting
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetDismissUninteresting)
+
+  object PromotedTweetDismissRepetitive
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetDismissRepetitive)
+
+  object PromotedTweetDismissSpam
+      extends BaseAdsCallbackEngagement(ActionType.ServerPromotedTweetDismissSpam)
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/AdsCallbackEngagementsAdapter.scala

@@ -0,0 +1,28 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.finatra.kafka.serde.UnKeyed
+import com.twitter.unified_user_actions.adapter.AbstractAdapter
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+
+class AdsCallbackEngagementsAdapter
+    extends AbstractAdapter[SpendServerEvent, UnKeyed, UnifiedUserAction] {
+
+  import AdsCallbackEngagementsAdapter._
+
+  override def adaptOneToKeyedMany(
+    input: SpendServerEvent,
+    statsReceiver: StatsReceiver = NullStatsReceiver
+  ): Seq[(UnKeyed, UnifiedUserAction)] =
+    adaptEvent(input).map { e => (UnKeyed, e) }
+}
+
+object AdsCallbackEngagementsAdapter {
+  def adaptEvent(input: SpendServerEvent): Seq[UnifiedUserAction] = {
+    val baseEngagements: Seq[BaseAdsCallbackEngagement] =
+      EngagementTypeMappings.getEngagementMappings(Option(input).flatMap(_.engagementEvent))
+    baseEngagements.flatMap(_.getUUA(input))
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/BUILD

@@ -0,0 +1,18 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    compiler_option_sets = ["fatal_warnings"],
+    tags = [
+        "bazel-compatible",
+        "bazel-only",
+    ],
+    dependencies = [
+        "kafka/finagle-kafka/finatra-kafka/src/main/scala",
+        "src/thrift/com/twitter/ads/billing/spendserver:spendserver_thrift-scala",
+        "src/thrift/com/twitter/ads/eventstream:eventstream-scala",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter:base",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common",
+        "unified_user_actions/thrift/src/main/thrift/com/twitter/unified_user_actions:unified_user_actions-scala",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/BaseAdsCallbackEngagement.scala

@@ -0,0 +1,68 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.adapter.common.AdapterUtils
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.AuthorInfo
+import com.twitter.unified_user_actions.thriftscala.EventMetadata
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.SourceLineage
+import com.twitter.unified_user_actions.thriftscala.TweetInfo
+import com.twitter.unified_user_actions.thriftscala.TweetActionInfo
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala.UserIdentifier
+
+abstract class BaseAdsCallbackEngagement(actionType: ActionType) {
+
+  protected def getItem(input: SpendServerEvent): Option[Item] = {
+    input.engagementEvent.flatMap { e =>
+      e.impressionData.flatMap { i =>
+        getPromotedTweetInfo(i.promotedTweetId, i.advertiserId)
+      }
+    }
+  }
+
+  protected def getPromotedTweetInfo(
+    promotedTweetIdOpt: Option[Long],
+    advertiserId: Long,
+    tweetActionInfoOpt: Option[TweetActionInfo] = None
+  ): Option[Item] = {
+    promotedTweetIdOpt.map { promotedTweetId =>
+      Item.TweetInfo(
+        TweetInfo(
+          actionTweetId = promotedTweetId,
+          actionTweetAuthorInfo = Some(AuthorInfo(authorId = Some(advertiserId))),
+          tweetActionInfo = tweetActionInfoOpt)
+      )
+    }
+  }
+
+  def getUUA(input: SpendServerEvent): Option[UnifiedUserAction] = {
+    val userIdentifier: UserIdentifier =
+      UserIdentifier(
+        userId = input.engagementEvent.flatMap(e => e.clientInfo.flatMap(_.userId64)),
+        guestIdMarketing = input.engagementEvent.flatMap(e => e.clientInfo.flatMap(_.guestId)),
+      )
+
+    getItem(input).map { item =>
+      UnifiedUserAction(
+        userIdentifier = userIdentifier,
+        item = item,
+        actionType = actionType,
+        eventMetadata = getEventMetadata(input),
+      )
+    }
+  }
+
+  protected def getEventMetadata(input: SpendServerEvent): EventMetadata =
+    EventMetadata(
+      sourceTimestampMs = input.engagementEvent
+        .map { e => e.engagementEpochTimeMilliSec }.getOrElse(AdapterUtils.currentTimestampMs),
+      receivedTimestampMs = AdapterUtils.currentTimestampMs,
+      sourceLineage = SourceLineage.ServerAdsCallbackEngagements,
+      language = input.engagementEvent.flatMap { e => e.clientInfo.flatMap(_.languageCode) },
+      countryCode = input.engagementEvent.flatMap { e => e.clientInfo.flatMap(_.countryCode) },
+      clientAppId =
+        input.engagementEvent.flatMap { e => e.clientInfo.flatMap(_.clientId) }.map { _.toLong },
+    )
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/BaseTrendAdsCallbackEngagement.scala

@@ -0,0 +1,18 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseTrendAdsCallbackEngagement(actionType: ActionType)
+    extends BaseAdsCallbackEngagement(actionType = actionType) {
+
+  override protected def getItem(input: SpendServerEvent): Option[Item] = {
+    input.engagementEvent.flatMap { e =>
+      e.impressionData.flatMap { i =>
+        i.promotedTrendId.map { promotedTrendId =>
+          Item.TrendInfo(TrendInfo(actionTrendId = promotedTrendId))
+        }
+      }
+    }
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/BaseVideoAdsCallbackEngagement.scala

@@ -0,0 +1,54 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.AuthorInfo
+import com.twitter.unified_user_actions.thriftscala.TweetVideoWatch
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.TweetActionInfo
+import com.twitter.unified_user_actions.thriftscala.TweetInfo
+
+abstract class BaseVideoAdsCallbackEngagement(actionType: ActionType)
+    extends BaseAdsCallbackEngagement(actionType = actionType) {
+
+  override def getItem(input: SpendServerEvent): Option[Item] = {
+    input.engagementEvent.flatMap { e =>
+      e.impressionData.flatMap { i =>
+        getTweetInfo(i.promotedTweetId, i.organicTweetId, i.advertiserId, input)
+      }
+    }
+  }
+
+  private def getTweetInfo(
+    promotedTweetId: Option[Long],
+    organicTweetId: Option[Long],
+    advertiserId: Long,
+    input: SpendServerEvent
+  ): Option[Item] = {
+    val actionedTweetIdOpt: Option[Long] =
+      if (promotedTweetId.isEmpty) organicTweetId else promotedTweetId
+    actionedTweetIdOpt.map { actionTweetId =>
+      Item.TweetInfo(
+        TweetInfo(
+          actionTweetId = actionTweetId,
+          actionTweetAuthorInfo = Some(AuthorInfo(authorId = Some(advertiserId))),
+          tweetActionInfo = Some(
+            TweetActionInfo.TweetVideoWatch(
+              TweetVideoWatch(
+                isMonetizable = Some(true),
+                videoOwnerId = input.engagementEvent
+                  .flatMap(e => e.cardEngagement).flatMap(_.amplifyDetails).flatMap(_.videoOwnerId),
+                videoUuid = input.engagementEvent
+                  .flatMap(_.cardEngagement).flatMap(_.amplifyDetails).flatMap(_.videoUuid),
+                prerollOwnerId = input.engagementEvent
+                  .flatMap(e => e.cardEngagement).flatMap(_.amplifyDetails).flatMap(
+                    _.prerollOwnerId),
+                prerollUuid = input.engagementEvent
+                  .flatMap(_.cardEngagement).flatMap(_.amplifyDetails).flatMap(_.prerollUuid)
+              ))
+          )
+        ),
+      )
+    }
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/EngagementTypeMappings.scala

@@ -0,0 +1,69 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.eventstream.thriftscala.EngagementEvent
+import com.twitter.adserver.thriftscala.EngagementType
+import com.twitter.unified_user_actions.adapter.ads_callback_engagements.AdsCallbackEngagement._
+
+object EngagementTypeMappings {
+
+  /**
+   * Ads could be Tweets or non-Tweets. Since UUA explicitly sets the item type, it is
+   * possible that one Ads Callback engagement type maps to multiple UUA action types.
+   */
+  def getEngagementMappings(
+    engagementEvent: Option[EngagementEvent]
+  ): Seq[BaseAdsCallbackEngagement] = {
+    val promotedTweetId: Option[Long] =
+      engagementEvent.flatMap(_.impressionData).flatMap(_.promotedTweetId)
+    engagementEvent
+      .map(event =>
+        event.engagementType match {
+          case EngagementType.Fav => Seq(PromotedTweetFav)
+          case EngagementType.Unfav => Seq(PromotedTweetUnfav)
+          case EngagementType.Reply => Seq(PromotedTweetReply)
+          case EngagementType.Retweet => Seq(PromotedTweetRetweet)
+          case EngagementType.Block => Seq(PromotedTweetBlockAuthor)
+          case EngagementType.Unblock => Seq(PromotedTweetUnblockAuthor)
+          case EngagementType.Send => Seq(PromotedTweetComposeTweet)
+          case EngagementType.Detail => Seq(PromotedTweetClick)
+          case EngagementType.Report => Seq(PromotedTweetReport)
+          case EngagementType.Follow => Seq(PromotedProfileFollow)
+          case EngagementType.Unfollow => Seq(PromotedProfileUnfollow)
+          case EngagementType.Mute => Seq(PromotedTweetMuteAuthor)
+          case EngagementType.ProfilePic => Seq(PromotedTweetClickProfile)
+          case EngagementType.ScreenName => Seq(PromotedTweetClickProfile)
+          case EngagementType.UserName => Seq(PromotedTweetClickProfile)
+          case EngagementType.Hashtag => Seq(PromotedTweetClickHashtag)
+          case EngagementType.Url => Seq(PromotedTweetOpenLink)
+          case EngagementType.CarouselSwipeNext => Seq(PromotedTweetCarouselSwipeNext)
+          case EngagementType.CarouselSwipePrevious => Seq(PromotedTweetCarouselSwipePrevious)
+          case EngagementType.DwellShort => Seq(PromotedTweetLingerImpressionShort)
+          case EngagementType.DwellMedium => Seq(PromotedTweetLingerImpressionMedium)
+          case EngagementType.DwellLong => Seq(PromotedTweetLingerImpressionLong)
+          case EngagementType.SpotlightClick => Seq(PromotedTweetClickSpotlight)
+          case EngagementType.SpotlightView => Seq(PromotedTweetViewSpotlight)
+          case EngagementType.TrendView => Seq(PromotedTrendView)
+          case EngagementType.TrendClick => Seq(PromotedTrendClick)
+          case EngagementType.VideoContentPlayback25 => Seq(PromotedTweetVideoPlayback25)
+          case EngagementType.VideoContentPlayback50 => Seq(PromotedTweetVideoPlayback50)
+          case EngagementType.VideoContentPlayback75 => Seq(PromotedTweetVideoPlayback75)
+          case EngagementType.VideoAdPlayback25 if promotedTweetId.isDefined =>
+            Seq(PromotedTweetVideoAdPlayback25)
+          case EngagementType.VideoAdPlayback25 if promotedTweetId.isEmpty =>
+            Seq(TweetVideoAdPlayback25)
+          case EngagementType.VideoAdPlayback50 if promotedTweetId.isDefined =>
+            Seq(PromotedTweetVideoAdPlayback50)
+          case EngagementType.VideoAdPlayback50 if promotedTweetId.isEmpty =>
+            Seq(TweetVideoAdPlayback50)
+          case EngagementType.VideoAdPlayback75 if promotedTweetId.isDefined =>
+            Seq(PromotedTweetVideoAdPlayback75)
+          case EngagementType.VideoAdPlayback75 if promotedTweetId.isEmpty =>
+            Seq(TweetVideoAdPlayback75)
+          case EngagementType.DismissRepetitive => Seq(PromotedTweetDismissRepetitive)
+          case EngagementType.DismissSpam => Seq(PromotedTweetDismissSpam)
+          case EngagementType.DismissUninteresting => Seq(PromotedTweetDismissUninteresting)
+          case EngagementType.DismissWithoutReason => Seq(PromotedTweetDismissWithoutReason)
+          case _ => Nil
+        }).toSeq.flatten
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/ads_callback_engagements/ProfileAdsCallbackEngagement.scala

@@ -0,0 +1,26 @@
+package com.twitter.unified_user_actions.adapter.ads_callback_engagements
+
+import com.twitter.ads.spendserver.thriftscala.SpendServerEvent
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProfileInfo
+
+abstract class ProfileAdsCallbackEngagement(actionType: ActionType)
+    extends BaseAdsCallbackEngagement(actionType) {
+
+  override protected def getItem(input: SpendServerEvent): Option[Item] = {
+    input.engagementEvent.flatMap { e =>
+      e.impressionData.flatMap { i =>
+        getProfileInfo(i.advertiserId)
+      }
+    }
+  }
+
+  protected def getProfileInfo(advertiserId: Long): Option[Item] = {
+    Some(
+      Item.ProfileInfo(
+        ProfileInfo(
+          actionProfileId = advertiserId
+        )))
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/BUILD

@@ -0,0 +1,13 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "client-events/thrift/src/thrift/storage/twitter/behavioral_event:behavioral_event-scala",
+        "kafka/finagle-kafka/finatra-kafka/src/main/scala",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter:base",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common",
+        "unified_user_actions/thrift/src/main/thrift/com/twitter/unified_user_actions:unified_user_actions-scala",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/BaseBCEAdapter.scala

@@ -0,0 +1,96 @@
+package com.twitter.unified_user_actions.adapter.behavioral_client_event
+
+import com.twitter.client_event_entities.serverside_context_key.latest.thriftscala.FlattenedServersideContextKey
+import com.twitter.storage.behavioral_event.thriftscala.EventLogContext
+import com.twitter.storage.behavioral_event.thriftscala.FlattenedEventLog
+import com.twitter.unified_user_actions.adapter.common.AdapterUtils
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.BreadcrumbTweet
+import com.twitter.unified_user_actions.thriftscala.ClientEventNamespace
+import com.twitter.unified_user_actions.thriftscala.EventMetadata
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.ProductSurfaceInfo
+import com.twitter.unified_user_actions.thriftscala.SourceLineage
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala.UserIdentifier
+
+case class ProductSurfaceRelated(
+  productSurface: Option[ProductSurface],
+  productSurfaceInfo: Option[ProductSurfaceInfo])
+
+trait BaseBCEAdapter {
+  def toUUA(e: FlattenedEventLog): Seq[UnifiedUserAction]
+
+  protected def getUserIdentifier(c: EventLogContext): UserIdentifier =
+    UserIdentifier(
+      userId = c.userId,
+      guestIdMarketing = c.guestIdMarketing
+    )
+
+  protected def getEventMetadata(e: FlattenedEventLog): EventMetadata =
+    EventMetadata(
+      sourceLineage = SourceLineage.BehavioralClientEvents,
+      sourceTimestampMs =
+        e.context.driftAdjustedEventCreatedAtMs.getOrElse(e.context.eventCreatedAtMs),
+      receivedTimestampMs = AdapterUtils.currentTimestampMs,
+      // Client UI language or from Gizmoduck which is what user set in Twitter App.
+      // Please see more at https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/finatra-internal/international/src/main/scala/com/twitter/finatra/international/LanguageIdentifier.scala
+      // The format should be ISO 639-1.
+      language = e.context.languageCode.map(AdapterUtils.normalizeLanguageCode),
+      // Country code could be IP address (geoduck) or User registration country (gizmoduck) and the former takes precedence.
+      // We don’t know exactly which one is applied, unfortunately,
+      // see https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/finatra-internal/international/src/main/scala/com/twitter/finatra/international/CountryIdentifier.scala
+      // The format should be ISO_3166-1_alpha-2.
+      countryCode = e.context.countryCode.map(AdapterUtils.normalizeCountryCode),
+      clientAppId = e.context.clientApplicationId,
+      clientVersion = e.context.clientVersion,
+      clientPlatform = e.context.clientPlatform,
+      viewHierarchy = e.v1ViewTypeHierarchy,
+      clientEventNamespace = Some(
+        ClientEventNamespace(
+          page = e.page,
+          section = e.section,
+          element = e.element,
+          action = e.actionName,
+          subsection = e.subsection
+        )),
+      breadcrumbViews = e.v1BreadcrumbViewTypeHierarchy,
+      breadcrumbTweets = e.v1BreadcrumbTweetIds.map { breadcrumbs =>
+        breadcrumbs.map { breadcrumb =>
+          BreadcrumbTweet(
+            tweetId = breadcrumb.serversideContextId.toLong,
+            sourceComponent = breadcrumb.sourceComponent)
+        }
+      }
+    )
+
+  protected def getBreadcrumbTweetIds(
+    breadcrumbTweetIds: Option[Seq[FlattenedServersideContextKey]]
+  ): Seq[BreadcrumbTweet] =
+    breadcrumbTweetIds
+      .getOrElse(Nil).map(breadcrumb => {
+        BreadcrumbTweet(
+          tweetId = breadcrumb.serversideContextId.toLong,
+          sourceComponent = breadcrumb.sourceComponent)
+      })
+
+  protected def getBreadcrumbViews(breadcrumbView: Option[Seq[String]]): Seq[String] =
+    breadcrumbView.getOrElse(Nil)
+
+  protected def getUnifiedUserAction(
+    event: FlattenedEventLog,
+    actionType: ActionType,
+    item: Item,
+    productSurface: Option[ProductSurface] = None,
+    productSurfaceInfo: Option[ProductSurfaceInfo] = None
+  ): UnifiedUserAction =
+    UnifiedUserAction(
+      userIdentifier = getUserIdentifier(event.context),
+      actionType = actionType,
+      item = item,
+      eventMetadata = getEventMetadata(event),
+      productSurface = productSurface,
+      productSurfaceInfo = productSurfaceInfo
+    )
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/BehavioralClientEventAdapter.scala

@@ -0,0 +1,39 @@
+package com.twitter.unified_user_actions.adapter.behavioral_client_event
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.finatra.kafka.serde.UnKeyed
+import com.twitter.storage.behavioral_event.thriftscala.FlattenedEventLog
+import com.twitter.unified_user_actions.adapter.AbstractAdapter
+import com.twitter.unified_user_actions.thriftscala._
+
+class BehavioralClientEventAdapter
+    extends AbstractAdapter[FlattenedEventLog, UnKeyed, UnifiedUserAction] {
+
+  import BehavioralClientEventAdapter._
+
+  override def adaptOneToKeyedMany(
+    input: FlattenedEventLog,
+    statsReceiver: StatsReceiver = NullStatsReceiver
+  ): Seq[(UnKeyed, UnifiedUserAction)] =
+    adaptEvent(input).map { e => (UnKeyed, e) }
+}
+
+object BehavioralClientEventAdapter {
+  def adaptEvent(e: FlattenedEventLog): Seq[UnifiedUserAction] =
+    // See go/bcecoverage for event namespaces, usage and coverage details
+    Option(e)
+      .map { e =>
+        (e.page, e.actionName) match {
+          case (Some("tweet_details"), Some("impress")) =>
+            TweetImpressionBCEAdapter.TweetDetails.toUUA(e)
+          case (Some("fullscreen_video"), Some("impress")) =>
+            TweetImpressionBCEAdapter.FullscreenVideo.toUUA(e)
+          case (Some("fullscreen_image"), Some("impress")) =>
+            TweetImpressionBCEAdapter.FullscreenImage.toUUA(e)
+          case (Some("profile"), Some("impress")) =>
+            ProfileImpressionBCEAdapter.Profile.toUUA(e)
+          case _ => Nil
+        }
+      }.getOrElse(Nil)
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/ImpressionBCEAdapter.scala

@@ -0,0 +1,34 @@
+package com.twitter.unified_user_actions.adapter.behavioral_client_event
+
+import com.twitter.client.behavioral_event.action.impress.latest.thriftscala.Impress
+import com.twitter.client_event_entities.serverside_context_key.latest.thriftscala.FlattenedServersideContextKey
+import com.twitter.unified_user_actions.thriftscala.Item
+
+trait ImpressionBCEAdapter extends BaseBCEAdapter {
+  type ImpressedItem <: Item
+
+  def getImpressedItem(
+    context: FlattenedServersideContextKey,
+    impression: Impress
+  ): ImpressedItem
+
+  /**
+   * The start time of an impression in milliseconds since epoch. In BCE, the impression
+   * tracking clock will start immediately after the page is visible with no initial delay.
+   */
+  def getImpressedStartTimestamp(impression: Impress): Long =
+    impression.visibilityPctDwellStartMs
+
+  /**
+   * The end time of an impression in milliseconds since epoch. In BCE, the impression
+   * tracking clock will end before the user exit the page.
+   */
+  def getImpressedEndTimestamp(impression: Impress): Long =
+    impression.visibilityPctDwellEndMs
+
+  /**
+   * The UI component that hosted the impressed item.
+   */
+  def getImpressedUISourceComponent(context: FlattenedServersideContextKey): String =
+    context.sourceComponent
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/ProfileImpressionBCEAdapter.scala

@@ -0,0 +1,52 @@
+package com.twitter.unified_user_actions.adapter.behavioral_client_event
+
+import com.twitter.client.behavioral_event.action.impress.latest.thriftscala.Impress
+import com.twitter.client_event_entities.serverside_context_key.latest.thriftscala.FlattenedServersideContextKey
+import com.twitter.storage.behavioral_event.thriftscala.FlattenedEventLog
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.ClientProfileV2Impression
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.ProfileActionInfo
+import com.twitter.unified_user_actions.thriftscala.ProfileInfo
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+
+object ProfileImpressionBCEAdapter {
+  val Profile = new ProfileImpressionBCEAdapter()
+}
+
+class ProfileImpressionBCEAdapter extends ImpressionBCEAdapter {
+  override type ImpressedItem = Item.ProfileInfo
+
+  override def toUUA(e: FlattenedEventLog): Seq[UnifiedUserAction] =
+    (e.v2Impress, e.v1UserIds) match {
+      case (Some(v2Impress), Some(v1UserIds)) =>
+        v1UserIds.map { user =>
+          getUnifiedUserAction(
+            event = e,
+            actionType = ActionType.ClientProfileV2Impression,
+            item = getImpressedItem(user, v2Impress),
+            productSurface = Some(ProductSurface.ProfilePage)
+          )
+        }
+      case _ => Nil
+    }
+
+  override def getImpressedItem(
+    context: FlattenedServersideContextKey,
+    impression: Impress
+  ): ImpressedItem =
+    Item.ProfileInfo(
+      ProfileInfo(
+        actionProfileId = context.serversideContextId.toLong,
+        profileActionInfo = Some(
+          ProfileActionInfo.ClientProfileV2Impression(
+            ClientProfileV2Impression(
+              impressStartTimestampMs = getImpressedStartTimestamp(impression),
+              impressEndTimestampMs = getImpressedEndTimestamp(impression),
+              sourceComponent = getImpressedUISourceComponent(context)
+            )
+          )
+        )
+      ))
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/behavioral_client_event/TweetImpressionBCEAdapter.scala

@@ -0,0 +1,84 @@
+package com.twitter.unified_user_actions.adapter.behavioral_client_event
+
+import com.twitter.client.behavioral_event.action.impress.latest.thriftscala.Impress
+import com.twitter.client_event_entities.serverside_context_key.latest.thriftscala.FlattenedServersideContextKey
+import com.twitter.storage.behavioral_event.thriftscala.FlattenedEventLog
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.ClientTweetV2Impression
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.TweetActionInfo
+import com.twitter.unified_user_actions.thriftscala.TweetInfo
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+
+object TweetImpressionBCEAdapter {
+  val TweetDetails = new TweetImpressionBCEAdapter(ActionType.ClientTweetV2Impression)
+  val FullscreenVideo = new TweetImpressionBCEAdapter(
+    ActionType.ClientTweetVideoFullscreenV2Impression)
+  val FullscreenImage = new TweetImpressionBCEAdapter(
+    ActionType.ClientTweetImageFullscreenV2Impression)
+}
+
+class TweetImpressionBCEAdapter(actionType: ActionType) extends ImpressionBCEAdapter {
+  override type ImpressedItem = Item.TweetInfo
+
+  override def toUUA(e: FlattenedEventLog): Seq[UnifiedUserAction] =
+    (actionType, e.v2Impress, e.v1TweetIds, e.v1BreadcrumbTweetIds) match {
+      case (ActionType.ClientTweetV2Impression, Some(v2Impress), Some(v1TweetIds), _) =>
+        toUUAEvents(e, v2Impress, v1TweetIds)
+      case (
+            ActionType.ClientTweetVideoFullscreenV2Impression,
+            Some(v2Impress),
+            _,
+            Some(v1BreadcrumbTweetIds)) =>
+        toUUAEvents(e, v2Impress, v1BreadcrumbTweetIds)
+      case (
+            ActionType.ClientTweetImageFullscreenV2Impression,
+            Some(v2Impress),
+            _,
+            Some(v1BreadcrumbTweetIds)) =>
+        toUUAEvents(e, v2Impress, v1BreadcrumbTweetIds)
+      case _ => Nil
+    }
+
+  private def toUUAEvents(
+    e: FlattenedEventLog,
+    v2Impress: Impress,
+    v1TweetIds: Seq[FlattenedServersideContextKey]
+  ): Seq[UnifiedUserAction] =
+    v1TweetIds.map { tweet =>
+      getUnifiedUserAction(
+        event = e,
+        actionType = actionType,
+        item = getImpressedItem(tweet, v2Impress),
+        productSurface = getProductSurfaceRelated.productSurface,
+        productSurfaceInfo = getProductSurfaceRelated.productSurfaceInfo
+      )
+    }
+
+  override def getImpressedItem(
+    context: FlattenedServersideContextKey,
+    impression: Impress
+  ): ImpressedItem =
+    Item.TweetInfo(
+      TweetInfo(
+        actionTweetId = context.serversideContextId.toLong,
+        tweetActionInfo = Some(
+          TweetActionInfo.ClientTweetV2Impression(
+            ClientTweetV2Impression(
+              impressStartTimestampMs = getImpressedStartTimestamp(impression),
+              impressEndTimestampMs = getImpressedEndTimestamp(impression),
+              sourceComponent = getImpressedUISourceComponent(context)
+            )
+          ))
+      ))
+
+  private def getProductSurfaceRelated: ProductSurfaceRelated =
+    actionType match {
+      case ActionType.ClientTweetV2Impression =>
+        ProductSurfaceRelated(
+          productSurface = Some(ProductSurface.TweetDetailsPage),
+          productSurfaceInfo = None)
+      case _ => ProductSurfaceRelated(productSurface = None, productSurfaceInfo = None)
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BUILD

@@ -0,0 +1,16 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "common-internal/analytics/client-analytics-data-layer/src/main/scala",
+        "kafka/finagle-kafka/finatra-kafka/src/main/scala",
+        "src/scala/com/twitter/loggedout/analytics/common",
+        "src/thrift/com/twitter/clientapp/gen:clientapp-scala",
+        "twadoop_config/configuration/log_categories/group/scribelib:client_event-scala",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter:base",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common",
+        "unified_user_actions/thrift/src/main/thrift/com/twitter/unified_user_actions:unified_user_actions-scala",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseCTAClientEvent.scala

@@ -0,0 +1,46 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala._
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+
+abstract class BaseCTAClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def toUnifiedUserAction(logEvent: LogEvent): Seq[UnifiedUserAction] = {
+    val logBase: Option[LogBase] = logEvent.logBase
+    val userIdentifier: UserIdentifier = UserIdentifier(
+      userId = logBase.flatMap(_.userId),
+      guestIdMarketing = logBase.flatMap(_.guestIdMarketing))
+    val uuaItem: Item = Item.CtaInfo(CTAInfo())
+    val eventTimestamp = logBase.flatMap(getSourceTimestamp).getOrElse(0L)
+    val ceItem = LogEventItem.unsafeEmpty
+
+    val productSurface: Option[ProductSurface] = ProductSurfaceUtils
+      .getProductSurface(logEvent.eventNamespace)
+
+    val eventMetaData: EventMetadata = ClientEventCommonUtils
+      .getEventMetadata(
+        eventTimestamp = eventTimestamp,
+        logEvent = logEvent,
+        ceItem = ceItem,
+        productSurface = productSurface
+      )
+
+    Seq(
+      UnifiedUserAction(
+        userIdentifier = userIdentifier,
+        item = uuaItem,
+        actionType = actionType,
+        eventMetadata = eventMetaData,
+        productSurface = productSurface,
+        productSurfaceInfo =
+          ProductSurfaceUtils.getProductSurfaceInfo(productSurface, ceItem, logEvent)
+      ))
+  }
+
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseCardClientEvent.scala

@@ -0,0 +1,26 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.CardInfo
+import com.twitter.unified_user_actions.thriftscala.Item
+
+abstract class BaseCardClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.ignoreItemType(itemTypeOpt)
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = Some(
+    Item.CardInfo(
+      CardInfo(
+        id = ceItem.id,
+        itemType = ceItem.itemType,
+        actionTweetAuthorInfo = ClientEventCommonUtils.getAuthorInfo(ceItem),
+      ))
+  )
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseClientEvent.scala

@@ -0,0 +1,68 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.logbase.thriftscala.ClientEventReceiver
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseClientEvent(actionType: ActionType) {
+  def toUnifiedUserAction(logEvent: LogEvent): Seq[UnifiedUserAction] = {
+    val logBase: Option[LogBase] = logEvent.logBase
+
+    for {
+      ed <- logEvent.eventDetails.toSeq
+      items <- ed.items.toSeq
+      ceItem <- items
+      eventTimestamp <- logBase.flatMap(getSourceTimestamp)
+      uuaItem <- getUuaItem(ceItem, logEvent)
+      if isItemTypeValid(ceItem.itemType)
+    } yield {
+      val userIdentifier: UserIdentifier = UserIdentifier(
+        userId = logBase.flatMap(_.userId),
+        guestIdMarketing = logBase.flatMap(_.guestIdMarketing))
+
+      val productSurface: Option[ProductSurface] = ProductSurfaceUtils
+        .getProductSurface(logEvent.eventNamespace)
+
+      val eventMetaData: EventMetadata = ClientEventCommonUtils
+        .getEventMetadata(
+          eventTimestamp = eventTimestamp,
+          logEvent = logEvent,
+          ceItem = ceItem,
+          productSurface = productSurface
+        )
+
+      UnifiedUserAction(
+        userIdentifier = userIdentifier,
+        item = uuaItem,
+        actionType = actionType,
+        eventMetadata = eventMetaData,
+        productSurface = productSurface,
+        productSurfaceInfo =
+          ProductSurfaceUtils.getProductSurfaceInfo(productSurface, ceItem, logEvent)
+      )
+    }
+  }
+
+  def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = for (actionTweetId <- ceItem.id)
+    yield Item.TweetInfo(
+      ClientEventCommonUtils
+        .getBasicTweetInfo(actionTweetId, ceItem, logEvent.eventNamespace))
+
+  // default implementation filters items of type tweet
+  // override in the subclass implementation to filter items of other types
+  def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.isItemTypeTweet(itemTypeOpt)
+
+  def getSourceTimestamp(logBase: LogBase): Option[Long] =
+    logBase.clientEventReceiver match {
+      case Some(ClientEventReceiver.CesHttp) | Some(ClientEventReceiver.CesThrift) =>
+        logBase.driftAdjustedEventCreatedAtMs
+      case _ => Some(logBase.driftAdjustedEventCreatedAtMs.getOrElse(logBase.timestamp))
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseFeedbackSubmitClientEvent.scala

@@ -0,0 +1,46 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseFeedbackSubmitClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = {
+    logEvent.eventNamespace.flatMap(_.page).flatMap {
+      case "search" =>
+        val searchInfoUtil = new SearchInfoUtils(ceItem)
+        searchInfoUtil.getQueryOptFromItem(logEvent).flatMap { query =>
+          val isRelevant: Boolean = logEvent.eventNamespace
+            .flatMap(_.element)
+            .contains("is_relevant")
+          logEvent.eventNamespace.flatMap(_.component).flatMap {
+            case "relevance_prompt_module" =>
+              for (actionTweetId <- ceItem.id)
+                yield Item.FeedbackPromptInfo(
+                  FeedbackPromptInfo(
+                    feedbackPromptActionInfo = FeedbackPromptActionInfo.TweetRelevantToSearch(
+                      TweetRelevantToSearch(
+                        searchQuery = query,
+                        tweetId = actionTweetId,
+                        isRelevant = Some(isRelevant)))))
+            case "did_you_find_it_module" =>
+              Some(
+                Item.FeedbackPromptInfo(FeedbackPromptInfo(feedbackPromptActionInfo =
+                  FeedbackPromptActionInfo.DidYouFindItSearch(
+                    DidYouFindItSearch(searchQuery = query, isRelevant = Some(isRelevant))))))
+          }
+        }
+      case _ => None
+    }
+
+  }
+
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.isItemTypeForSearchResultsPageFeedbackSubmit(itemTypeOpt)
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseNotificationTabClientEvent.scala

@@ -0,0 +1,48 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseNotificationTabClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  // itemType is `None` for Notification Tab events
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.ignoreItemType(itemTypeOpt)
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = for {
+    notificationTabDetails <- ceItem.notificationTabDetails
+    clientEventMetadata <- notificationTabDetails.clientEventMetadata
+    notificationId <- NotificationClientEventUtils.getNotificationIdForNotificationTab(ceItem)
+  } yield {
+    clientEventMetadata.tweetIds match {
+      // if `tweetIds` contain more than one Tweet id, create `MultiTweetNotification`
+      case Some(tweetIds) if tweetIds.size > 1 =>
+        Item.NotificationInfo(
+          NotificationInfo(
+            actionNotificationId = notificationId,
+            content = NotificationContent.MultiTweetNotification(
+              MultiTweetNotification(tweetIds = tweetIds))
+          ))
+      // if `tweetIds` contain exactly one Tweet id, create `TweetNotification`
+      case Some(tweetIds) if tweetIds.size == 1 =>
+        Item.NotificationInfo(
+          NotificationInfo(
+            actionNotificationId = notificationId,
+            content =
+              NotificationContent.TweetNotification(TweetNotification(tweetId = tweetIds.head))))
+      // if `tweetIds` are missing, create `UnknownNotification`
+      case _ =>
+        Item.NotificationInfo(
+          NotificationInfo(
+            actionNotificationId = notificationId,
+            content = NotificationContent.UnknownNotification(UnknownNotification())
+          ))
+    }
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseProfileClientEvent.scala

@@ -0,0 +1,25 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.adapter.client_event.ClientEventCommonUtils.getProfileIdFromUserItem
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProfileInfo
+
+abstract class BaseProfileClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.isItemTypeProfile(itemTypeOpt)
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] =
+    getProfileIdFromUserItem(ceItem).map { id =>
+      Item.ProfileInfo(
+        ProfileInfo(actionProfileId = id)
+      )
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BasePushNotificationClientEvent.scala

@@ -0,0 +1,22 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BasePushNotificationClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = for {
+    itemId <- ceItem.id
+    notificationId <- NotificationClientEventUtils.getNotificationIdForPushNotification(logEvent)
+  } yield {
+    Item.NotificationInfo(
+      NotificationInfo(
+        actionNotificationId = notificationId,
+        content = NotificationContent.TweetNotification(TweetNotification(tweetId = itemId))))
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseSearchTypeaheadEvent.scala

@@ -0,0 +1,87 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.adapter.client_event.ClientEventCommonUtils.getProfileIdFromUserItem
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.EventMetadata
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.TopicQueryResult
+import com.twitter.unified_user_actions.thriftscala.TypeaheadActionInfo
+import com.twitter.unified_user_actions.thriftscala.TypeaheadInfo
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala.UserIdentifier
+import com.twitter.unified_user_actions.thriftscala.UserResult
+
+abstract class BaseSearchTypeaheadEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def toUnifiedUserAction(logEvent: LogEvent): Seq[UnifiedUserAction] = {
+    val logBase: Option[LogBase] = logEvent.logBase
+
+    for {
+      ed <- logEvent.eventDetails.toSeq
+      targets <- ed.targets.toSeq
+      ceTarget <- targets
+      eventTimestamp <- logBase.flatMap(getSourceTimestamp)
+      uuaItem <- getUuaItem(ceTarget, logEvent)
+      if isItemTypeValid(ceTarget.itemType)
+    } yield {
+      val userIdentifier: UserIdentifier = UserIdentifier(
+        userId = logBase.flatMap(_.userId),
+        guestIdMarketing = logBase.flatMap(_.guestIdMarketing))
+
+      val productSurface: Option[ProductSurface] = ProductSurfaceUtils
+        .getProductSurface(logEvent.eventNamespace)
+
+      val eventMetaData: EventMetadata = ClientEventCommonUtils
+        .getEventMetadata(
+          eventTimestamp = eventTimestamp,
+          logEvent = logEvent,
+          ceItem = ceTarget,
+          productSurface = productSurface
+        )
+
+      UnifiedUserAction(
+        userIdentifier = userIdentifier,
+        item = uuaItem,
+        actionType = actionType,
+        eventMetadata = eventMetaData,
+        productSurface = productSurface,
+        productSurfaceInfo =
+          ProductSurfaceUtils.getProductSurfaceInfo(productSurface, ceTarget, logEvent)
+      )
+    }
+  }
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.isItemTypeTypeaheadResult(itemTypeOpt)
+
+  override def getUuaItem(
+    ceTarget: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] =
+    logEvent.searchDetails.flatMap(_.query).flatMap { query =>
+      ceTarget.itemType match {
+        case Some(ItemType.User) =>
+          getProfileIdFromUserItem(ceTarget).map { profileId =>
+            Item.TypeaheadInfo(
+              TypeaheadInfo(
+                actionQuery = query,
+                typeaheadActionInfo =
+                  TypeaheadActionInfo.UserResult(UserResult(profileId = profileId))))
+          }
+        case Some(ItemType.Search) =>
+          ceTarget.name.map { name =>
+            Item.TypeaheadInfo(
+              TypeaheadInfo(
+                actionQuery = query,
+                typeaheadActionInfo = TypeaheadActionInfo.TopicQueryResult(
+                  TopicQueryResult(suggestedTopicQuery = name))))
+          }
+        case _ => None
+      }
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseTopicClientEvent.scala

@@ -0,0 +1,23 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.TopicInfo
+
+abstract class BaseTopicClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+  override def isItemTypeValid(itemTypeOpt: Option[ItemType]): Boolean =
+    ItemTypeFilterPredicates.isItemTypeTopic(itemTypeOpt)
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] =
+    for (actionTopicId <- ClientEventCommonUtils.getTopicId(
+        ceItem = ceItem,
+        ceNamespaceOpt = logEvent.eventNamespace))
+      yield Item.TopicInfo(TopicInfo(actionTopicId = actionTopicId))
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseUASClientEvent.scala

@@ -0,0 +1,62 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseUASClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def toUnifiedUserAction(logEvent: LogEvent): Seq[UnifiedUserAction] = {
+    val logBase: Option[LogBase] = logEvent.logBase
+    val ceItem = LogEventItem.unsafeEmpty
+
+    val uuaOpt: Option[UnifiedUserAction] = for {
+      eventTimestamp <- logBase.flatMap(getSourceTimestamp)
+      uuaItem <- getUuaItem(ceItem, logEvent)
+    } yield {
+      val userIdentifier: UserIdentifier = UserIdentifier(
+        userId = logBase.flatMap(_.userId),
+        guestIdMarketing = logBase.flatMap(_.guestIdMarketing))
+
+      val productSurface: Option[ProductSurface] = ProductSurfaceUtils
+        .getProductSurface(logEvent.eventNamespace)
+
+      val eventMetaData: EventMetadata = ClientEventCommonUtils
+        .getEventMetadata(
+          eventTimestamp = eventTimestamp,
+          logEvent = logEvent,
+          ceItem = ceItem,
+          productSurface = productSurface
+        )
+
+      UnifiedUserAction(
+        userIdentifier = userIdentifier,
+        item = uuaItem,
+        actionType = actionType,
+        eventMetadata = eventMetaData,
+        productSurface = productSurface,
+        productSurfaceInfo =
+          ProductSurfaceUtils.getProductSurfaceInfo(productSurface, ceItem, logEvent)
+      )
+    }
+
+    uuaOpt match {
+      case Some(uua) => Seq(uua)
+      case _ => Nil
+    }
+  }
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = for {
+    performanceDetails <- logEvent.performanceDetails
+    duration <- performanceDetails.durationMs
+  } yield {
+    Item.UasInfo(UASInfo(timeSpentMs = duration))
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/BaseVideoClientEvent.scala

@@ -0,0 +1,34 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala._
+
+abstract class BaseVideoClientEvent(actionType: ActionType)
+    extends BaseClientEvent(actionType = actionType) {
+
+  override def getUuaItem(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[Item] = for {
+    actionTweetId <- ceItem.id
+    clientMediaEvent <- ceItem.clientMediaEvent
+    sessionState <- clientMediaEvent.sessionState
+    mediaIdentifier <- sessionState.contentVideoIdentifier
+    mediaId <- VideoClientEventUtils.videoIdFromMediaIdentifier(mediaIdentifier)
+    mediaDetails <- ceItem.mediaDetailsV2
+    mediaItems <- mediaDetails.mediaItems
+    videoMetadata <- VideoClientEventUtils.getVideoMetadata(
+      mediaId,
+      mediaItems,
+      ceItem.cardDetails.flatMap(_.amplifyDetails))
+  } yield {
+    Item.TweetInfo(
+      ClientEventCommonUtils
+        .getBasicTweetInfo(
+          actionTweetId = actionTweetId,
+          ceItem = ceItem,
+          ceNamespaceOpt = logEvent.eventNamespace)
+        .copy(tweetActionInfo = Some(videoMetadata)))
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ClientEventAdapter.scala

@@ -0,0 +1,272 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.clientapp.thriftscala.EventNamespace
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.finatra.kafka.serde.UnKeyed
+import com.twitter.unified_user_actions.adapter.AbstractAdapter
+import com.twitter.unified_user_actions.adapter.client_event.ClientEventImpression._
+import com.twitter.unified_user_actions.adapter.client_event.ClientEventEngagement._
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import scala.util.matching.Regex
+
+class ClientEventAdapter extends AbstractAdapter[LogEvent, UnKeyed, UnifiedUserAction] {
+  import ClientEventAdapter._
+
+  override def adaptOneToKeyedMany(
+    input: LogEvent,
+    statsReceiver: StatsReceiver = NullStatsReceiver
+  ): Seq[(UnKeyed, UnifiedUserAction)] =
+    adaptEvent(input).map { e => (UnKeyed, e) }
+}
+
+object ClientEventAdapter {
+  // Refer to go/cme-scribing and go/interaction-event-spec for details
+  def isVideoEvent(element: String): Boolean = Seq[String](
+    "gif_player",
+    "periscope_player",
+    "platform_amplify_card",
+    "video_player",
+    "vine_player").contains(element)
+
+  /**
+   * Tweet clicks on the Notification Tab on iOS are a special case because the `element` is different
+   * from Tweet clicks everywhere else on the platform.
+   *
+   * For Notification Tab on iOS, `element` could be one of `user_mentioned_you`,
+   * `user_mentioned_you_in_a_quote_tweet`, `user_replied_to_your_tweet`, or `user_quoted_your_tweet`.
+   *
+   * In other places, `element` = `tweet`.
+   */
+  def isTweetClickEvent(element: String): Boolean =
+    Seq[String](
+      "tweet",
+      "user_mentioned_you",
+      "user_mentioned_you_in_a_quote_tweet",
+      "user_replied_to_your_tweet",
+      "user_quoted_your_tweet"
+    ).contains(element)
+
+  final val validUASIosClientIds = Seq[Long](
+    129032L, // Twitter for iPhone
+    191841L // Twitter for iPad
+  )
+  // Twitter for Android
+  final val validUASAndroidClientIds = Seq[Long](258901L)
+
+  def adaptEvent(inputLogEvent: LogEvent): Seq[UnifiedUserAction] =
+    Option(inputLogEvent).toSeq
+      .filterNot { logEvent: LogEvent =>
+        shouldIgnoreClientEvent(logEvent.eventNamespace)
+      }
+      .flatMap { logEvent: LogEvent =>
+        val actionTypesPerEvent: Seq[BaseClientEvent] = logEvent.eventNamespace.toSeq.flatMap {
+          name =>
+            (name.page, name.section, name.component, name.element, name.action) match {
+              case (_, _, _, _, Some("favorite")) => Seq(TweetFav)
+              case (_, _, _, _, Some("unfavorite")) => Seq(TweetUnfav)
+              case (_, _, Some("stream"), Some("linger"), Some("results")) =>
+                Seq(TweetLingerImpression)
+              case (_, _, Some("stream"), None, Some("results")) =>
+                Seq(TweetRenderImpression)
+              case (_, _, _, _, Some("send_reply")) => Seq(TweetReply)
+              // Different clients may have different actions of the same "send quote"
+              // but it turns out that both send_quote and retweet_with_comment should correspond to
+              // "send quote"
+              case (_, _, _, _, Some("send_quote_tweet")) |
+                  (_, _, _, _, Some("retweet_with_comment")) =>
+                Seq(TweetQuote)
+              case (_, _, _, _, Some("retweet")) => Seq(TweetRetweet)
+              case (_, _, _, _, Some("unretweet")) => Seq(TweetUnretweet)
+              case (_, _, _, _, Some("reply")) => Seq(TweetClickReply)
+              case (_, _, _, _, Some("quote")) => Seq(TweetClickQuote)
+              case (_, _, _, Some(element), Some("playback_start")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlaybackStart)
+              case (_, _, _, Some(element), Some("playback_complete")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlaybackComplete)
+              case (_, _, _, Some(element), Some("playback_25")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlayback25)
+              case (_, _, _, Some(element), Some("playback_50")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlayback50)
+              case (_, _, _, Some(element), Some("playback_75")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlayback75)
+              case (_, _, _, Some(element), Some("playback_95")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlayback95)
+              case (_, _, _, Some(element), Some("play_from_tap")) if isVideoEvent(element) =>
+                Seq(TweetVideoPlayFromTap)
+              case (_, _, _, Some(element), Some("video_quality_view")) if isVideoEvent(element) =>
+                Seq(TweetVideoQualityView)
+              case (_, _, _, Some(element), Some("video_view")) if isVideoEvent(element) =>
+                Seq(TweetVideoView)
+              case (_, _, _, Some(element), Some("video_mrc_view")) if isVideoEvent(element) =>
+                Seq(TweetVideoMrcView)
+              case (_, _, _, Some(element), Some("view_threshold")) if isVideoEvent(element) =>
+                Seq(TweetVideoViewThreshold)
+              case (_, _, _, Some(element), Some("cta_url_click")) if isVideoEvent(element) =>
+                Seq(TweetVideoCtaUrlClick)
+              case (_, _, _, Some(element), Some("cta_watch_click")) if isVideoEvent(element) =>
+                Seq(TweetVideoCtaWatchClick)
+              case (_, _, _, Some("platform_photo_card"), Some("click")) => Seq(TweetPhotoExpand)
+              case (_, _, _, Some("platform_card"), Some("click")) => Seq(CardClick)
+              case (_, _, _, _, Some("open_app")) => Seq(CardOpenApp)
+              case (_, _, _, _, Some("install_app")) => Seq(CardAppInstallAttempt)
+              case (_, _, _, Some("platform_card"), Some("vote")) |
+                  (_, _, _, Some("platform_forward_card"), Some("vote")) =>
+                Seq(PollCardVote)
+              case (_, _, _, Some("mention"), Some("click")) |
+                  (_, _, _, _, Some("mention_click")) =>
+                Seq(TweetClickMentionScreenName)
+              case (_, _, _, Some(element), Some("click")) if isTweetClickEvent(element) =>
+                Seq(TweetClick)
+              case // Follow from the Topic page (or so-called landing page)
+                  (_, _, _, Some("topic"), Some("follow")) |
+                  // Actually not sure how this is generated ... but saw quite some events in BQ
+                  (_, _, _, Some("social_proof"), Some("follow")) |
+                  // Click on Tweet's caret menu of "Follow (the topic)", it needs to be:
+                  // 1) user follows the Topic already, 2) and clicked on the "Unfollow Topic" first.
+                  (_, _, _, Some("feedback_follow_topic"), Some("click")) =>
+                Seq(TopicFollow)
+              case (_, _, _, Some("topic"), Some("unfollow")) |
+                  (_, _, _, Some("social_proof"), Some("unfollow")) |
+                  (_, _, _, Some("feedback_unfollow_topic"), Some("click")) =>
+                Seq(TopicUnfollow)
+              case (_, _, _, Some("topic"), Some("not_interested")) |
+                  (_, _, _, Some("feedback_not_interested_in_topic"), Some("click")) =>
+                Seq(TopicNotInterestedIn)
+              case (_, _, _, Some("topic"), Some("un_not_interested")) |
+                  (_, _, _, Some("feedback_not_interested_in_topic"), Some("undo")) =>
+                Seq(TopicUndoNotInterestedIn)
+              case (_, _, _, Some("feedback_givefeedback"), Some("click")) =>
+                Seq(TweetNotHelpful)
+              case (_, _, _, Some("feedback_givefeedback"), Some("undo")) =>
+                Seq(TweetUndoNotHelpful)
+              case (_, _, _, Some("report_tweet"), Some("click")) |
+                  (_, _, _, Some("report_tweet"), Some("done")) =>
+                Seq(TweetReport)
+              case (_, _, _, Some("feedback_dontlike"), Some("click")) =>
+                Seq(TweetNotInterestedIn)
+              case (_, _, _, Some("feedback_dontlike"), Some("undo")) =>
+                Seq(TweetUndoNotInterestedIn)
+              case (_, _, _, Some("feedback_notabouttopic"), Some("click")) =>
+                Seq(TweetNotAboutTopic)
+              case (_, _, _, Some("feedback_notabouttopic"), Some("undo")) =>
+                Seq(TweetUndoNotAboutTopic)
+              case (_, _, _, Some("feedback_notrecent"), Some("click")) =>
+                Seq(TweetNotRecent)
+              case (_, _, _, Some("feedback_notrecent"), Some("undo")) =>
+                Seq(TweetUndoNotRecent)
+              case (_, _, _, Some("feedback_seefewer"), Some("click")) =>
+                Seq(TweetSeeFewer)
+              case (_, _, _, Some("feedback_seefewer"), Some("undo")) =>
+                Seq(TweetUndoSeeFewer)
+              // Only when action = "submit" we get all fields in ReportDetails, such as reportType
+              // See https://confluence.twitter.biz/pages/viewpage.action?spaceKey=HEALTH&title=Understanding+ReportDetails
+              case (Some(page), _, _, Some("ticket"), Some("submit"))
+                  if page.startsWith("report_") =>
+                Seq(TweetReportServer)
+              case (Some("profile"), _, _, _, Some("block")) =>
+                Seq(ProfileBlock)
+              case (Some("profile"), _, _, _, Some("unblock")) =>
+                Seq(ProfileUnblock)
+              case (Some("profile"), _, _, _, Some("mute_user")) =>
+                Seq(ProfileMute)
+              case (Some("profile"), _, _, _, Some("report")) =>
+                Seq(ProfileReport)
+              case (Some("profile"), _, _, _, Some("show")) =>
+                Seq(ProfileShow)
+              case (_, _, _, Some("follow"), Some("click")) => Seq(TweetFollowAuthor)
+              case (_, _, _, _, Some("follow")) => Seq(TweetFollowAuthor, ProfileFollow)
+              case (_, _, _, Some("unfollow"), Some("click")) => Seq(TweetUnfollowAuthor)
+              case (_, _, _, _, Some("unfollow")) => Seq(TweetUnfollowAuthor)
+              case (_, _, _, Some("block"), Some("click")) => Seq(TweetBlockAuthor)
+              case (_, _, _, Some("unblock"), Some("click")) => Seq(TweetUnblockAuthor)
+              case (_, _, _, Some("mute"), Some("click")) => Seq(TweetMuteAuthor)
+              case (_, _, _, Some(element), Some("click")) if isTweetClickEvent(element) =>
+                Seq(TweetClick)
+              case (_, _, _, _, Some("profile_click")) => Seq(TweetClickProfile, ProfileClick)
+              case (_, _, _, _, Some("share_menu_click")) => Seq(TweetClickShare)
+              case (_, _, _, _, Some("copy_link")) => Seq(TweetShareViaCopyLink)
+              case (_, _, _, _, Some("share_via_dm")) => Seq(TweetClickSendViaDirectMessage)
+              case (_, _, _, _, Some("bookmark")) => Seq(TweetShareViaBookmark, TweetBookmark)
+              case (_, _, _, _, Some("unbookmark")) => Seq(TweetUnbookmark)
+              case (_, _, _, _, Some("hashtag_click")) |
+                  // This scribe is triggered on mobile platforms (android/iphone) when user click on hashtag in a tweet.
+                  (_, _, _, Some("hashtag"), Some("search")) =>
+                Seq(TweetClickHashtag)
+              case (_, _, _, _, Some("open_link")) => Seq(TweetOpenLink)
+              case (_, _, _, _, Some("take_screenshot")) => Seq(TweetTakeScreenshot)
+              case (_, _, _, Some("feedback_notrelevant"), Some("click")) =>
+                Seq(TweetNotRelevant)
+              case (_, _, _, Some("feedback_notrelevant"), Some("undo")) =>
+                Seq(TweetUndoNotRelevant)
+              case (_, _, _, _, Some("follow_attempt")) => Seq(ProfileFollowAttempt)
+              case (_, _, _, _, Some("favorite_attempt")) => Seq(TweetFavoriteAttempt)
+              case (_, _, _, _, Some("retweet_attempt")) => Seq(TweetRetweetAttempt)
+              case (_, _, _, _, Some("reply_attempt")) => Seq(TweetReplyAttempt)
+              case (_, _, _, _, Some("login")) => Seq(CTALoginClick)
+              case (Some("login"), _, _, _, Some("show")) => Seq(CTALoginStart)
+              case (Some("login"), _, _, _, Some("success")) => Seq(CTALoginSuccess)
+              case (_, _, _, _, Some("signup")) => Seq(CTASignupClick)
+              case (Some("signup"), _, _, _, Some("success")) => Seq(CTASignupSuccess)
+              case // Android app running in the background
+                  (Some("notification"), Some("status_bar"), None, _, Some("background_open")) |
+                  // Android app running in the foreground
+                  (Some("notification"), Some("status_bar"), None, _, Some("open")) |
+                  // iOS app running in the background
+                  (Some("notification"), Some("notification_center"), None, _, Some("open")) |
+                  // iOS app running in the foreground
+                  (None, Some("toasts"), Some("social"), Some("favorite"), Some("open")) |
+                  // m5
+                  (Some("app"), Some("push"), _, _, Some("open")) =>
+                Seq(NotificationOpen)
+              case (Some("ntab"), Some("all"), Some("urt"), _, Some("navigate")) =>
+                Seq(NotificationClick)
+              case (Some("ntab"), Some("all"), Some("urt"), _, Some("see_less_often")) =>
+                Seq(NotificationSeeLessOften)
+              case (Some("notification"), Some("status_bar"), None, _, Some("background_dismiss")) |
+                  (Some("notification"), Some("status_bar"), None, _, Some("dismiss")) | (
+                    Some("notification"),
+                    Some("notification_center"),
+                    None,
+                    _,
+                    Some("dismiss")
+                  ) =>
+                Seq(NotificationDismiss)
+              case (_, _, _, Some("typeahead"), Some("click")) => Seq(TypeaheadClick)
+              case (Some("search"), _, Some(component), _, Some("click"))
+                  if component == "relevance_prompt_module" || component == "did_you_find_it_module" =>
+                Seq(FeedbackPromptSubmit)
+              case (Some("app"), Some("enter_background"), _, _, Some("become_inactive"))
+                  if logEvent.logBase
+                    .flatMap(_.clientAppId)
+                    .exists(validUASIosClientIds.contains(_)) =>
+                Seq(AppExit)
+              case (Some("app"), _, _, _, Some("become_inactive"))
+                  if logEvent.logBase
+                    .flatMap(_.clientAppId)
+                    .exists(validUASAndroidClientIds.contains(_)) =>
+                Seq(AppExit)
+              case (_, _, Some("gallery"), Some("photo"), Some("impression")) =>
+                Seq(TweetGalleryImpression)
+              case (_, _, _, _, _)
+                  if TweetDetailsImpression.isTweetDetailsImpression(logEvent.eventNamespace) =>
+                Seq(TweetDetailsImpression)
+              case _ => Nil
+            }
+        }
+        actionTypesPerEvent.map(_.toUnifiedUserAction(logEvent))
+      }.flatten
+
+  def shouldIgnoreClientEvent(eventNamespace: Option[EventNamespace]): Boolean =
+    eventNamespace.exists { name =>
+      (name.page, name.section, name.component, name.element, name.action) match {
+        case (Some("ddg"), _, _, _, Some("experiment")) => true
+        case (Some("qig_ranker"), _, _, _, _) => true
+        case (Some("timelinemixer"), _, _, _, _) => true
+        case (Some("timelineservice"), _, _, _, _) => true
+        case (Some("tweetconvosvc"), _, _, _, _) => true
+        case _ => false
+      }
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ClientEventCommonUtils.scala

@@ -0,0 +1,169 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.EventNamespace
+import com.twitter.clientapp.thriftscala.Item
+import com.twitter.clientapp.thriftscala.ItemType.User
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.adapter.common.AdapterUtils
+import com.twitter.unified_user_actions.thriftscala.AuthorInfo
+import com.twitter.unified_user_actions.thriftscala.ClientEventNamespace
+import com.twitter.unified_user_actions.thriftscala.EventMetadata
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.SourceLineage
+import com.twitter.unified_user_actions.thriftscala.TweetAuthorFollowClickSource
+import com.twitter.unified_user_actions.thriftscala.TweetAuthorUnfollowClickSource
+import com.twitter.unified_user_actions.thriftscala.TweetInfo
+
+/**
+ * Comprises helper methods that:
+ * 1. need not be overridden by subclasses of `BaseClientEvent`
+ * 2. need not be invoked by instances of subclasses of `BaseClientEvent`
+ * 3. need to be accessible to subclasses of `BaseClientEvent` and other utils
+ */
+object ClientEventCommonUtils {
+
+  def getBasicTweetInfo(
+    actionTweetId: Long,
+    ceItem: LogEventItem,
+    ceNamespaceOpt: Option[EventNamespace]
+  ): TweetInfo = TweetInfo(
+    actionTweetId = actionTweetId,
+    actionTweetTopicSocialProofId = getTopicId(ceItem, ceNamespaceOpt),
+    retweetingTweetId = ceItem.tweetDetails.flatMap(_.retweetingTweetId),
+    quotedTweetId = ceItem.tweetDetails.flatMap(_.quotedTweetId),
+    inReplyToTweetId = ceItem.tweetDetails.flatMap(_.inReplyToTweetId),
+    quotingTweetId = ceItem.tweetDetails.flatMap(_.quotingTweetId),
+    // only set AuthorInfo when authorId is present
+    actionTweetAuthorInfo = getAuthorInfo(ceItem),
+    retweetingAuthorId = ceItem.tweetDetails.flatMap(_.retweetAuthorId),
+    quotedAuthorId = ceItem.tweetDetails.flatMap(_.quotedAuthorId),
+    inReplyToAuthorId = ceItem.tweetDetails.flatMap(_.inReplyToAuthorId),
+    tweetPosition = ceItem.position,
+    promotedId = ceItem.promotedId
+  )
+
+  def getTopicId(
+    ceItem: LogEventItem,
+    ceNamespaceOpt: Option[EventNamespace] = None,
+  ): Option[Long] =
+    ceNamespaceOpt.flatMap {
+      TopicIdUtils.getTopicId(item = ceItem, _)
+    }
+
+  def getAuthorInfo(
+    ceItem: LogEventItem,
+  ): Option[AuthorInfo] =
+    ceItem.tweetDetails.flatMap(_.authorId).map { authorId =>
+      AuthorInfo(
+        authorId = Some(authorId),
+        isFollowedByActingUser = ceItem.isViewerFollowsTweetAuthor,
+        isFollowingActingUser = ceItem.isTweetAuthorFollowsViewer,
+      )
+    }
+
+  def getEventMetadata(
+    eventTimestamp: Long,
+    logEvent: LogEvent,
+    ceItem: LogEventItem,
+    productSurface: Option[ProductSurface] = None
+  ): EventMetadata = EventMetadata(
+    sourceTimestampMs = eventTimestamp,
+    receivedTimestampMs = AdapterUtils.currentTimestampMs,
+    sourceLineage = SourceLineage.ClientEvents,
+    // Client UI language or from Gizmoduck which is what user set in Twitter App.
+    // Please see more at https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/finatra-internal/international/src/main/scala/com/twitter/finatra/international/LanguageIdentifier.scala
+    // The format should be ISO 639-1.
+    language = logEvent.logBase.flatMap(_.language).map(AdapterUtils.normalizeLanguageCode),
+    // Country code could be IP address (geoduck) or User registration country (gizmoduck) and the former takes precedence.
+    // We don’t know exactly which one is applied, unfortunately,
+    // see https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/finatra-internal/international/src/main/scala/com/twitter/finatra/international/CountryIdentifier.scala
+    // The format should be ISO_3166-1_alpha-2.
+    countryCode = logEvent.logBase.flatMap(_.country).map(AdapterUtils.normalizeCountryCode),
+    clientAppId = logEvent.logBase.flatMap(_.clientAppId),
+    clientVersion = logEvent.clientVersion,
+    clientEventNamespace = logEvent.eventNamespace.map(en => toClientEventNamespace(en)),
+    traceId = getTraceId(productSurface, ceItem),
+    requestJoinId = getRequestJoinId(productSurface, ceItem),
+    clientEventTriggeredOn = logEvent.eventDetails.flatMap(_.triggeredOn)
+  )
+
+  def toClientEventNamespace(eventNamespace: EventNamespace): ClientEventNamespace =
+    ClientEventNamespace(
+      page = eventNamespace.page,
+      section = eventNamespace.section,
+      component = eventNamespace.component,
+      element = eventNamespace.element,
+      action = eventNamespace.action
+    )
+
+  /**
+   * Get the profileId from Item.id, which itemType = 'USER'.
+   *
+   * The profileId can be also be found in the event_details.profile_id.
+   * However, the item.id is more reliable than event_details.profile_id,
+   * in particular, 45% of the client events with USER items have
+   * Null for event_details.profile_id while 0.13% item.id is Null.
+   * As such, we only use item.id to populate the profile_id.
+   */
+  def getProfileIdFromUserItem(item: Item): Option[Long] =
+    if (item.itemType.contains(User))
+      item.id
+    else None
+
+  /**
+   * TraceId is going to be deprecated and replaced by requestJoinId.
+   *
+   * Get the traceId from LogEventItem based on productSurface.
+   *
+   * The traceId is hydrated in controller data from backend. Different product surfaces
+   * populate different controller data. Thus, the product surface is checked first to decide
+   * which controller data should be read to ge the requestJoinId.
+   */
+  def getTraceId(productSurface: Option[ProductSurface], ceItem: LogEventItem): Option[Long] =
+    productSurface match {
+      case Some(ProductSurface.HomeTimeline) => HomeInfoUtils.getTraceId(ceItem)
+      case Some(ProductSurface.SearchResultsPage) => { new SearchInfoUtils(ceItem) }.getTraceId
+      case _ => None
+    }
+
+  /**
+   * Get the requestJoinId from LogEventItem based on productSurface.
+   *
+   * The requestJoinId is hydrated in controller data from backend. Different product surfaces
+   * populate different controller data. Thus, the product surface is checked first to decide
+   * which controller data should be read to get the requestJoinId.
+   *
+   * Support Home / Home_latest / SearchResults for now, to add other surfaces based on requirement.
+   */
+  def getRequestJoinId(productSurface: Option[ProductSurface], ceItem: LogEventItem): Option[Long] =
+    productSurface match {
+      case Some(ProductSurface.HomeTimeline) => HomeInfoUtils.getRequestJoinId(ceItem)
+      case Some(ProductSurface.SearchResultsPage) => {
+          new SearchInfoUtils(ceItem)
+        }.getRequestJoinId
+      case _ => None
+    }
+
+  def getTweetAuthorFollowSource(
+    eventNamespace: Option[EventNamespace]
+  ): TweetAuthorFollowClickSource = {
+    eventNamespace
+      .map(ns => (ns.element, ns.action)).map {
+        case (Some("follow"), Some("click")) => TweetAuthorFollowClickSource.CaretMenu
+        case (_, Some("follow")) => TweetAuthorFollowClickSource.ProfileImage
+        case _ => TweetAuthorFollowClickSource.Unknown
+      }.getOrElse(TweetAuthorFollowClickSource.Unknown)
+  }
+
+  def getTweetAuthorUnfollowSource(
+    eventNamespace: Option[EventNamespace]
+  ): TweetAuthorUnfollowClickSource = {
+    eventNamespace
+      .map(ns => (ns.element, ns.action)).map {
+        case (Some("unfollow"), Some("click")) => TweetAuthorUnfollowClickSource.CaretMenu
+        case (_, Some("unfollow")) => TweetAuthorUnfollowClickSource.ProfileImage
+        case _ => TweetAuthorUnfollowClickSource.Unknown
+      }.getOrElse(TweetAuthorUnfollowClickSource.Unknown)
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ClientEventEngagement.scala

@@ -0,0 +1,687 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.unified_user_actions.thriftscala._
+
+object ClientEventEngagement {
+  object TweetFav extends BaseClientEvent(ActionType.ClientTweetFav)
+
+  /**
+   * This is fired when a user unlikes a liked(favorited) tweet
+   */
+  object TweetUnfav extends BaseClientEvent(ActionType.ClientTweetUnfav)
+
+  /**
+   * This is "Send Reply" event to indicate publishing of a reply Tweet as opposed to clicking
+   * on the reply button to initiate a reply Tweet (captured in ClientTweetClickReply).
+   * The difference between this and the ServerTweetReply are:
+   * 1) ServerTweetReply already has the new Tweet Id, 2) A sent reply may be lost during transfer
+   * over the wire and thus may not end up with a follow-up ServerTweetReply.
+   */
+  object TweetReply extends BaseClientEvent(ActionType.ClientTweetReply)
+
+  /**
+   * This is the "send quote" event to indicate publishing of a quote tweet as opposed to clicking
+   * on the quote button to initiate a quote tweet (captured in ClientTweetClickQuote).
+   * The difference between this and the ServerTweetQuote are:
+   * 1) ServerTweetQuote already has the new Tweet Id, 2) A sent quote may be lost during transfer
+   * over the wire and thus may not ended up with a follow-up ServerTweetQuote.
+   */
+  object TweetQuote extends BaseClientEvent(ActionType.ClientTweetQuote)
+
+  /**
+   * This is the "retweet" event to indicate publishing of a retweet.
+   */
+  object TweetRetweet extends BaseClientEvent(ActionType.ClientTweetRetweet)
+
+  /**
+   * "action = reply" indicates that a user expressed the intention to reply to a Tweet by clicking
+   * the reply button. No new tweet is created in this event.
+   */
+  object TweetClickReply extends BaseClientEvent(ActionType.ClientTweetClickReply)
+
+  /**
+   * Please note that the "action == quote" is NOT the create quote Tweet event like what
+   * we can get from TweetyPie.
+   * It is just click on the "quote tweet" (after clicking on the retweet button there are 2 options,
+   * one is "retweet" and the other is "quote tweet")
+   *
+   * Also checked the CE (BQ Table), the `item.tweet_details.quoting_tweet_id` is always NULL but
+   * `item.tweet_details.retweeting_tweet_id`, `item.tweet_details.in_reply_to_tweet_id`, `item.tweet_details.quoted_tweet_id`
+   * could be NON-NULL and UUA would just include these NON-NULL fields as is. This is also checked in the unit test.
+   */
+  object TweetClickQuote extends BaseClientEvent(ActionType.ClientTweetClickQuote)
+
+  /**
+   * Refer to go/cme-scribing and go/interaction-event-spec for details.
+   * Fired on the first tick of a track regardless of where in the video it is playing.
+   * For looping playback, this is only fired once and does not reset at loop boundaries.
+   */
+  object TweetVideoPlaybackStart
+      extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlaybackStart)
+
+  /**
+   * Refer to go/cme-scribing and go/interaction-event-spec for details.
+   * Fired when playback reaches 100% of total track duration.
+   * Not valid for live videos.
+   * For looping playback, this is only fired once and does not reset at loop boundaries.
+   */
+  object TweetVideoPlaybackComplete
+      extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlaybackComplete)
+
+  /**
+   * Refer to go/cme-scribing and go/interaction-event-spec for details.
+   * This is fired when playback reaches 25% of total track duration. Not valid for live videos.
+   * For looping playback, this is only fired once and does not reset at loop boundaries.
+   */
+  object TweetVideoPlayback25 extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlayback25)
+  object TweetVideoPlayback50 extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlayback50)
+  object TweetVideoPlayback75 extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlayback75)
+  object TweetVideoPlayback95 extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlayback95)
+
+  /**
+   * Refer to go/cme-scribing and go/interaction-event-spec for details.
+   * This if fired when the video has been played in non-preview
+   * (i.e. not autoplaying in the timeline) mode, and was not started via auto-advance.
+   * For looping playback, this is only fired once and does not reset at loop boundaries.
+   */
+  object TweetVideoPlayFromTap extends BaseVideoClientEvent(ActionType.ClientTweetVideoPlayFromTap)
+
+  /**
+   * Refer to go/cme-scribing and go/interaction-event-spec for details.
+   * This is fired when 50% of the video has been on-screen and playing for 10 consecutive seconds
+   * or 95% of the video duration, whichever comes first.
+   * For looping playback, this is only fired once and does not reset at loop boundaries.
+   */
+  object TweetVideoQualityView extends BaseVideoClientEvent(ActionType.ClientTweetVideoQualityView)
+
+  object TweetVideoView extends BaseVideoClientEvent(ActionType.ClientTweetVideoView)
+  object TweetVideoMrcView extends BaseVideoClientEvent(ActionType.ClientTweetVideoMrcView)
+  object TweetVideoViewThreshold
+      extends BaseVideoClientEvent(ActionType.ClientTweetVideoViewThreshold)
+  object TweetVideoCtaUrlClick extends BaseVideoClientEvent(ActionType.ClientTweetVideoCtaUrlClick)
+  object TweetVideoCtaWatchClick
+      extends BaseVideoClientEvent(ActionType.ClientTweetVideoCtaWatchClick)
+
+  /**
+   * This is fired when a user clicks on "Undo retweet" after re-tweeting a tweet
+   *
+   */
+  object TweetUnretweet extends BaseClientEvent(ActionType.ClientTweetUnretweet)
+
+  /**
+   * This is fired when a user clicks on a photo attached to a tweet and the photo expands to fit
+   * the screen.
+   */
+  object TweetPhotoExpand extends BaseClientEvent(ActionType.ClientTweetPhotoExpand)
+
+  /**
+   * This is fired when a user clicks on a card, a card could be a photo or video for example
+   */
+  object CardClick extends BaseCardClientEvent(ActionType.ClientCardClick)
+  object CardOpenApp extends BaseCardClientEvent(ActionType.ClientCardOpenApp)
+  object CardAppInstallAttempt extends BaseCardClientEvent(ActionType.ClientCardAppInstallAttempt)
+  object PollCardVote extends BaseCardClientEvent(ActionType.ClientPollCardVote)
+
+  /**
+   * This is fired when a user clicks on a profile mention inside a tweet.
+   */
+  object TweetClickMentionScreenName
+      extends BaseClientEvent(ActionType.ClientTweetClickMentionScreenName) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      (
+        ceItem.id,
+        logEvent.eventDetails.flatMap(
+          _.targets.flatMap(_.find(_.itemType.contains(ItemType.User))))) match {
+        case (Some(tweetId), Some(target)) =>
+          (target.id, target.name) match {
+            case (Some(profileId), Some(profileHandle)) =>
+              Some(
+                Item.TweetInfo(
+                  ClientEventCommonUtils
+                    .getBasicTweetInfo(tweetId, ceItem, logEvent.eventNamespace)
+                    .copy(tweetActionInfo = Some(
+                      TweetActionInfo.ClientTweetClickMentionScreenName(
+                        ClientTweetClickMentionScreenName(
+                          actionProfileId = profileId,
+                          handle = profileHandle
+                        ))))))
+            case _ => None
+          }
+        case _ => None
+      }
+  }
+
+  /**
+   * These are fired when user follows/unfollows a Topic. Please see the comment in the
+   * ClientEventAdapter namespace matching to see the subtle details.
+   */
+  object TopicFollow extends BaseTopicClientEvent(ActionType.ClientTopicFollow)
+  object TopicUnfollow extends BaseTopicClientEvent(ActionType.ClientTopicUnfollow)
+
+  /**
+   * This is fired when the user clicks the "x" icon next to the topic on their timeline,
+   * and clicks "Not interested in {TOPIC}" in the pop-up prompt
+   * Alternatively, they can also click "See more" button to visit the topic page, and click "Not interested" there.
+   */
+  object TopicNotInterestedIn extends BaseTopicClientEvent(ActionType.ClientTopicNotInterestedIn)
+
+  /**
+   * This is fired when the user clicks the "Undo" button after clicking "x" or "Not interested" on a Topic
+   * which is captured in ClientTopicNotInterestedIn
+   */
+  object TopicUndoNotInterestedIn
+      extends BaseTopicClientEvent(ActionType.ClientTopicUndoNotInterestedIn)
+
+  /**
+   * This is fired when a user clicks on  "This Tweet's not helpful" flow in the caret menu
+   * of a Tweet result on the Search Results Page
+   */
+  object TweetNotHelpful extends BaseClientEvent(ActionType.ClientTweetNotHelpful)
+
+  /**
+   * This is fired when a user clicks Undo after clicking on
+   * "This Tweet's not helpful" flow in the caret menu of a Tweet result on the Search Results Page
+   */
+  object TweetUndoNotHelpful extends BaseClientEvent(ActionType.ClientTweetUndoNotHelpful)
+
+  object TweetReport extends BaseClientEvent(ActionType.ClientTweetReport) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] = {
+      for {
+        actionTweetId <- ceItem.id
+      } yield {
+        Item.TweetInfo(
+          ClientEventCommonUtils
+            .getBasicTweetInfo(
+              actionTweetId = actionTweetId,
+              ceItem = ceItem,
+              ceNamespaceOpt = logEvent.eventNamespace)
+            .copy(tweetActionInfo = Some(
+              TweetActionInfo.ClientTweetReport(
+                ClientTweetReport(
+                  isReportTweetDone =
+                    logEvent.eventNamespace.flatMap(_.action).exists(_.contains("done")),
+                  reportFlowId = logEvent.reportDetails.flatMap(_.reportFlowId)
+                )
+              ))))
+      }
+    }
+  }
+
+  /**
+   * Not Interested In (Do Not like) event
+   */
+  object TweetNotInterestedIn extends BaseClientEvent(ActionType.ClientTweetNotInterestedIn)
+  object TweetUndoNotInterestedIn extends BaseClientEvent(ActionType.ClientTweetUndoNotInterestedIn)
+
+  /**
+   * This is fired when a user FIRST clicks the "Not interested in this Tweet" button in the caret menu of a Tweet
+   * then clicks "This Tweet is not about {TOPIC}" in the subsequent prompt
+   * Note: this button is hidden unless a user clicks "Not interested in this Tweet" first.
+   */
+  object TweetNotAboutTopic extends BaseClientEvent(ActionType.ClientTweetNotAboutTopic)
+
+  /**
+   * This is fired when a user clicks "Undo" immediately after clicking "This Tweet is not about {TOPIC}",
+   * which is captured in TweetNotAboutTopic
+   */
+  object TweetUndoNotAboutTopic extends BaseClientEvent(ActionType.ClientTweetUndoNotAboutTopic)
+
+  /**
+   * This is fired when a user FIRST clicks the "Not interested in this Tweet" button in the caret menu of a Tweet
+   * then clicks  "This Tweet isn't recent" in the subsequent prompt
+   * Note: this button is hidden unless a user clicks "Not interested in this Tweet" first.
+   */
+  object TweetNotRecent extends BaseClientEvent(ActionType.ClientTweetNotRecent)
+
+  /**
+   * This is fired when a user clicks "Undo" immediately after clicking "his Tweet isn't recent",
+   * which is captured in TweetNotRecent
+   */
+  object TweetUndoNotRecent extends BaseClientEvent(ActionType.ClientTweetUndoNotRecent)
+
+  /**
+   * This is fired when a user clicks "Not interested in this Tweet" button in the caret menu of a Tweet
+   * then clicks  "Show fewer tweets from" in the subsequent prompt
+   * Note: this button is hidden unless a user clicks "Not interested in this Tweet" first.
+   */
+  object TweetSeeFewer extends BaseClientEvent(ActionType.ClientTweetSeeFewer)
+
+  /**
+   * This is fired when a user clicks "Undo" immediately after clicking "Show fewer tweets from",
+   * which is captured in TweetSeeFewer
+   */
+  object TweetUndoSeeFewer extends BaseClientEvent(ActionType.ClientTweetUndoSeeFewer)
+
+  /**
+   * This is fired when a user click "Submit" at the end of a "Report Tweet" flow
+   * ClientTweetReport = 1041 is scribed by HealthClient team, on the client side
+   * This is scribed by spamacaw, on the server side
+   * They can be joined on reportFlowId
+   * See https://confluence.twitter.biz/pages/viewpage.action?spaceKey=HEALTH&title=Understanding+ReportDetails
+   */
+  object TweetReportServer extends BaseClientEvent(ActionType.ServerTweetReport) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      for {
+        actionTweetId <- ceItem.id
+      } yield Item.TweetInfo(
+        ClientEventCommonUtils
+          .getBasicTweetInfo(
+            actionTweetId = actionTweetId,
+            ceItem = ceItem,
+            ceNamespaceOpt = logEvent.eventNamespace)
+          .copy(tweetActionInfo = Some(
+            TweetActionInfo.ServerTweetReport(
+              ServerTweetReport(
+                reportFlowId = logEvent.reportDetails.flatMap(_.reportFlowId),
+                reportType = logEvent.reportDetails.flatMap(_.reportType)
+              )
+            ))))
+  }
+
+  /**
+   * This is fired when a user clicks Block in a Profile page
+   * A Profile can also be blocked when a user clicks Block in the menu of a Tweet, which
+   * is captured in ClientTweetBlockAuthor
+   */
+  object ProfileBlock extends BaseProfileClientEvent(ActionType.ClientProfileBlock)
+
+  /**
+   * This is fired when a user clicks unblock in a pop-up prompt right after blocking a profile
+   * in the profile page or clicks unblock in a drop-down menu in the profile page.
+   */
+  object ProfileUnblock extends BaseProfileClientEvent(ActionType.ClientProfileUnblock)
+
+  /**
+   * This is fired when a user clicks Mute in a Profile page
+   * A Profile can also be muted when a user clicks Mute in the menu of a Tweet, which
+   * is captured in ClientTweetMuteAuthor
+   */
+  object ProfileMute extends BaseProfileClientEvent(ActionType.ClientProfileMute)
+
+  /*
+   * This is fired when a user clicks "Report User" action from user profile page
+   * */
+  object ProfileReport extends BaseProfileClientEvent(ActionType.ClientProfileReport)
+
+  // This is fired when a user profile is open in a Profile page
+  object ProfileShow extends BaseProfileClientEvent(ActionType.ClientProfileShow)
+
+  object ProfileClick extends BaseProfileClientEvent(ActionType.ClientProfileClick) {
+
+    /**
+     * ClientTweetClickProfile would emit 2 events, 1 with item type Tweet and 1 with item type User
+     * Both events will go to both actions (the actual classes). For ClientTweetClickProfile,
+     * item type of Tweet will filter out the event with item type User. But for ClientProfileClick,
+     * because we need to include item type of User, then we will also include the event of TweetClickProfile
+     * if we don't do anything here. This override ensures we don't include tweet author clicks events in ProfileClick
+     */
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      if (logEvent.eventDetails
+          .flatMap(_.items).exists(items => items.exists(_.itemType.contains(ItemType.Tweet)))) {
+        None
+      } else {
+        super.getUuaItem(ceItem, logEvent)
+      }
+  }
+
+  /**
+   * This is fired when a user follows a profile from the
+   * profile page / people module and people tab on the Search Results Page / sidebar on the Home page
+   * A Profile can also be followed when a user clicks follow in the
+   * caret menu of a Tweet / follow button on hovering on profile avatar,
+   * which is captured in ClientTweetFollowAuthor
+   */
+  object ProfileFollow extends BaseProfileClientEvent(ActionType.ClientProfileFollow) {
+
+    /**
+     * ClientTweetFollowAuthor would emit 2 events, 1 with item type Tweet and 1 with item type User
+     *  Both events will go to both actions (the actual classes). For ClientTweetFollowAuthor,
+     *  item type of Tweet will filter out the event with item type User. But for ClientProfileFollow,
+     *  because we need to include item type of User, then we will also include the event of TweetFollowAuthor
+     *  if we don't do anything here. This override ensures we don't include tweet author follow events in ProfileFollow
+     */
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      if (logEvent.eventDetails
+          .flatMap(_.items).exists(items => items.exists(_.itemType.contains(ItemType.Tweet)))) {
+        None
+      } else {
+        super.getUuaItem(ceItem, logEvent)
+      }
+  }
+
+  /**
+   * This is fired when a user clicks Follow in the caret menu of a Tweet or hovers on the avatar of the tweet author
+   * and clicks on the Follow button. A profile can also be followed by clicking the Follow button on the Profile
+   * page and confirm, which is captured in ClientProfileFollow.
+   * The event emits two items, one of user type and another of tweet type, since the default implementation of
+   * BaseClientEvent only looks for Tweet type, the other item is dropped which is the expected behaviour
+   */
+  object TweetFollowAuthor extends BaseClientEvent(ActionType.ClientTweetFollowAuthor) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] = {
+      for {
+        actionTweetId <- ceItem.id
+      } yield {
+        Item.TweetInfo(
+          ClientEventCommonUtils
+            .getBasicTweetInfo(
+              actionTweetId = actionTweetId,
+              ceItem = ceItem,
+              ceNamespaceOpt = logEvent.eventNamespace)
+            .copy(tweetActionInfo = Some(
+              TweetActionInfo.ClientTweetFollowAuthor(
+                ClientTweetFollowAuthor(
+                  ClientEventCommonUtils.getTweetAuthorFollowSource(logEvent.eventNamespace))
+              ))))
+      }
+    }
+  }
+
+  /**
+   * This is fired when a user clicks Unfollow in the caret menu of a Tweet or hovers on the avatar of the tweet author
+   * and clicks on the Unfollow button. A profile can also be unfollowed by clicking the Unfollow button on the Profile
+   * page and confirm, which will be captured in ClientProfileUnfollow.
+   * The event emits two items, one of user type and another of tweet type, since the default implementation of
+   * BaseClientEvent only looks for Tweet type, the other item is dropped which is the expected behaviour
+   */
+  object TweetUnfollowAuthor extends BaseClientEvent(ActionType.ClientTweetUnfollowAuthor) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] = {
+      for {
+        actionTweetId <- ceItem.id
+      } yield {
+        Item.TweetInfo(
+          ClientEventCommonUtils
+            .getBasicTweetInfo(
+              actionTweetId = actionTweetId,
+              ceItem = ceItem,
+              ceNamespaceOpt = logEvent.eventNamespace)
+            .copy(tweetActionInfo = Some(
+              TweetActionInfo.ClientTweetUnfollowAuthor(
+                ClientTweetUnfollowAuthor(
+                  ClientEventCommonUtils.getTweetAuthorUnfollowSource(logEvent.eventNamespace))
+              ))))
+      }
+    }
+  }
+
+  /**
+   * This is fired when a user clicks Block in the caret menu of a Tweet to block the profile
+   * that authors this Tweet. A profile can also be blocked in the Profile page, which is captured
+   * in ClientProfileBlock
+   */
+  object TweetBlockAuthor extends BaseClientEvent(ActionType.ClientTweetBlockAuthor)
+
+  /**
+   * This is fired when a user clicks unblock in a pop-up prompt right after blocking an author
+   * in the drop-down menu of a tweet
+   */
+  object TweetUnblockAuthor extends BaseClientEvent(ActionType.ClientTweetUnblockAuthor)
+
+  /**
+   * This is fired when a user clicks Mute in the caret menu of a Tweet to mute the profile
+   * that authors this Tweet. A profile can also be muted in the Profile page, which is captured
+   * in ClientProfileMute
+   */
+  object TweetMuteAuthor extends BaseClientEvent(ActionType.ClientTweetMuteAuthor)
+
+  /**
+   * This is fired when a user clicks on a Tweet to open the Tweet details page. Note that for
+   * Tweets in the Notification Tab product surface, a click can be registered differently
+   * depending on whether the Tweet is a rendered Tweet (a click results in ClientTweetClick)
+   * or a wrapper Notification (a click results in ClientNotificationClick).
+   */
+  object TweetClick extends BaseClientEvent(ActionType.ClientTweetClick)
+
+  /**
+   * This is fired when a user clicks to view the profile page of another user from a Tweet
+   */
+  object TweetClickProfile extends BaseClientEvent(ActionType.ClientTweetClickProfile)
+
+  /**
+   * This is fired when a user clicks on the "share" icon on a Tweet to open the share menu.
+   * The user may or may not proceed and finish sharing the Tweet.
+   */
+  object TweetClickShare extends BaseClientEvent(ActionType.ClientTweetClickShare)
+
+  /**
+   * This is fired when a user clicks "Copy link to Tweet" in a menu appeared after hitting
+   * the "share" icon on a Tweet OR when a user selects share_via -> copy_link after long-click
+   * a link inside a tweet on a mobile device
+   */
+  object TweetShareViaCopyLink extends BaseClientEvent(ActionType.ClientTweetShareViaCopyLink)
+
+  /**
+   * This is fired when a user clicks "Send via Direct Message" after
+   * clicking on the "share" icon on a Tweet to open the share menu.
+   * The user may or may not proceed and finish Sending the DM.
+   */
+  object TweetClickSendViaDirectMessage
+      extends BaseClientEvent(ActionType.ClientTweetClickSendViaDirectMessage)
+
+  /**
+   * This is fired when a user clicks "Bookmark" after
+   * clicking on the "share" icon on a Tweet to open the share menu.
+   */
+  object TweetShareViaBookmark extends BaseClientEvent(ActionType.ClientTweetShareViaBookmark)
+
+  /**
+   * This is fired when a user clicks "Remove Tweet from Bookmarks" after
+   * clicking on the "share" icon on a Tweet to open the share menu.
+   */
+  object TweetUnbookmark extends BaseClientEvent(ActionType.ClientTweetUnbookmark)
+
+  /**
+   * This event is fired when the user clicks on a hashtag in a Tweet.
+   */
+  object TweetClickHashtag extends BaseClientEvent(ActionType.ClientTweetClickHashtag) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] = for {
+      actionTweetId <- ceItem.id
+    } yield Item.TweetInfo(
+      ClientEventCommonUtils
+        .getBasicTweetInfo(
+          actionTweetId = actionTweetId,
+          ceItem = ceItem,
+          ceNamespaceOpt = logEvent.eventNamespace)
+        .copy(tweetActionInfo = logEvent.eventDetails
+          .map(
+            _.targets.flatMap(_.headOption.flatMap(_.name))
+          ) // fetch the first item in the details and then the name will have the hashtag value with the '#' sign
+          .map { hashtagOpt =>
+            TweetActionInfo.ClientTweetClickHashtag(
+              ClientTweetClickHashtag(hashtag = hashtagOpt)
+            )
+          }))
+  }
+
+  /**
+   * This is fired when a user clicks "Bookmark" after clicking on the "share" icon on a Tweet to
+   * open the share menu, or when a user clicks on the 'bookmark' icon on a Tweet (bookmark icon
+   * is available to ios only as of March 2023).
+   * TweetBookmark and TweetShareByBookmark log the same events but serve for individual use cases.
+   */
+  object TweetBookmark extends BaseClientEvent(ActionType.ClientTweetBookmark)
+
+  /**
+   * This is fired when a user clicks on a link in a tweet.
+   * The link could be displayed as a URL or embedded
+   * in a component such as an image or a card in a tweet.
+   */
+  object TweetOpenLink extends BaseClientEvent(ActionType.ClientTweetOpenLink) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      for {
+        actionTweetId <- ceItem.id
+      } yield Item.TweetInfo(
+        ClientEventCommonUtils
+          .getBasicTweetInfo(
+            actionTweetId = actionTweetId,
+            ceItem = ceItem,
+            ceNamespaceOpt = logEvent.eventNamespace)
+          .copy(tweetActionInfo = Some(
+            TweetActionInfo.ClientTweetOpenLink(
+              ClientTweetOpenLink(url = logEvent.eventDetails.flatMap(_.url))
+            ))))
+  }
+
+  /**
+   * This is fired when a user takes a screenshot.
+   * This is available for only mobile clients.
+   */
+  object TweetTakeScreenshot extends BaseClientEvent(ActionType.ClientTweetTakeScreenshot) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] =
+      for {
+        actionTweetId <- ceItem.id
+      } yield Item.TweetInfo(
+        ClientEventCommonUtils
+          .getBasicTweetInfo(
+            actionTweetId = actionTweetId,
+            ceItem = ceItem,
+            ceNamespaceOpt = logEvent.eventNamespace)
+          .copy(tweetActionInfo = Some(
+            TweetActionInfo.ClientTweetTakeScreenshot(
+              ClientTweetTakeScreenshot(percentVisibleHeight100k = ceItem.percentVisibleHeight100k)
+            ))))
+  }
+
+  /**
+   * This is fired when a user clicks the "This Tweet isn't relevant" button in a prompt displayed
+   * after clicking "This Tweet's not helpful" in search result page or "Not Interested in this Tweet"
+   * in the home timeline page.
+   * Note: this button is hidden unless a user clicks "This Tweet isn't relevant" or
+   * "This Tweet's not helpful" first
+   */
+  object TweetNotRelevant extends BaseClientEvent(ActionType.ClientTweetNotRelevant)
+
+  /**
+   * This is fired when a user clicks "Undo" immediately after clicking "this Tweet isn't relevant",
+   * which is captured in TweetNotRelevant
+   */
+  object TweetUndoNotRelevant extends BaseClientEvent(ActionType.ClientTweetUndoNotRelevant)
+
+  /**
+   * This is fired when a user is logged out and follows a profile from the
+   * profile page / people module from web.
+   * One can only try to follow from web, iOS and Android do not support logged out browsing
+   */
+  object ProfileFollowAttempt extends BaseProfileClientEvent(ActionType.ClientProfileFollowAttempt)
+
+  /**
+   * This is fired when a user is logged out and favourite a tweet from web.
+   * One can only try to favourite from web, iOS and Android do not support logged out browsing
+   */
+  object TweetFavoriteAttempt extends BaseClientEvent(ActionType.ClientTweetFavoriteAttempt)
+
+  /**
+   * This is fired when a user is logged out and Retweet a tweet from web.
+   * One can only try to favourite from web, iOS and Android do not support logged out browsing
+   */
+  object TweetRetweetAttempt extends BaseClientEvent(ActionType.ClientTweetRetweetAttempt)
+
+  /**
+   * This is fired when a user is logged out and reply on tweet from web.
+   * One can only try to favourite from web, iOS and Android do not support logged out browsing
+   */
+  object TweetReplyAttempt extends BaseClientEvent(ActionType.ClientTweetReplyAttempt)
+
+  /**
+   * This is fired when a user is logged out and clicks on login button.
+   * Currently seem to be generated only on [m5, LiteNativeWrapper] as of Jan 2023.
+   */
+  object CTALoginClick extends BaseCTAClientEvent(ActionType.ClientCTALoginClick)
+
+  /**
+   * This is fired when a user is logged out and login window is shown.
+   */
+  object CTALoginStart extends BaseCTAClientEvent(ActionType.ClientCTALoginStart)
+
+  /**
+   * This is fired when a user is logged out and login is successful.
+   */
+  object CTALoginSuccess extends BaseCTAClientEvent(ActionType.ClientCTALoginSuccess)
+
+  /**
+   * This is fired when a user is logged out and clicks on signup button.
+   */
+  object CTASignupClick extends BaseCTAClientEvent(ActionType.ClientCTASignupClick)
+
+  /**
+   * This is fired when a user is logged out and signup is successful.
+   */
+  object CTASignupSuccess extends BaseCTAClientEvent(ActionType.ClientCTASignupSuccess)
+
+  /**
+   * This is fired when a user opens a Push Notification.
+   * Refer to https://confluence.twitter.biz/pages/viewpage.action?pageId=161811800
+   * for Push Notification scribe details
+   */
+  object NotificationOpen extends BasePushNotificationClientEvent(ActionType.ClientNotificationOpen)
+
+  /**
+   * This is fired when a user clicks on a notification in the Notification Tab.
+   * Refer to go/ntab-urt-scribe for Notification Tab scribe details.
+   */
+  object NotificationClick
+      extends BaseNotificationTabClientEvent(ActionType.ClientNotificationClick)
+
+  /**
+   * This is fired when a user taps the "See Less Often" caret menu item of a notification in
+   * the Notification Tab.
+   * Refer to go/ntab-urt-scribe for Notification Tab scribe details.
+   */
+  object NotificationSeeLessOften
+      extends BaseNotificationTabClientEvent(ActionType.ClientNotificationSeeLessOften)
+
+  /**
+   * This is fired when a user closes or swipes away a Push Notification.
+   * Refer to https://confluence.twitter.biz/pages/viewpage.action?pageId=161811800
+   * for Push Notification scribe details
+   */
+  object NotificationDismiss
+      extends BasePushNotificationClientEvent(ActionType.ClientNotificationDismiss)
+
+  /**
+   *  This is fired when a user clicks on a typeahead suggestion(queries, events, topics, users)
+   *  in a drop-down menu of a search box or a tweet compose box.
+   */
+  object TypeaheadClick extends BaseSearchTypeaheadEvent(ActionType.ClientTypeaheadClick)
+
+  /**
+   * This is a generic event fired when the user submits feedback on a prompt.
+   * Some examples include Did You Find It Prompt and Tweet Relevance on Search Results Page.
+   */
+  object FeedbackPromptSubmit
+      extends BaseFeedbackSubmitClientEvent(ActionType.ClientFeedbackPromptSubmit)
+
+  object AppExit extends BaseUASClientEvent(ActionType.ClientAppExit)
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ClientEventImpression.scala

@@ -0,0 +1,207 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.EventNamespace
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.thriftscala._
+import com.twitter.unified_user_actions.thriftscala.Item.TweetInfo
+
+object ClientEventImpression {
+  object TweetLingerImpression extends BaseClientEvent(ActionType.ClientTweetLingerImpression) {
+    override def getUuaItem(
+      ceItem: LogEventItem,
+      logEvent: LogEvent
+    ): Option[Item] = {
+      for {
+        actionTweetId <- ceItem.id
+        impressionDetails <- ceItem.impressionDetails
+        lingerStartTimestampMs <- impressionDetails.visibilityStart
+        lingerEndTimestampMs <- impressionDetails.visibilityEnd
+      } yield {
+        Item.TweetInfo(
+          ClientEventCommonUtils
+            .getBasicTweetInfo(actionTweetId, ceItem, logEvent.eventNamespace)
+            .copy(tweetActionInfo = Some(
+              TweetActionInfo.ClientTweetLingerImpression(
+                ClientTweetLingerImpression(
+                  lingerStartTimestampMs = lingerStartTimestampMs,
+                  lingerEndTimestampMs = lingerEndTimestampMs
+                )
+              ))))
+      }
+    }
+  }
+
+  /**
+   * To make parity with iesource's definition, render impression for quoted Tweets would emit
+   * 2 events: 1 for the quoting Tweet and 1 for the original Tweet!!!
+   */
+  object TweetRenderImpression extends BaseClientEvent(ActionType.ClientTweetRenderImpression) {
+    override def toUnifiedUserAction(logEvent: LogEvent): Seq[UnifiedUserAction] = {
+
+      val logBase: Option[LogBase] = logEvent.logBase
+
+      val raw = for {
+        ed <- logEvent.eventDetails.toSeq
+        items <- ed.items.toSeq
+        ceItem <- items
+        eventTimestamp <- logBase.flatMap(getSourceTimestamp)
+        uuaItem <- getUuaItem(ceItem, logEvent)
+        if isItemTypeValid(ceItem.itemType)
+      } yield {
+        val userIdentifier: UserIdentifier = UserIdentifier(
+          userId = logBase.flatMap(_.userId),
+          guestIdMarketing = logBase.flatMap(_.guestIdMarketing))
+
+        val productSurface: Option[ProductSurface] = ProductSurfaceUtils
+          .getProductSurface(logEvent.eventNamespace)
+
+        val eventMetaData: EventMetadata = ClientEventCommonUtils
+          .getEventMetadata(
+            eventTimestamp = eventTimestamp,
+            logEvent = logEvent,
+            ceItem = ceItem,
+            productSurface = productSurface
+          )
+
+        UnifiedUserAction(
+          userIdentifier = userIdentifier,
+          item = uuaItem,
+          actionType = ActionType.ClientTweetRenderImpression,
+          eventMetadata = eventMetaData,
+          productSurface = productSurface,
+          productSurfaceInfo =
+            ProductSurfaceUtils.getProductSurfaceInfo(productSurface, ceItem, logEvent)
+        )
+      }
+
+      raw.flatMap { e =>
+        e.item match {
+          case TweetInfo(t) =>
+            // If it is an impression toward quoted Tweet we emit 2 impressions, 1 for quoting Tweet
+            // and 1 for the original Tweet.
+            if (t.quotedTweetId.isDefined) {
+              val originalItem = t.copy(
+                actionTweetId = t.quotedTweetId.get,
+                actionTweetAuthorInfo = t.quotedAuthorId.map(id => AuthorInfo(authorId = Some(id))),
+                quotingTweetId = Some(t.actionTweetId),
+                quotedTweetId = None,
+                inReplyToTweetId = None,
+                replyingTweetId = None,
+                retweetingTweetId = None,
+                retweetedTweetId = None,
+                quotedAuthorId = None,
+                retweetingAuthorId = None,
+                inReplyToAuthorId = None
+              )
+              val original = e.copy(item = TweetInfo(originalItem))
+              Seq(original, e)
+            } else Seq(e)
+          case _ => Nil
+        }
+      }
+    }
+  }
+
+  object TweetGalleryImpression extends BaseClientEvent(ActionType.ClientTweetGalleryImpression)
+
+  object TweetDetailsImpression extends BaseClientEvent(ActionType.ClientTweetDetailsImpression) {
+
+    case class EventNamespaceInternal(
+      client: String,
+      page: String,
+      section: String,
+      component: String,
+      element: String,
+      action: String)
+
+    def isTweetDetailsImpression(eventNamespaceOpt: Option[EventNamespace]): Boolean =
+      eventNamespaceOpt.exists { eventNamespace =>
+        val eventNamespaceInternal = EventNamespaceInternal(
+          client = eventNamespace.client.getOrElse(""),
+          page = eventNamespace.page.getOrElse(""),
+          section = eventNamespace.section.getOrElse(""),
+          component = eventNamespace.component.getOrElse(""),
+          element = eventNamespace.element.getOrElse(""),
+          action = eventNamespace.action.getOrElse(""),
+        )
+
+        isIphoneAppOrMacAppOrIpadAppClientTweetDetailsImpression(
+          eventNamespaceInternal) || isAndroidAppClientTweetDetailsImpression(
+          eventNamespaceInternal) || isWebClientTweetDetailImpression(
+          eventNamespaceInternal) || isTweetDeckAppClientTweetDetailsImpression(
+          eventNamespaceInternal) || isOtherAppClientTweetDetailsImpression(eventNamespaceInternal)
+      }
+
+    private def isWebClientTweetDetailImpression(
+      eventNamespace: EventNamespaceInternal
+    ): Boolean = {
+      val eventNameSpaceStr =
+        eventNamespace.client + ":" + eventNamespace.page + ":" + eventNamespace.section + ":" + eventNamespace.component + ":" + eventNamespace.element + ":" + eventNamespace.action
+      eventNameSpaceStr.equalsIgnoreCase("m5:tweet::::show") || eventNameSpaceStr.equalsIgnoreCase(
+        "m5:tweet:landing:::show") || eventNameSpaceStr
+        .equalsIgnoreCase("m2:tweet::::impression") || eventNameSpaceStr.equalsIgnoreCase(
+        "m2:tweet::tweet::impression") || eventNameSpaceStr
+        .equalsIgnoreCase("LiteNativeWrapper:tweet::::show") || eventNameSpaceStr.equalsIgnoreCase(
+        "LiteNativeWrapper:tweet:landing:::show")
+    }
+
+    private def isOtherAppClientTweetDetailsImpression(
+      eventNamespace: EventNamespaceInternal
+    ): Boolean = {
+      val excludedClients = Set(
+        "web",
+        "m5",
+        "m2",
+        "LiteNativeWrapper",
+        "iphone",
+        "ipad",
+        "mac",
+        "android",
+        "android_tablet",
+        "deck")
+      (!excludedClients.contains(eventNamespace.client)) && eventNamespace.page
+        .equalsIgnoreCase("tweet") && eventNamespace.section
+        .equalsIgnoreCase("") && eventNamespace.component
+        .equalsIgnoreCase("tweet") && eventNamespace.element
+        .equalsIgnoreCase("") && eventNamespace.action.equalsIgnoreCase("impression")
+    }
+
+    private def isTweetDeckAppClientTweetDetailsImpression(
+      eventNamespace: EventNamespaceInternal
+    ): Boolean =
+      eventNamespace.client
+        .equalsIgnoreCase("deck") && eventNamespace.page
+        .equalsIgnoreCase("tweet") && eventNamespace.section
+        .equalsIgnoreCase("") && eventNamespace.component
+        .equalsIgnoreCase("tweet") && eventNamespace.element
+        .equalsIgnoreCase("") && eventNamespace.action.equalsIgnoreCase("impression")
+
+    private def isAndroidAppClientTweetDetailsImpression(
+      eventNamespace: EventNamespaceInternal
+    ): Boolean =
+      (eventNamespace.client
+        .equalsIgnoreCase("android") || eventNamespace.client
+        .equalsIgnoreCase("android_tablet")) && eventNamespace.page
+        .equalsIgnoreCase("tweet") && eventNamespace.section.equalsIgnoreCase(
+        "") && (eventNamespace.component
+        .equalsIgnoreCase("tweet") || eventNamespace.component
+        .matches("^suggest.*_tweet.*$") || eventNamespace.component
+        .equalsIgnoreCase("")) && eventNamespace.element
+        .equalsIgnoreCase("") && eventNamespace.action.equalsIgnoreCase("impression")
+
+    private def isIphoneAppOrMacAppOrIpadAppClientTweetDetailsImpression(
+      eventNamespace: EventNamespaceInternal
+    ): Boolean =
+      (eventNamespace.client
+        .equalsIgnoreCase("iphone") || eventNamespace.client
+        .equalsIgnoreCase("ipad") || eventNamespace.client
+        .equalsIgnoreCase("mac")) && eventNamespace.page.equalsIgnoreCase(
+        "tweet") && eventNamespace.section
+        .equalsIgnoreCase("") && (eventNamespace.component
+        .equalsIgnoreCase("tweet") || eventNamespace.component
+        .matches("^suggest.*_tweet.*$")) && eventNamespace.element
+        .equalsIgnoreCase("") && eventNamespace.action.equalsIgnoreCase("impression")
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/HomeInfoUtils.scala

@@ -0,0 +1,32 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.suggests.controller_data.home_tweets.thriftscala.HomeTweetsControllerData
+import com.twitter.suggests.controller_data.home_tweets.thriftscala.HomeTweetsControllerDataAliases.V1Alias
+import com.twitter.suggests.controller_data.thriftscala.ControllerData
+import com.twitter.suggests.controller_data.v2.thriftscala.{ControllerData => ControllerDataV2}
+
+object HomeInfoUtils {
+
+  def getHomeTweetControllerDataV1(ceItem: LogEventItem): Option[V1Alias] = {
+    ceItem.suggestionDetails
+      .flatMap(_.decodedControllerData)
+      .flatMap(_ match {
+        case ControllerData.V2(
+              ControllerDataV2.HomeTweets(
+                HomeTweetsControllerData.V1(homeTweetsControllerDataV1)
+              )) =>
+          Some(homeTweetsControllerDataV1)
+        case _ => None
+      })
+  }
+
+  def getTraceId(ceItem: LogEventItem): Option[Long] =
+    getHomeTweetControllerDataV1(ceItem).flatMap(_.traceId)
+
+  def getSuggestType(ceItem: LogEventItem): Option[String] =
+    ceItem.suggestionDetails.flatMap(_.suggestionType)
+
+  def getRequestJoinId(ceItem: LogEventItem): Option[Long] =
+    getHomeTweetControllerDataV1(ceItem).flatMap(_.requestJoinId)
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ItemTypeFilterPredicates.scala

@@ -0,0 +1,40 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.ItemType
+
+object ItemTypeFilterPredicates {
+  private val TweetItemTypes = Set[ItemType](ItemType.Tweet, ItemType.QuotedTweet)
+  private val TopicItemTypes = Set[ItemType](ItemType.Tweet, ItemType.QuotedTweet, ItemType.Topic)
+  private val ProfileItemTypes = Set[ItemType](ItemType.User)
+  private val TypeaheadResultItemTypes = Set[ItemType](ItemType.Search, ItemType.User)
+  private val SearchResultsPageFeedbackSubmitItemTypes =
+    Set[ItemType](ItemType.Tweet, ItemType.RelevancePrompt)
+
+  /**
+   *  DDG lambda metrics count Tweets based on the `itemType`
+   *  Reference code - https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/src/scala/com/twitter/experiments/lambda/shared/Timelines.scala?L156
+   *  Since enums `PROMOTED_TWEET` and `POPULAR_TWEET` are deprecated in the following thrift
+   *  https://sourcegraph.twitter.biz/git.twitter.biz/source/-/blob/src/thrift/com/twitter/clientapp/gen/client_app.thrift?L131
+   *  UUA filters two types of Tweets only: `TWEET` and `QUOTED_TWEET`
+   */
+  def isItemTypeTweet(itemTypeOpt: Option[ItemType]): Boolean =
+    itemTypeOpt.exists(itemType => TweetItemTypes.contains(itemType))
+
+  def isItemTypeTopic(itemTypeOpt: Option[ItemType]): Boolean =
+    itemTypeOpt.exists(itemType => TopicItemTypes.contains(itemType))
+
+  def isItemTypeProfile(itemTypeOpt: Option[ItemType]): Boolean =
+    itemTypeOpt.exists(itemType => ProfileItemTypes.contains(itemType))
+
+  def isItemTypeTypeaheadResult(itemTypeOpt: Option[ItemType]): Boolean =
+    itemTypeOpt.exists(itemType => TypeaheadResultItemTypes.contains(itemType))
+
+  def isItemTypeForSearchResultsPageFeedbackSubmit(itemTypeOpt: Option[ItemType]): Boolean =
+    itemTypeOpt.exists(itemType => SearchResultsPageFeedbackSubmitItemTypes.contains(itemType))
+
+  /**
+   * Always return true. Use this when there is no need to filter based on `item_type` and all
+   * values of `item_type` are acceptable.
+   */
+  def ignoreItemType(itemTypeOpt: Option[ItemType]): Boolean = true
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/NotificationClientEventUtils.scala

@@ -0,0 +1,26 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+
+object NotificationClientEventUtils {
+
+  // Notification id for notification in the Notification Tab
+  def getNotificationIdForNotificationTab(
+    ceItem: LogEventItem
+  ): Option[String] = {
+    for {
+      notificationTabDetails <- ceItem.notificationTabDetails
+      clientEventMetaData <- notificationTabDetails.clientEventMetadata
+      notificationId <- clientEventMetaData.upstreamId
+    } yield {
+      notificationId
+    }
+  }
+
+  // Notification id for Push Notification
+  def getNotificationIdForPushNotification(logEvent: LogEvent): Option[String] = for {
+    pushNotificationDetails <- logEvent.notificationDetails
+    notificationId <- pushNotificationDetails.impressionId
+  } yield notificationId
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/ProductSurfaceUtils.scala

@@ -0,0 +1,109 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.EventNamespace
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.suggests.controller_data.home_tweets.thriftscala.HomeTweetsControllerDataAliases.V1Alias
+import com.twitter.unified_user_actions.thriftscala._
+
+object ProductSurfaceUtils {
+
+  def getProductSurface(eventNamespace: Option[EventNamespace]): Option[ProductSurface] = {
+    (
+      eventNamespace.flatMap(_.page),
+      eventNamespace.flatMap(_.section),
+      eventNamespace.flatMap(_.element)) match {
+      case (Some("home") | Some("home_latest"), _, _) => Some(ProductSurface.HomeTimeline)
+      case (Some("ntab"), _, _) => Some(ProductSurface.NotificationTab)
+      case (Some(page), Some(section), _) if isPushNotification(page, section) =>
+        Some(ProductSurface.PushNotification)
+      case (Some("search"), _, _) => Some(ProductSurface.SearchResultsPage)
+      case (_, _, Some("typeahead")) => Some(ProductSurface.SearchTypeahead)
+      case _ => None
+    }
+  }
+
+  private def isPushNotification(page: String, section: String): Boolean = {
+    Seq[String]("notification", "toasts").contains(page) ||
+    (page == "app" && section == "push")
+  }
+
+  def getProductSurfaceInfo(
+    productSurface: Option[ProductSurface],
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[ProductSurfaceInfo] = {
+    productSurface match {
+      case Some(ProductSurface.HomeTimeline) => createHomeTimelineInfo(ceItem)
+      case Some(ProductSurface.NotificationTab) => createNotificationTabInfo(ceItem)
+      case Some(ProductSurface.PushNotification) => createPushNotificationInfo(logEvent)
+      case Some(ProductSurface.SearchResultsPage) => createSearchResultPageInfo(ceItem, logEvent)
+      case Some(ProductSurface.SearchTypeahead) => createSearchTypeaheadInfo(ceItem, logEvent)
+      case _ => None
+    }
+  }
+
+  private def createPushNotificationInfo(logEvent: LogEvent): Option[ProductSurfaceInfo] =
+    NotificationClientEventUtils.getNotificationIdForPushNotification(logEvent) match {
+      case Some(notificationId) =>
+        Some(
+          ProductSurfaceInfo.PushNotificationInfo(
+            PushNotificationInfo(notificationId = notificationId)))
+      case _ => None
+    }
+
+  private def createNotificationTabInfo(ceItem: LogEventItem): Option[ProductSurfaceInfo] =
+    NotificationClientEventUtils.getNotificationIdForNotificationTab(ceItem) match {
+      case Some(notificationId) =>
+        Some(
+          ProductSurfaceInfo.NotificationTabInfo(
+            NotificationTabInfo(notificationId = notificationId)))
+      case _ => None
+    }
+
+  private def createHomeTimelineInfo(ceItem: LogEventItem): Option[ProductSurfaceInfo] = {
+    def suggestType: Option[String] = HomeInfoUtils.getSuggestType(ceItem)
+    def controllerData: Option[V1Alias] = HomeInfoUtils.getHomeTweetControllerDataV1(ceItem)
+
+    if (suggestType.isDefined || controllerData.isDefined) {
+      Some(
+        ProductSurfaceInfo.HomeTimelineInfo(
+          HomeTimelineInfo(
+            suggestionType = suggestType,
+            injectedPosition = controllerData.flatMap(_.injectedPosition)
+          )))
+    } else None
+  }
+
+  private def createSearchResultPageInfo(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[ProductSurfaceInfo] = {
+    val searchInfoUtil = new SearchInfoUtils(ceItem)
+    searchInfoUtil.getQueryOptFromItem(logEvent).map { query =>
+      ProductSurfaceInfo.SearchResultsPageInfo(
+        SearchResultsPageInfo(
+          query = query,
+          querySource = searchInfoUtil.getQuerySourceOptFromControllerDataFromItem,
+          itemPosition = ceItem.position,
+          tweetResultSources = searchInfoUtil.getTweetResultSources,
+          userResultSources = searchInfoUtil.getUserResultSources,
+          queryFilterType = searchInfoUtil.getQueryFilterType(logEvent)
+        ))
+    }
+  }
+
+  private def createSearchTypeaheadInfo(
+    ceItem: LogEventItem,
+    logEvent: LogEvent
+  ): Option[ProductSurfaceInfo] = {
+    logEvent.searchDetails.flatMap(_.query).map { query =>
+      ProductSurfaceInfo.SearchTypeaheadInfo(
+        SearchTypeaheadInfo(
+          query = query,
+          itemPosition = ceItem.position
+        )
+      )
+    }
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/SearchInfoUtils.scala

@@ -0,0 +1,129 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.LogEvent
+import com.twitter.clientapp.thriftscala.{Item => LogEventItem}
+import com.twitter.search.common.constants.thriftscala.ThriftQuerySource
+import com.twitter.search.common.constants.thriftscala.TweetResultSource
+import com.twitter.search.common.constants.thriftscala.UserResultSource
+import com.twitter.suggests.controller_data.search_response.item_types.thriftscala.ItemTypesControllerData
+import com.twitter.suggests.controller_data.search_response.item_types.thriftscala.ItemTypesControllerData.TweetTypesControllerData
+import com.twitter.suggests.controller_data.search_response.item_types.thriftscala.ItemTypesControllerData.UserTypesControllerData
+import com.twitter.suggests.controller_data.search_response.request.thriftscala.RequestControllerData
+import com.twitter.suggests.controller_data.search_response.thriftscala.SearchResponseControllerData.V1
+import com.twitter.suggests.controller_data.search_response.thriftscala.SearchResponseControllerDataAliases.V1Alias
+import com.twitter.suggests.controller_data.thriftscala.ControllerData.V2
+import com.twitter.suggests.controller_data.v2.thriftscala.ControllerData.SearchResponse
+import com.twitter.unified_user_actions.thriftscala.SearchQueryFilterType
+import com.twitter.unified_user_actions.thriftscala.SearchQueryFilterType._
+
+class SearchInfoUtils(item: LogEventItem) {
+  private val searchControllerDataOpt: Option[V1Alias] = item.suggestionDetails.flatMap { sd =>
+    sd.decodedControllerData.flatMap { decodedControllerData =>
+      decodedControllerData match {
+        case V2(v2ControllerData) =>
+          v2ControllerData match {
+            case SearchResponse(searchResponseControllerData) =>
+              searchResponseControllerData match {
+                case V1(searchResponseControllerDataV1) =>
+                  Some(searchResponseControllerDataV1)
+                case _ => None
+              }
+            case _ =>
+              None
+          }
+        case _ => None
+      }
+    }
+  }
+
+  private val requestControllerDataOptFromItem: Option[RequestControllerData] =
+    searchControllerDataOpt.flatMap { searchControllerData =>
+      searchControllerData.requestControllerData
+    }
+  private val itemTypesControllerDataOptFromItem: Option[ItemTypesControllerData] =
+    searchControllerDataOpt.flatMap { searchControllerData =>
+      searchControllerData.itemTypesControllerData
+    }
+
+  def checkBit(bitmap: Long, idx: Int): Boolean = {
+    (bitmap / Math.pow(2, idx)).toInt % 2 == 1
+  }
+
+  def getQueryOptFromSearchDetails(logEvent: LogEvent): Option[String] = {
+    logEvent.searchDetails.flatMap { sd => sd.query }
+  }
+
+  def getQueryOptFromControllerDataFromItem: Option[String] = {
+    requestControllerDataOptFromItem.flatMap { rd => rd.rawQuery }
+  }
+
+  def getQueryOptFromItem(logEvent: LogEvent): Option[String] = {
+    // First we try to get the query from controller data, and if that's not available, we fall
+    // back to query in search details. If both are None, queryOpt is None.
+    getQueryOptFromControllerDataFromItem.orElse(getQueryOptFromSearchDetails(logEvent))
+  }
+
+  def getTweetTypesOptFromControllerDataFromItem: Option[TweetTypesControllerData] = {
+    itemTypesControllerDataOptFromItem.flatMap { itemTypes =>
+      itemTypes match {
+        case TweetTypesControllerData(tweetTypesControllerData) =>
+          Some(TweetTypesControllerData(tweetTypesControllerData))
+        case _ => None
+      }
+    }
+  }
+
+  def getUserTypesOptFromControllerDataFromItem: Option[UserTypesControllerData] = {
+    itemTypesControllerDataOptFromItem.flatMap { itemTypes =>
+      itemTypes match {
+        case UserTypesControllerData(userTypesControllerData) =>
+          Some(UserTypesControllerData(userTypesControllerData))
+        case _ => None
+      }
+    }
+  }
+
+  def getQuerySourceOptFromControllerDataFromItem: Option[ThriftQuerySource] = {
+    requestControllerDataOptFromItem
+      .flatMap { rd => rd.querySource }
+      .flatMap { querySourceVal => ThriftQuerySource.get(querySourceVal) }
+  }
+
+  def getTweetResultSources: Option[Set[TweetResultSource]] = {
+    getTweetTypesOptFromControllerDataFromItem
+      .flatMap { cd => cd.tweetTypesControllerData.tweetTypesBitmap }
+      .map { tweetTypesBitmap =>
+        TweetResultSource.list.filter { t => checkBit(tweetTypesBitmap, t.value) }.toSet
+      }
+  }
+
+  def getUserResultSources: Option[Set[UserResultSource]] = {
+    getUserTypesOptFromControllerDataFromItem
+      .flatMap { cd => cd.userTypesControllerData.userTypesBitmap }
+      .map { userTypesBitmap =>
+        UserResultSource.list.filter { t => checkBit(userTypesBitmap, t.value) }.toSet
+      }
+  }
+
+  def getQueryFilterType(logEvent: LogEvent): Option[SearchQueryFilterType] = {
+    val searchTab = logEvent.eventNamespace.map(_.client).flatMap {
+      case Some("m5") | Some("android") => logEvent.eventNamespace.flatMap(_.element)
+      case _ => logEvent.eventNamespace.flatMap(_.section)
+    }
+    searchTab.flatMap {
+      case "search_filter_top" => Some(Top)
+      case "search_filter_live" => Some(Latest)
+      // android uses search_filter_tweets instead of search_filter_live
+      case "search_filter_tweets" => Some(Latest)
+      case "search_filter_user" => Some(People)
+      case "search_filter_image" => Some(Photos)
+      case "search_filter_video" => Some(Videos)
+      case _ => None
+    }
+  }
+
+  def getRequestJoinId: Option[Long] = requestControllerDataOptFromItem.flatMap(_.requestJoinId)
+
+  def getTraceId: Option[Long] = requestControllerDataOptFromItem.flatMap(_.traceId)
+
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/TopicIdUtils.scala

@@ -0,0 +1,157 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.EventNamespace
+import com.twitter.clientapp.thriftscala.Item
+import com.twitter.clientapp.thriftscala.ItemType.Topic
+import com.twitter.guide.scribing.thriftscala.TopicModuleMetadata
+import com.twitter.guide.scribing.thriftscala.TransparentGuideDetails
+import com.twitter.suggests.controller_data.home_hitl_topic_annotation_prompt.thriftscala.HomeHitlTopicAnnotationPromptControllerData
+import com.twitter.suggests.controller_data.home_hitl_topic_annotation_prompt.v1.thriftscala.{
+  HomeHitlTopicAnnotationPromptControllerData => HomeHitlTopicAnnotationPromptControllerDataV1
+}
+import com.twitter.suggests.controller_data.home_topic_annotation_prompt.thriftscala.HomeTopicAnnotationPromptControllerData
+import com.twitter.suggests.controller_data.home_topic_annotation_prompt.v1.thriftscala.{
+  HomeTopicAnnotationPromptControllerData => HomeTopicAnnotationPromptControllerDataV1
+}
+import com.twitter.suggests.controller_data.home_topic_follow_prompt.thriftscala.HomeTopicFollowPromptControllerData
+import com.twitter.suggests.controller_data.home_topic_follow_prompt.v1.thriftscala.{
+  HomeTopicFollowPromptControllerData => HomeTopicFollowPromptControllerDataV1
+}
+import com.twitter.suggests.controller_data.home_tweets.thriftscala.HomeTweetsControllerData
+import com.twitter.suggests.controller_data.home_tweets.v1.thriftscala.{
+  HomeTweetsControllerData => HomeTweetsControllerDataV1
+}
+import com.twitter.suggests.controller_data.search_response.item_types.thriftscala.ItemTypesControllerData
+import com.twitter.suggests.controller_data.search_response.thriftscala.SearchResponseControllerData
+import com.twitter.suggests.controller_data.search_response.topic_follow_prompt.thriftscala.SearchTopicFollowPromptControllerData
+import com.twitter.suggests.controller_data.search_response.tweet_types.thriftscala.TweetTypesControllerData
+import com.twitter.suggests.controller_data.search_response.v1.thriftscala.{
+  SearchResponseControllerData => SearchResponseControllerDataV1
+}
+import com.twitter.suggests.controller_data.thriftscala.ControllerData
+import com.twitter.suggests.controller_data.timelines_topic.thriftscala.TimelinesTopicControllerData
+import com.twitter.suggests.controller_data.timelines_topic.v1.thriftscala.{
+  TimelinesTopicControllerData => TimelinesTopicControllerDataV1
+}
+import com.twitter.suggests.controller_data.v2.thriftscala.{ControllerData => ControllerDataV2}
+import com.twitter.util.Try
+
+object TopicIdUtils {
+  val DomainId: Long = 131 // Topical Domain
+
+  def getTopicId(
+    item: Item,
+    namespace: EventNamespace
+  ): Option[Long] =
+    getTopicIdFromHomeSearch(item)
+      .orElse(getTopicFromGuide(item))
+      .orElse(getTopicFromOnboarding(item, namespace))
+      .orElse(getTopicIdFromItem(item))
+
+  def getTopicIdFromItem(item: Item): Option[Long] =
+    if (item.itemType.contains(Topic))
+      item.id
+    else None
+
+  def getTopicIdFromHomeSearch(
+    item: Item
+  ): Option[Long] = {
+    val decodedControllerData = item.suggestionDetails.flatMap(_.decodedControllerData)
+    decodedControllerData match {
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.HomeTweets(
+                HomeTweetsControllerData.V1(homeTweets: HomeTweetsControllerDataV1)))
+          ) =>
+        homeTweets.topicId
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.HomeTopicFollowPrompt(
+                HomeTopicFollowPromptControllerData.V1(
+                  homeTopicFollowPrompt: HomeTopicFollowPromptControllerDataV1)))
+          ) =>
+        homeTopicFollowPrompt.topicId
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.TimelinesTopic(
+                TimelinesTopicControllerData.V1(
+                  timelinesTopic: TimelinesTopicControllerDataV1
+                )))
+          ) =>
+        Some(timelinesTopic.topicId)
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.SearchResponse(
+                SearchResponseControllerData.V1(s: SearchResponseControllerDataV1)))
+          ) =>
+        s.itemTypesControllerData match {
+          case Some(
+                ItemTypesControllerData.TopicFollowControllerData(
+                  topicFollowControllerData: SearchTopicFollowPromptControllerData)) =>
+            topicFollowControllerData.topicId
+          case Some(
+                ItemTypesControllerData.TweetTypesControllerData(
+                  tweetTypesControllerData: TweetTypesControllerData)) =>
+            tweetTypesControllerData.topicId
+          case _ => None
+        }
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.HomeTopicAnnotationPrompt(
+                HomeTopicAnnotationPromptControllerData.V1(
+                  homeTopicAnnotationPrompt: HomeTopicAnnotationPromptControllerDataV1
+                )))
+          ) =>
+        Some(homeTopicAnnotationPrompt.topicId)
+      case Some(
+            ControllerData.V2(
+              ControllerDataV2.HomeHitlTopicAnnotationPrompt(
+                HomeHitlTopicAnnotationPromptControllerData.V1(
+                  homeHitlTopicAnnotationPrompt: HomeHitlTopicAnnotationPromptControllerDataV1
+                )))
+          ) =>
+        Some(homeHitlTopicAnnotationPrompt.topicId)
+
+      case _ => None
+    }
+  }
+
+  def getTopicFromOnboarding(
+    item: Item,
+    namespace: EventNamespace
+  ): Option[Long] =
+    if (namespace.page.contains("onboarding") &&
+      (namespace.section.exists(_.contains("topic")) ||
+      namespace.component.exists(_.contains("topic")) ||
+      namespace.element.exists(_.contains("topic")))) {
+      item.description.flatMap { description =>
+        // description: "id=123,main=xyz,row=1"
+        val tokens = description.split(",").headOption.map(_.split("="))
+        tokens match {
+          case Some(Array("id", token, _*)) => Try(token.toLong).toOption
+          case _ => None
+        }
+      }
+    } else None
+
+  def getTopicFromGuide(
+    item: Item
+  ): Option[Long] =
+    item.guideItemDetails.flatMap {
+      _.transparentGuideDetails match {
+        case Some(TransparentGuideDetails.TopicMetadata(topicMetadata)) =>
+          topicMetadata match {
+            case TopicModuleMetadata.TttInterest(_) =>
+              None
+            case TopicModuleMetadata.SemanticCoreInterest(semanticCoreInterest) =>
+              if (semanticCoreInterest.domainId == DomainId.toString)
+                Try(semanticCoreInterest.entityId.toLong).toOption
+              else None
+            case TopicModuleMetadata.SimClusterInterest(_) =>
+              None
+            case TopicModuleMetadata.UnknownUnionField(_) => None
+          }
+        case _ => None
+      }
+    }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/client_event/VideoClientEventUtils.scala

@@ -0,0 +1,42 @@
+package com.twitter.unified_user_actions.adapter.client_event
+
+import com.twitter.clientapp.thriftscala.AmplifyDetails
+import com.twitter.clientapp.thriftscala.MediaDetails
+import com.twitter.unified_user_actions.thriftscala.TweetVideoWatch
+import com.twitter.unified_user_actions.thriftscala.TweetActionInfo
+import com.twitter.video.analytics.thriftscala.MediaIdentifier
+
+object VideoClientEventUtils {
+
+  /**
+   * For Tweets with multiple videos, find the id of the video that generated the client-event
+   */
+  def videoIdFromMediaIdentifier(mediaIdentifier: MediaIdentifier): Option[String] =
+    mediaIdentifier match {
+      case MediaIdentifier.MediaPlatformIdentifier(mediaPlatformIdentifier) =>
+        mediaPlatformIdentifier.mediaId.map(_.toString)
+      case _ => None
+    }
+
+  /**
+   * Given:
+   * 1. the id of the video (`mediaId`)
+   * 2. details about all the media items in the Tweet (`mediaItems`),
+   * iterate over the `mediaItems` to lookup the metadata about the video with id `mediaId`.
+   */
+  def getVideoMetadata(
+    mediaId: String,
+    mediaItems: Seq[MediaDetails],
+    amplifyDetails: Option[AmplifyDetails]
+  ): Option[TweetActionInfo] = {
+    mediaItems.collectFirst {
+      case media if media.contentId.contains(mediaId) =>
+        TweetActionInfo.TweetVideoWatch(
+          TweetVideoWatch(
+            mediaType = media.mediaType,
+            isMonetizable = media.dynamicAds,
+            videoType = amplifyDetails.flatMap(_.videoType)
+          ))
+    }
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common/AdapterUtils.scala

@@ -0,0 +1,15 @@
+package com.twitter.unified_user_actions.adapter.common
+
+import com.twitter.snowflake.id.SnowflakeId
+import com.twitter.util.Time
+
+object AdapterUtils {
+  def currentTimestampMs: Long = Time.now.inMilliseconds
+  def getTimestampMsFromTweetId(tweetId: Long): Long = SnowflakeId.unixTimeMillisFromId(tweetId)
+
+  // For now just make sure both language code and country code are in upper cases for consistency
+  // For language code, there are mixed lower and upper cases
+  // For country code, there are mixed lower and upper cases
+  def normalizeLanguageCode(inputLanguageCode: String): String = inputLanguageCode.toUpperCase
+  def normalizeCountryCode(inputCountryCode: String): String = inputCountryCode.toUpperCase
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common/BUILD

@@ -0,0 +1,10 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "snowflake/src/main/scala/com/twitter/snowflake/id",
+        "util/util-core:util-core-util",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/email_notification_event/BUILD

@@ -0,0 +1,14 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    compiler_option_sets = ["fatal_warnings"],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "kafka/finagle-kafka/finatra-kafka/src/main/scala",
+        "src/thrift/com/twitter/ibis:logging-scala",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter:base",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common",
+        "unified_user_actions/thrift/src/main/thrift/com/twitter/unified_user_actions:unified_user_actions-scala",
+    ],
+)

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/email_notification_event/EmailNotificationEventAdapter.scala

@@ -0,0 +1,55 @@
+package com.twitter.unified_user_actions.adapter.email_notification_event
+
+import com.twitter.finagle.stats.NullStatsReceiver
+import com.twitter.finagle.stats.StatsReceiver
+import com.twitter.finatra.kafka.serde.UnKeyed
+import com.twitter.ibis.thriftscala.NotificationScribe
+import com.twitter.ibis.thriftscala.NotificationScribeType
+import com.twitter.unified_user_actions.adapter.AbstractAdapter
+import com.twitter.unified_user_actions.thriftscala.ActionType
+import com.twitter.unified_user_actions.thriftscala.EmailNotificationInfo
+import com.twitter.unified_user_actions.thriftscala.Item
+import com.twitter.unified_user_actions.thriftscala.ProductSurface
+import com.twitter.unified_user_actions.thriftscala.ProductSurfaceInfo
+import com.twitter.unified_user_actions.thriftscala.TweetInfo
+import com.twitter.unified_user_actions.thriftscala.UnifiedUserAction
+import com.twitter.unified_user_actions.thriftscala.UserIdentifier
+
+class EmailNotificationEventAdapter
+    extends AbstractAdapter[NotificationScribe, UnKeyed, UnifiedUserAction] {
+  import EmailNotificationEventAdapter._
+  override def adaptOneToKeyedMany(
+    input: NotificationScribe,
+    statsReceiver: StatsReceiver = NullStatsReceiver
+  ): Seq[(UnKeyed, UnifiedUserAction)] =
+    adaptEvent(input).map { e => (UnKeyed, e) }
+}
+
+object EmailNotificationEventAdapter {
+
+  def adaptEvent(scribe: NotificationScribe): Seq[UnifiedUserAction] = {
+    Option(scribe).flatMap { e =>
+      e.`type` match {
+        case NotificationScribeType.Click =>
+          val tweetIdOpt = e.logBase.flatMap(EmailNotificationEventUtils.extractTweetId)
+          (tweetIdOpt, e.impressionId) match {
+            case (Some(tweetId), Some(impressionId)) =>
+              Some(
+                UnifiedUserAction(
+                  userIdentifier = UserIdentifier(userId = e.userId),
+                  item = Item.TweetInfo(TweetInfo(actionTweetId = tweetId)),
+                  actionType = ActionType.ClientTweetEmailClick,
+                  eventMetadata = EmailNotificationEventUtils.extractEventMetaData(e),
+                  productSurface = Some(ProductSurface.EmailNotification),
+                  productSurfaceInfo = Some(
+                    ProductSurfaceInfo.EmailNotificationInfo(
+                      EmailNotificationInfo(notificationId = impressionId)))
+                )
+              )
+            case _ => None
+          }
+        case _ => None
+      }
+    }.toSeq
+  }
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/email_notification_event/EmailNotificationEventUtils.scala

@@ -0,0 +1,39 @@
+package com.twitter.unified_user_actions.adapter.email_notification_event
+
+import com.twitter.ibis.thriftscala.NotificationScribe
+import com.twitter.logbase.thriftscala.LogBase
+import com.twitter.unified_user_actions.adapter.common.AdapterUtils
+import com.twitter.unified_user_actions.thriftscala.EventMetadata
+import com.twitter.unified_user_actions.thriftscala.SourceLineage
+
+object EmailNotificationEventUtils {
+
+  /*
+   * Extract TweetId from Logbase.page, here is a sample page below
+   * https://twitter.com/i/events/1580827044245544962?cn=ZmxleGlibGVfcmVjcw%3D%3D&refsrc=email
+   * */
+  def extractTweetId(path: String): Option[Long] = {
+    val ptn = raw".*/([0-9]+)\\??.*".r
+    path match {
+      case ptn(tweetId) =>
+        Some(tweetId.toLong)
+      case _ =>
+        None
+    }
+  }
+
+  def extractTweetId(logBase: LogBase): Option[Long] = logBase.page match {
+    case Some(path) => extractTweetId(path)
+    case None => None
+  }
+
+  def extractEventMetaData(scribe: NotificationScribe): EventMetadata =
+    EventMetadata(
+      sourceTimestampMs = scribe.timestamp,
+      receivedTimestampMs = AdapterUtils.currentTimestampMs,
+      sourceLineage = SourceLineage.EmailNotificationEvents,
+      language = scribe.logBase.flatMap(_.language),
+      countryCode = scribe.logBase.flatMap(_.country),
+      clientAppId = scribe.logBase.flatMap(_.clientAppId),
+    )
+}

unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/favorite_archival_events/BUILD

@@ -0,0 +1,14 @@
+scala_library(
+    sources = [
+        "*.scala",
+    ],
+    compiler_option_sets = ["fatal_warnings"],
+    tags = ["bazel-compatible"],
+    dependencies = [
+        "fanoutservice/thrift/src/main/thrift:thrift-scala",
+        "kafka/finagle-kafka/finatra-kafka/src/main/scala",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter:base",
+        "unified_user_actions/adapter/src/main/scala/com/twitter/unified_user_actions/adapter/common",
+        "unified_user_actions/thrift/src/main/thrift/com/twitter/unified_user_actions:unified_user_actions-scala",
+    ],
+)