Compare commits

..

3 Commits

Author SHA1 Message Date
twitter-team
138bb51997 [cr-mixer/home-mixer] Remove getLinearRankingParams in EarlybirdTensorflowBasedSimilarityEngine
Remove unused ranking params which are specified by services when making an Earlybird relevance search.

For cr-mixer: since we always set useTensorflowRanking = true in EarlybirdSimilarityEngineRouter, we will only ever use the TensorFlowBasedScoringFunction for ranking search results. That function doesn't rely on any of the linear params specified in getLinearRankingParams, nor the boosts because we set applyBoosts = false in the request. These parameters are therefore strictly redundant.

The parameters in home-mixer can be removed for essentially the same reason—the parameters are redundant given that we use the Tensorflow scoring function and don't apply boosts.
2023-04-04 20:39:52 -05:00
twitter-team
3496189edb [docs] Fix broken navi link in README
Closes twitter/the-algorithm#1655
2023-04-04 20:38:43 -05:00
twitter-team
3f6974687f [VF] updates includes addressing Ukraine labels
Closes twitter/the-algorithm#1489
Closes twitter/the-algorithm#1524
Closes twitter/the-algorithm#1659
Closes twitter/the-algorithm#1520
2023-04-04 20:35:00 -05:00
17 changed files with 111 additions and 167 deletions

View File

@ -26,7 +26,7 @@ These are the main components of the Recommendation Algorithm included in this r
| Tweet mixing & filtering | [home-mixer](home-mixer/README.md) | Main service used to construct and serve the Home Timeline. Built on [product-mixer](product-mixer/README.md). | | Tweet mixing & filtering | [home-mixer](home-mixer/README.md) | Main service used to construct and serve the Home Timeline. Built on [product-mixer](product-mixer/README.md). |
| | [visibility-filters](visibilitylib/README.md) | Responsible for filtering Twitter content to support legal compliance, improve product quality, increase user trust, protect revenue through the use of hard-filtering, visible product treatments, and coarse-grained downranking. | | | [visibility-filters](visibilitylib/README.md) | Responsible for filtering Twitter content to support legal compliance, improve product quality, increase user trust, protect revenue through the use of hard-filtering, visible product treatments, and coarse-grained downranking. |
| | [timelineranker](timelineranker/README.md) | Legacy service which provides relevance-scored tweets from the Earlybird Search Index and UTEG service. | | | [timelineranker](timelineranker/README.md) | Legacy service which provides relevance-scored tweets from the Earlybird Search Index and UTEG service. |
| Software framework | [navi](navi/navi/README.md) | High performance, machine learning model serving written in Rust. | | Software framework | [navi](navi/README.md) | High performance, machine learning model serving written in Rust. |
| | [product-mixer](product-mixer/README.md) | Software framework for building feeds of content. | | | [product-mixer](product-mixer/README.md) | Software framework for building feeds of content. |
| | [twml](twml/README.md) | Legacy machine learning framework built on TensorFlow v1. | | | [twml](twml/README.md) | Legacy machine learning framework built on TensorFlow v1. |

View File

@ -6,8 +6,6 @@ import com.twitter.search.earlybird.thriftscala.EarlybirdService
import com.twitter.search.earlybird.thriftscala.ThriftSearchQuery import com.twitter.search.earlybird.thriftscala.ThriftSearchQuery
import com.twitter.util.Time import com.twitter.util.Time
import com.twitter.search.common.query.thriftjava.thriftscala.CollectorParams import com.twitter.search.common.query.thriftjava.thriftscala.CollectorParams
import com.twitter.search.common.ranking.thriftscala.ThriftAgeDecayRankingParams
import com.twitter.search.common.ranking.thriftscala.ThriftLinearFeatureRankingParams
import com.twitter.search.common.ranking.thriftscala.ThriftRankingParams import com.twitter.search.common.ranking.thriftscala.ThriftRankingParams
import com.twitter.search.common.ranking.thriftscala.ThriftScoringFunctionType import com.twitter.search.common.ranking.thriftscala.ThriftScoringFunctionType
import com.twitter.search.earlybird.thriftscala.ThriftSearchRelevanceOptions import com.twitter.search.earlybird.thriftscala.ThriftSearchRelevanceOptions
@ -97,7 +95,7 @@ object EarlybirdTensorflowBasedSimilarityEngine {
// Whether to collect conversation IDs. Remove it for now. // Whether to collect conversation IDs. Remove it for now.
// collectConversationId = Gate.True(), // true for Home // collectConversationId = Gate.True(), // true for Home
rankingMode = ThriftSearchRankingMode.Relevance, rankingMode = ThriftSearchRankingMode.Relevance,
relevanceOptions = Some(getRelevanceOptions(query.useTensorflowRanking)), relevanceOptions = Some(getRelevanceOptions),
collectorParams = Some( collectorParams = Some(
CollectorParams( CollectorParams(
// numResultsToReturn defines how many results each EB shard will return to search root // numResultsToReturn defines how many results each EB shard will return to search root
@ -116,13 +114,11 @@ object EarlybirdTensorflowBasedSimilarityEngine {
// The specific values of recap relevance/reranking options correspond to // The specific values of recap relevance/reranking options correspond to
// experiment: enable_recap_reranking_2988,timeline_internal_disable_recap_filter // experiment: enable_recap_reranking_2988,timeline_internal_disable_recap_filter
// bucket : enable_rerank,disable_filter // bucket : enable_rerank,disable_filter
private def getRelevanceOptions(useTensorflowRanking: Boolean): ThriftSearchRelevanceOptions = { private def getRelevanceOptions: ThriftSearchRelevanceOptions = {
ThriftSearchRelevanceOptions( ThriftSearchRelevanceOptions(
proximityScoring = true, proximityScoring = true,
maxConsecutiveSameUser = Some(2), maxConsecutiveSameUser = Some(2),
rankingParams = rankingParams = Some(getTensorflowBasedRankingParams),
if (useTensorflowRanking) Some(getTensorflowBasedRankingParams)
else Some(getLinearRankingParams),
maxHitsToProcess = Some(500), maxHitsToProcess = Some(500),
maxUserBlendCount = Some(3), maxUserBlendCount = Some(3),
proximityPhraseWeight = 9.0, proximityPhraseWeight = 9.0,
@ -131,41 +127,12 @@ object EarlybirdTensorflowBasedSimilarityEngine {
} }
private def getTensorflowBasedRankingParams: ThriftRankingParams = { private def getTensorflowBasedRankingParams: ThriftRankingParams = {
getLinearRankingParams.copy( ThriftRankingParams(
`type` = Some(ThriftScoringFunctionType.TensorflowBased), `type` = Some(ThriftScoringFunctionType.TensorflowBased),
selectedTensorflowModel = Some("timelines_rectweet_replica"), selectedTensorflowModel = Some("timelines_rectweet_replica"),
minScore = -1.0e100,
applyBoosts = false, applyBoosts = false,
authorSpecificScoreAdjustments = None authorSpecificScoreAdjustments = None
) )
} }
private def getLinearRankingParams: ThriftRankingParams = {
ThriftRankingParams(
`type` = Some(ThriftScoringFunctionType.Linear),
minScore = -1.0e100,
retweetCountParams = Some(ThriftLinearFeatureRankingParams(weight = 20.0)),
replyCountParams = Some(ThriftLinearFeatureRankingParams(weight = 1.0)),
reputationParams = Some(ThriftLinearFeatureRankingParams(weight = 0.2)),
luceneScoreParams = Some(ThriftLinearFeatureRankingParams(weight = 2.0)),
textScoreParams = Some(ThriftLinearFeatureRankingParams(weight = 0.18)),
urlParams = Some(ThriftLinearFeatureRankingParams(weight = 2.0)),
isReplyParams = Some(ThriftLinearFeatureRankingParams(weight = 1.0)),
favCountParams = Some(ThriftLinearFeatureRankingParams(weight = 30.0)),
langEnglishUIBoost = 0.5,
langEnglishTweetBoost = 0.2,
langDefaultBoost = 0.02,
unknownLanguageBoost = 0.05,
offensiveBoost = 0.1,
inTrustedCircleBoost = 3.0,
multipleHashtagsOrTrendsBoost = 0.6,
inDirectFollowBoost = 4.0,
tweetHasTrendBoost = 1.1,
selfTweetBoost = 2.0,
tweetHasImageUrlBoost = 2.0,
tweetHasVideoUrlBoost = 2.0,
useUserLanguageInfo = true,
ageDecayParams = Some(ThriftAgeDecayRankingParams(slope = 0.005, base = 1.0))
)
}
} }

View File

@ -15,28 +15,6 @@ object RelevanceSearchUtil {
`type` = Some(scr.ThriftScoringFunctionType.TensorflowBased), `type` = Some(scr.ThriftScoringFunctionType.TensorflowBased),
selectedTensorflowModel = Some("timelines_rectweet_replica"), selectedTensorflowModel = Some("timelines_rectweet_replica"),
minScore = -1.0e100, minScore = -1.0e100,
retweetCountParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 20.0)),
replyCountParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 1.0)),
reputationParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 0.2)),
luceneScoreParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 2.0)),
textScoreParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 0.18)),
urlParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 2.0)),
isReplyParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 1.0)),
favCountParams = Some(scr.ThriftLinearFeatureRankingParams(weight = 30.0)),
langEnglishUIBoost = 0.5,
langEnglishTweetBoost = 0.2,
langDefaultBoost = 0.02,
unknownLanguageBoost = 0.05,
offensiveBoost = 0.1,
inTrustedCircleBoost = 3.0,
multipleHashtagsOrTrendsBoost = 0.6,
inDirectFollowBoost = 4.0,
tweetHasTrendBoost = 1.1,
selfTweetBoost = 2.0,
tweetHasImageUrlBoost = 2.0,
tweetHasVideoUrlBoost = 2.0,
useUserLanguageInfo = true,
ageDecayParams = Some(scr.ThriftAgeDecayRankingParams(slope = 0.005, base = 1.0)),
selectedModels = Some(Map("home_mixer_unified_engagement_prod" -> 1.0)), selectedModels = Some(Map("home_mixer_unified_engagement_prod" -> 1.0)),
applyBoosts = false, applyBoosts = false,
) )

View File

@ -494,6 +494,9 @@ visibility_library_enable_trends_representative_tweet_safety_level:
visibility_library_enable_trusted_friends_user_list_safety_level: visibility_library_enable_trusted_friends_user_list_safety_level:
default_availability: 10000 default_availability: 10000
visibility_library_enable_twitter_delegate_user_list_safety_level:
default_availability: 10000
visibility_library_enable_tweet_detail_safety_level: visibility_library_enable_tweet_detail_safety_level:
default_availability: 10000 default_availability: 10000
@ -758,7 +761,7 @@ visibility_library_enable_short_circuiting_from_blender_visibility_library:
visibility_library_enable_short_circuiting_from_search_visibility_library: visibility_library_enable_short_circuiting_from_search_visibility_library:
default_availability: 0 default_availability: 0
visibility_library_enable_nsfw_text_topics_drop_rule: visibility_library_enable_nsfw_text_high_precision_drop_rule:
default_availability: 10000 default_availability: 10000
visibility_library_enable_spammy_tweet_rule_verdict_logging: visibility_library_enable_spammy_tweet_rule_verdict_logging:

View File

@ -535,6 +535,9 @@ private[visibility] object DeciderKey extends DeciderKeyEnum {
val EnableTrustedFriendsUserListSafetyLevel: Value = Value( val EnableTrustedFriendsUserListSafetyLevel: Value = Value(
"visibility_library_enable_trusted_friends_user_list_safety_level" "visibility_library_enable_trusted_friends_user_list_safety_level"
) )
val EnableTwitterDelegateUserListSafetyLevel: Value = Value(
"visibility_library_enable_twitter_delegate_user_list_safety_level"
)
val EnableTweetDetailSafetyLevel: Value = Value( val EnableTweetDetailSafetyLevel: Value = Value(
"visibility_library_enable_tweet_detail_safety_level" "visibility_library_enable_tweet_detail_safety_level"
) )
@ -869,8 +872,8 @@ private[visibility] object DeciderKey extends DeciderKeyEnum {
"visibility_library_enable_short_circuiting_from_search_visibility_library" "visibility_library_enable_short_circuiting_from_search_visibility_library"
) )
val EnableNsfwTextTopicsDropRule: Value = Value( val EnableNsfwTextHighPrecisionDropRule: Value = Value(
"visibility_library_enable_nsfw_text_topics_drop_rule" "visibility_library_enable_nsfw_text_high_precision_drop_rule"
) )
val EnableSpammyTweetRuleVerdictLogging: Value = Value( val EnableSpammyTweetRuleVerdictLogging: Value = Value(

View File

@ -198,6 +198,7 @@ private[visibility] object VisibilityDeciders {
TopicRecommendations -> DeciderKey.EnableTopicRecommendationsSafetyLevel, TopicRecommendations -> DeciderKey.EnableTopicRecommendationsSafetyLevel,
TrendsRepresentativeTweet -> DeciderKey.EnableTrendsRepresentativeTweetSafetyLevel, TrendsRepresentativeTweet -> DeciderKey.EnableTrendsRepresentativeTweetSafetyLevel,
TrustedFriendsUserList -> DeciderKey.EnableTrustedFriendsUserListSafetyLevel, TrustedFriendsUserList -> DeciderKey.EnableTrustedFriendsUserListSafetyLevel,
TwitterDelegateUserList -> DeciderKey.EnableTwitterDelegateUserListSafetyLevel,
TweetDetail -> DeciderKey.EnableTweetDetailSafetyLevel, TweetDetail -> DeciderKey.EnableTweetDetailSafetyLevel,
TweetDetailNonToo -> DeciderKey.EnableTweetDetailNonTooSafetyLevel, TweetDetailNonToo -> DeciderKey.EnableTweetDetailNonTooSafetyLevel,
TweetEngagers -> DeciderKey.EnableTweetEngagersSafetyLevel, TweetEngagers -> DeciderKey.EnableTweetEngagersSafetyLevel,
@ -287,7 +288,7 @@ private[visibility] object VisibilityDeciders {
RuleParams.EnableDropAllTrustedFriendsTweetsRuleParam -> DeciderKey.EnableDropAllTrustedFriendsTweetsRule, RuleParams.EnableDropAllTrustedFriendsTweetsRuleParam -> DeciderKey.EnableDropAllTrustedFriendsTweetsRule,
RuleParams.EnableDropTrustedFriendsTweetContentRuleParam -> DeciderKey.EnableDropTrustedFriendsTweetContentRule, RuleParams.EnableDropTrustedFriendsTweetContentRuleParam -> DeciderKey.EnableDropTrustedFriendsTweetContentRule,
RuleParams.EnableDropAllCollabInvitationTweetsRuleParam -> DeciderKey.EnableDropCollabInvitationTweetsRule, RuleParams.EnableDropAllCollabInvitationTweetsRuleParam -> DeciderKey.EnableDropCollabInvitationTweetsRule,
RuleParams.EnableNsfwTextTopicsDropRuleParam -> DeciderKey.EnableNsfwTextTopicsDropRule, RuleParams.EnableNsfwTextHighPrecisionDropRuleParam -> DeciderKey.EnableNsfwTextHighPrecisionDropRule,
RuleParams.EnableLikelyIvsUserLabelDropRule -> DeciderKey.EnableLikelyIvsUserLabelDropRule, RuleParams.EnableLikelyIvsUserLabelDropRule -> DeciderKey.EnableLikelyIvsUserLabelDropRule,
RuleParams.EnableCardUriRootDomainCardDenylistRule -> DeciderKey.EnableCardUriRootDomainDenylistRule, RuleParams.EnableCardUriRootDomainCardDenylistRule -> DeciderKey.EnableCardUriRootDomainDenylistRule,
RuleParams.EnableCommunityNonMemberPollCardRule -> DeciderKey.EnableCommunityNonMemberPollCardRule, RuleParams.EnableCommunityNonMemberPollCardRule -> DeciderKey.EnableCommunityNonMemberPollCardRule,

View File

@ -85,7 +85,7 @@ private[visibility] object RuleParams {
object EnableDropAllCollabInvitationTweetsRuleParam extends RuleParam(false) object EnableDropAllCollabInvitationTweetsRuleParam extends RuleParam(false)
object EnableNsfwTextTopicsDropRuleParam extends RuleParam(false) object EnableNsfwTextHighPrecisionDropRuleParam extends RuleParam(false)
object EnableLikelyIvsUserLabelDropRule extends RuleParam(false) object EnableLikelyIvsUserLabelDropRule extends RuleParam(false)

View File

@ -186,6 +186,7 @@ private[visibility] object SafetyLevelParams {
object EnableTopicRecommendationsSafetyLevelParam extends SafetyLevelParam(false) object EnableTopicRecommendationsSafetyLevelParam extends SafetyLevelParam(false)
object EnableTrendsRepresentativeTweetSafetyLevelParam extends SafetyLevelParam(false) object EnableTrendsRepresentativeTweetSafetyLevelParam extends SafetyLevelParam(false)
object EnableTrustedFriendsUserListSafetyLevelParam extends SafetyLevelParam(false) object EnableTrustedFriendsUserListSafetyLevelParam extends SafetyLevelParam(false)
object EnableTwitterDelegateUserListSafetyLevelParam extends SafetyLevelParam(false)
object EnableTweetDetailSafetyLevelParam extends SafetyLevelParam(false) object EnableTweetDetailSafetyLevelParam extends SafetyLevelParam(false)
object EnableTweetDetailNonTooSafetyLevelParam extends SafetyLevelParam(false) object EnableTweetDetailNonTooSafetyLevelParam extends SafetyLevelParam(false)
object EnableTweetDetailWithInjectionsHydrationSafetyLevelParam extends SafetyLevelParam(false) object EnableTweetDetailWithInjectionsHydrationSafetyLevelParam extends SafetyLevelParam(false)

View File

@ -143,7 +143,7 @@ class VisibilityRuleEngine private[VisibilityRuleEngine] (
builder.withRuleResult(rule, RuleResult(builder.verdict, ShortCircuited)) builder.withRuleResult(rule, RuleResult(builder.verdict, ShortCircuited))
} else { } else {
if (rule.fallbackActionBuilder.nonEmpty) { if (failedFeatureDependencies.nonEmpty && rule.fallbackActionBuilder.nonEmpty) {
metricsRecorder.recordRuleFallbackAction(rule.name) metricsRecorder.recordRuleFallbackAction(rule.name)
} }

View File

@ -194,6 +194,7 @@ object SafetyLevel {
ThriftSafetyLevel.TopicsLandingPageTopicRecommendations -> TopicsLandingPageTopicRecommendations, ThriftSafetyLevel.TopicsLandingPageTopicRecommendations -> TopicsLandingPageTopicRecommendations,
ThriftSafetyLevel.TrendsRepresentativeTweet -> TrendsRepresentativeTweet, ThriftSafetyLevel.TrendsRepresentativeTweet -> TrendsRepresentativeTweet,
ThriftSafetyLevel.TrustedFriendsUserList -> TrustedFriendsUserList, ThriftSafetyLevel.TrustedFriendsUserList -> TrustedFriendsUserList,
ThriftSafetyLevel.TwitterDelegateUserList -> TwitterDelegateUserList,
ThriftSafetyLevel.GryphonDecksAndColumns -> GryphonDecksAndColumns, ThriftSafetyLevel.GryphonDecksAndColumns -> GryphonDecksAndColumns,
ThriftSafetyLevel.TweetDetail -> TweetDetail, ThriftSafetyLevel.TweetDetail -> TweetDetail,
ThriftSafetyLevel.TweetDetailNonToo -> TweetDetailNonToo, ThriftSafetyLevel.TweetDetailNonToo -> TweetDetailNonToo,
@ -772,6 +773,9 @@ object SafetyLevel {
case object TrustedFriendsUserList extends SafetyLevel { case object TrustedFriendsUserList extends SafetyLevel {
override val enabledParam: SafetyLevelParam = EnableTrustedFriendsUserListSafetyLevelParam override val enabledParam: SafetyLevelParam = EnableTrustedFriendsUserListSafetyLevelParam
} }
case object TwitterDelegateUserList extends SafetyLevel {
override val enabledParam: SafetyLevelParam = EnableTwitterDelegateUserListSafetyLevelParam
}
case object TweetDetail extends SafetyLevel { case object TweetDetail extends SafetyLevel {
override val enabledParam: SafetyLevelParam = EnableTweetDetailSafetyLevelParam override val enabledParam: SafetyLevelParam = EnableTweetDetailSafetyLevelParam
} }

View File

@ -379,13 +379,6 @@ object SafetyLevelGroup {
) )
} }
case object ProfileMixer extends SafetyLevelGroup {
override val levels: Set[SafetyLevel] = Set(
ProfileMixerMedia,
ProfileMixerFavorites,
)
}
case object Reactions extends SafetyLevelGroup { case object Reactions extends SafetyLevelGroup {
override val levels: Set[SafetyLevel] = Set( override val levels: Set[SafetyLevel] = Set(
SignalsReactions, SignalsReactions,
@ -516,6 +509,10 @@ object SafetyLevelGroup {
SafetyLevel.TimelineProfile, SafetyLevel.TimelineProfile,
TimelineProfileAll, TimelineProfileAll,
TimelineProfileSpaces, TimelineProfileSpaces,
TimelineMedia,
ProfileMixerMedia,
TimelineFavorites,
ProfileMixerFavorites
) )
} }

View File

@ -36,8 +36,8 @@ object SpaceSafetyLabelType extends SafetyLabelType {
s.SpaceSafetyLabelType.HatefulHighRecall -> HatefulHighRecall, s.SpaceSafetyLabelType.HatefulHighRecall -> HatefulHighRecall,
s.SpaceSafetyLabelType.ViolenceHighRecall -> ViolenceHighRecall, s.SpaceSafetyLabelType.ViolenceHighRecall -> ViolenceHighRecall,
s.SpaceSafetyLabelType.HighToxicityModelScore -> HighToxicityModelScore, s.SpaceSafetyLabelType.HighToxicityModelScore -> HighToxicityModelScore,
s.SpaceSafetyLabelType.UkraineCrisisTopic -> UkraineCrisisTopic, s.SpaceSafetyLabelType.DeprecatedSpaceSafetyLabel14 -> Deprecated,
s.SpaceSafetyLabelType.DoNotPublicPublish -> DoNotPublicPublish, s.SpaceSafetyLabelType.DeprecatedSpaceSafetyLabel15 -> Deprecated,
s.SpaceSafetyLabelType.Reserved16 -> Deprecated, s.SpaceSafetyLabelType.Reserved16 -> Deprecated,
s.SpaceSafetyLabelType.Reserved17 -> Deprecated, s.SpaceSafetyLabelType.Reserved17 -> Deprecated,
s.SpaceSafetyLabelType.Reserved18 -> Deprecated, s.SpaceSafetyLabelType.Reserved18 -> Deprecated,
@ -69,10 +69,6 @@ object SpaceSafetyLabelType extends SafetyLabelType {
case object ViolenceHighRecall extends SpaceSafetyLabelType case object ViolenceHighRecall extends SpaceSafetyLabelType
case object HighToxicityModelScore extends SpaceSafetyLabelType case object HighToxicityModelScore extends SpaceSafetyLabelType
case object UkraineCrisisTopic extends SpaceSafetyLabelType
case object DoNotPublicPublish extends SpaceSafetyLabelType
case object Deprecated extends SpaceSafetyLabelType case object Deprecated extends SpaceSafetyLabelType
case object Unknown extends SpaceSafetyLabelType case object Unknown extends SpaceSafetyLabelType

View File

@ -3,6 +3,7 @@ package com.twitter.visibility.rules
import com.twitter.spam.rtf.thriftscala.SafetyResultReason import com.twitter.spam.rtf.thriftscala.SafetyResultReason
import com.twitter.util.Memoize import com.twitter.util.Memoize
import com.twitter.visibility.common.actions.AppealableReason import com.twitter.visibility.common.actions.AppealableReason
import com.twitter.visibility.common.actions.AvoidReason.MightNotBeSuitableForAds
import com.twitter.visibility.common.actions.LimitedEngagementReason import com.twitter.visibility.common.actions.LimitedEngagementReason
import com.twitter.visibility.common.actions.SoftInterventionDisplayType import com.twitter.visibility.common.actions.SoftInterventionDisplayType
import com.twitter.visibility.common.actions.SoftInterventionReason import com.twitter.visibility.common.actions.SoftInterventionReason
@ -440,36 +441,6 @@ object FreedomOfSpeechNotReachActions {
} }
} }
case class ConversationSectionAbusiveQualityAction(
violationLevel: ViolationLevel = DefaultViolationLevel)
extends FreedomOfSpeechNotReachActionBuilder[ConversationSectionAbusiveQuality.type] {
override def actionType: Class[_] = ConversationSectionAbusiveQuality.getClass
override val actionSeverity = 5
private def toRuleResult: Reason => RuleResult = Memoize { r =>
RuleResult(ConversationSectionAbusiveQuality, Evaluated)
}
def build(evaluationContext: EvaluationContext, featureMap: Map[Feature[_], _]): RuleResult = {
val appealableReason =
FreedomOfSpeechNotReach.extractTweetSafetyLabel(featureMap).map(_.labelType) match {
case Some(label) =>
FreedomOfSpeechNotReach.eligibleTweetSafetyLabelTypesToAppealableReason(
label,
violationLevel)
case _ =>
AppealableReason.Unspecified(violationLevel.level)
}
toRuleResult(Reason.fromAppealableReason(appealableReason))
}
override def withViolationLevel(violationLevel: ViolationLevel) = {
copy(violationLevel = violationLevel)
}
}
case class SoftInterventionAvoidAction(violationLevel: ViolationLevel = DefaultViolationLevel) case class SoftInterventionAvoidAction(violationLevel: ViolationLevel = DefaultViolationLevel)
extends FreedomOfSpeechNotReachActionBuilder[TweetInterstitial] { extends FreedomOfSpeechNotReachActionBuilder[TweetInterstitial] {
@ -662,6 +633,9 @@ object FreedomOfSpeechNotReachRules {
override def enabled: Seq[RuleParam[Boolean]] = override def enabled: Seq[RuleParam[Boolean]] =
Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam) Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
case class ViewerIsNonFollowerNonAuthorAndTweetHasViolationOfLevel( case class ViewerIsNonFollowerNonAuthorAndTweetHasViolationOfLevel(
@ -678,6 +652,9 @@ object FreedomOfSpeechNotReachRules {
override def enabled: Seq[RuleParam[Boolean]] = override def enabled: Seq[RuleParam[Boolean]] =
Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam) Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
case class ViewerIsNonAuthorAndTweetHasViolationOfLevel( case class ViewerIsNonAuthorAndTweetHasViolationOfLevel(
@ -692,6 +669,9 @@ object FreedomOfSpeechNotReachRules {
override def enabled: Seq[RuleParam[Boolean]] = override def enabled: Seq[RuleParam[Boolean]] =
Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam) Seq(EnableFosnrRuleParam, FosnrRulesEnabledParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
case object TweetHasViolationOfAnyLevelFallbackDropRule case object TweetHasViolationOfAnyLevelFallbackDropRule

View File

@ -188,6 +188,7 @@ object RuleBase {
TopicRecommendations -> TopicRecommendationsPolicy, TopicRecommendations -> TopicRecommendationsPolicy,
TrendsRepresentativeTweet -> TrendsRepresentativeTweetPolicy, TrendsRepresentativeTweet -> TrendsRepresentativeTweetPolicy,
TrustedFriendsUserList -> TrustedFriendsUserListPolicy, TrustedFriendsUserList -> TrustedFriendsUserListPolicy,
TwitterDelegateUserList -> TwitterDelegateUserListPolicy,
TweetDetail -> TweetDetailPolicy, TweetDetail -> TweetDetailPolicy,
TweetDetailNonToo -> TweetDetailNonTooPolicy, TweetDetailNonToo -> TweetDetailNonTooPolicy,
TweetDetailWithInjectionsHydration -> TweetDetailWithInjectionsHydrationPolicy, TweetDetailWithInjectionsHydration -> TweetDetailWithInjectionsHydrationPolicy,

View File

@ -144,6 +144,9 @@ object NsfwCardImageAvoidAllUsersTweetLabelRule
action = Avoid(Some(AvoidReason.ContainsNsfwMedia)), action = Avoid(Some(AvoidReason.ContainsNsfwMedia)),
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object NsfwCardImageAvoidAdPlacementAllUsersTweetLabelRule object NsfwCardImageAvoidAdPlacementAllUsersTweetLabelRule
@ -247,6 +250,9 @@ object GoreAndViolenceHighPrecisionAvoidAllUsersTweetLabelRule
TweetSafetyLabelType.GoreAndViolenceHighPrecision TweetSafetyLabelType.GoreAndViolenceHighPrecision
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object GoreAndViolenceHighPrecisionAllUsersTweetLabelRule object GoreAndViolenceHighPrecisionAllUsersTweetLabelRule
@ -266,6 +272,9 @@ object NsfwReportedHeuristicsAvoidAllUsersTweetLabelRule
TweetSafetyLabelType.NsfwReportedHeuristics TweetSafetyLabelType.NsfwReportedHeuristics
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object NsfwReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule object NsfwReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule
@ -274,6 +283,9 @@ object NsfwReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule
TweetSafetyLabelType.NsfwReportedHeuristics TweetSafetyLabelType.NsfwReportedHeuristics
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object NsfwReportedHeuristicsAllUsersTweetLabelRule object NsfwReportedHeuristicsAllUsersTweetLabelRule
@ -294,6 +306,9 @@ object GoreAndViolenceReportedHeuristicsAvoidAllUsersTweetLabelRule
TweetSafetyLabelType.GoreAndViolenceReportedHeuristics TweetSafetyLabelType.GoreAndViolenceReportedHeuristics
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object GoreAndViolenceReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule object GoreAndViolenceReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule
@ -302,6 +317,9 @@ object GoreAndViolenceReportedHeuristicsAvoidAdPlacementAllUsersTweetLabelRule
TweetSafetyLabelType.GoreAndViolenceReportedHeuristics TweetSafetyLabelType.GoreAndViolenceReportedHeuristics
) { ) {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableAvoidNsfwRulesParam)
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
} }
object GoreAndViolenceHighPrecisionAllUsersTweetLabelDropRule object GoreAndViolenceHighPrecisionAllUsersTweetLabelDropRule
@ -791,7 +809,7 @@ object SkipTweetDetailLimitedEngagementTweetLabelRule
object DynamicProductAdDropTweetLabelRule object DynamicProductAdDropTweetLabelRule
extends TweetHasLabelRule(Drop(Unspecified), TweetSafetyLabelType.DynamicProductAd) extends TweetHasLabelRule(Drop(Unspecified), TweetSafetyLabelType.DynamicProductAd)
object NsfwTextTweetLabelTopicsDropRule object NsfwTextHighPrecisionTweetLabelDropRule
extends RuleWithConstantAction( extends RuleWithConstantAction(
Drop(Reason.Nsfw), Drop(Reason.Nsfw),
And( And(
@ -803,7 +821,7 @@ object NsfwTextTweetLabelTopicsDropRule
) )
) )
with DoesLogVerdict { with DoesLogVerdict {
override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableNsfwTextTopicsDropRuleParam) override def enabled: Seq[RuleParam[Boolean]] = Seq(EnableNsfwTextHighPrecisionDropRuleParam)
override def actionSourceBuilder: Option[RuleActionSourceBuilder] = Some( override def actionSourceBuilder: Option[RuleActionSourceBuilder] = Some(
TweetSafetyLabelSourceBuilder(TweetSafetyLabelType.NsfwTextHighPrecision)) TweetSafetyLabelSourceBuilder(TweetSafetyLabelType.NsfwTextHighPrecision))
} }
@ -832,7 +850,10 @@ object DoNotAmplifyTweetLabelAvoidRule
extends TweetHasLabelRule( extends TweetHasLabelRule(
Avoid(), Avoid(),
TweetSafetyLabelType.DoNotAmplify TweetSafetyLabelType.DoNotAmplify
) ) {
override val fallbackActionBuilder: Option[ActionBuilder[_ <: Action]] = Some(
new ConstantActionBuilder(Avoid(Some(MightNotBeSuitableForAds))))
}
object NsfaHighPrecisionTweetLabelAvoidRule object NsfaHighPrecisionTweetLabelAvoidRule
extends TweetHasLabelRule( extends TweetHasLabelRule(

View File

@ -776,7 +776,10 @@ case object MagicRecsPolicy
tweetRules = MagicRecsPolicyOverrides.union( tweetRules = MagicRecsPolicyOverrides.union(
RecommendationsPolicy.tweetRules.filterNot(_ == SafetyCrisisLevel3DropRule), RecommendationsPolicy.tweetRules.filterNot(_ == SafetyCrisisLevel3DropRule),
NotificationsIbisPolicy.tweetRules, NotificationsIbisPolicy.tweetRules,
Seq(NsfaHighRecallTweetLabelRule, NsfwHighRecallTweetLabelRule), Seq(
NsfaHighRecallTweetLabelRule,
NsfwHighRecallTweetLabelRule,
NsfwTextHighPrecisionTweetLabelDropRule),
Seq( Seq(
AuthorBlocksViewerDropRule, AuthorBlocksViewerDropRule,
ViewerBlocksAuthorRule, ViewerBlocksAuthorRule,
@ -1171,7 +1174,7 @@ case object ReturningUserExperiencePolicy
NsfwHighRecallTweetLabelRule, NsfwHighRecallTweetLabelRule,
NsfwVideoTweetLabelDropRule, NsfwVideoTweetLabelDropRule,
NsfwTextTweetLabelDropRule, NsfwTextTweetLabelDropRule,
NsfwTextTweetLabelTopicsDropRule, NsfwTextHighPrecisionTweetLabelDropRule,
SpamHighRecallTweetLabelDropRule, SpamHighRecallTweetLabelDropRule,
DuplicateContentTweetLabelDropRule, DuplicateContentTweetLabelDropRule,
GoreAndViolenceTweetLabelRule, GoreAndViolenceTweetLabelRule,
@ -1785,6 +1788,14 @@ case object TimelineListsPolicy
NsfwReportedHeuristicsAllUsersTweetLabelRule, NsfwReportedHeuristicsAllUsersTweetLabelRule,
GoreAndViolenceReportedHeuristicsAllUsersTweetLabelRule, GoreAndViolenceReportedHeuristicsAllUsersTweetLabelRule,
NsfwCardImageAllUsersTweetLabelRule, NsfwCardImageAllUsersTweetLabelRule,
NsfwHighPrecisionTweetLabelAvoidRule,
NsfwHighRecallTweetLabelAvoidRule,
GoreAndViolenceHighPrecisionAvoidAllUsersTweetLabelRule,
NsfwReportedHeuristicsAvoidAllUsersTweetLabelRule,
GoreAndViolenceReportedHeuristicsAvoidAllUsersTweetLabelRule,
NsfwCardImageAvoidAllUsersTweetLabelRule,
DoNotAmplifyTweetLabelAvoidRule,
NsfaHighPrecisionTweetLabelAvoidRule,
) ++ LimitedEngagementBaseRules.tweetRules ) ++ LimitedEngagementBaseRules.tweetRules
) )
@ -2132,7 +2143,13 @@ case object TimelineHomePolicy
userRules = Seq( userRules = Seq(
ViewerMutesAuthorRule, ViewerMutesAuthorRule,
ViewerBlocksAuthorRule, ViewerBlocksAuthorRule,
DeciderableAuthorBlocksViewerDropRule DeciderableAuthorBlocksViewerDropRule,
ProtectedAuthorDropRule,
SuspendedAuthorRule,
DeactivatedAuthorRule,
ErasedAuthorRule,
OffboardedAuthorRule,
DropTakendownUserRule
), ),
policyRuleParams = SensitiveMediaSettingsTimelineHomeBaseRules.policyRuleParams policyRuleParams = SensitiveMediaSettingsTimelineHomeBaseRules.policyRuleParams
) )
@ -2171,7 +2188,13 @@ case object BaseTimelineHomePolicy
userRules = Seq( userRules = Seq(
ViewerMutesAuthorRule, ViewerMutesAuthorRule,
ViewerBlocksAuthorRule, ViewerBlocksAuthorRule,
DeciderableAuthorBlocksViewerDropRule DeciderableAuthorBlocksViewerDropRule,
ProtectedAuthorDropRule,
SuspendedAuthorRule,
DeactivatedAuthorRule,
ErasedAuthorRule,
OffboardedAuthorRule,
DropTakendownUserRule
) )
) )
@ -2255,7 +2278,13 @@ case object TimelineHomeLatestPolicy
userRules = Seq( userRules = Seq(
ViewerMutesAuthorRule, ViewerMutesAuthorRule,
ViewerBlocksAuthorRule, ViewerBlocksAuthorRule,
DeciderableAuthorBlocksViewerDropRule DeciderableAuthorBlocksViewerDropRule,
ProtectedAuthorDropRule,
SuspendedAuthorRule,
DeactivatedAuthorRule,
ErasedAuthorRule,
OffboardedAuthorRule,
DropTakendownUserRule
), ),
policyRuleParams = SensitiveMediaSettingsTimelineHomeBaseRules.policyRuleParams policyRuleParams = SensitiveMediaSettingsTimelineHomeBaseRules.policyRuleParams
) )
@ -3283,7 +3312,7 @@ case object TopicRecommendationsPolicy
tweetRules = tweetRules =
Seq( Seq(
NsfwHighRecallTweetLabelRule, NsfwHighRecallTweetLabelRule,
NsfwTextTweetLabelTopicsDropRule NsfwTextHighPrecisionTweetLabelDropRule
) )
++ RecommendationsPolicy.tweetRules, ++ RecommendationsPolicy.tweetRules,
userRules = RecommendationsPolicy.userRules userRules = RecommendationsPolicy.userRules
@ -3536,6 +3565,17 @@ case object TrustedFriendsUserListPolicy
) )
) )
case object TwitterDelegateUserListPolicy
extends VisibilityPolicy(
userRules = Seq(
ViewerBlocksAuthorRule,
ViewerIsAuthorDropRule,
DeactivatedAuthorRule,
AuthorBlocksViewerDropRule
),
tweetRules = Seq(DropAllRule)
)
case object QuickPromoteTweetEligibilityPolicy case object QuickPromoteTweetEligibilityPolicy
extends VisibilityPolicy( extends VisibilityPolicy(
tweetRules = TweetDetailPolicy.tweetRules, tweetRules = TweetDetailPolicy.tweetRules,

View File

@ -100,30 +100,6 @@ object TweetRuleGenerator {
FreedomOfSpeechNotReachActions.SoftInterventionAvoidLimitedEngagementsAction( FreedomOfSpeechNotReachActions.SoftInterventionAvoidLimitedEngagementsAction(
limitedActionStrings = Some(level3LimitedActions)) limitedActionStrings = Some(level3LimitedActions))
) )
.addSafetyLevelRule(
SafetyLevel.TimelineMedia,
FreedomOfSpeechNotReachActions
.SoftInterventionAvoidLimitedEngagementsAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.ProfileMixerMedia,
FreedomOfSpeechNotReachActions
.SoftInterventionAvoidLimitedEngagementsAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.TimelineFavorites,
FreedomOfSpeechNotReachActions
.SoftInterventionAvoidLimitedEngagementsAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.ProfileMixerFavorites,
FreedomOfSpeechNotReachActions
.SoftInterventionAvoidLimitedEngagementsAction(limitedActionStrings =
Some(level3LimitedActions))
)
.build, .build,
UserType.Author -> TweetVisibilityPolicy UserType.Author -> TweetVisibilityPolicy
.builder() .builder()
@ -159,30 +135,6 @@ object TweetRuleGenerator {
.InterstitialLimitedEngagementsAvoidAction(limitedActionStrings = .InterstitialLimitedEngagementsAvoidAction(limitedActionStrings =
Some(level3LimitedActions)) Some(level3LimitedActions))
) )
.addSafetyLevelRule(
SafetyLevel.TimelineMedia,
FreedomOfSpeechNotReachActions
.InterstitialLimitedEngagementsAvoidAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.ProfileMixerMedia,
FreedomOfSpeechNotReachActions
.InterstitialLimitedEngagementsAvoidAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.TimelineFavorites,
FreedomOfSpeechNotReachActions
.InterstitialLimitedEngagementsAvoidAction(limitedActionStrings =
Some(level3LimitedActions))
)
.addSafetyLevelRule(
SafetyLevel.ProfileMixerFavorites,
FreedomOfSpeechNotReachActions
.InterstitialLimitedEngagementsAvoidAction(limitedActionStrings =
Some(level3LimitedActions))
)
.build, .build,
), ),
) )