Skip to content

BurnerWah/the-algorithm

This branch is 1 commit ahead of twitter/the-algorithm:main.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Jul 26, 2023
3608998 · Jul 26, 2023

History

31 Commits
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023
Mar 31, 2023
Jul 26, 2023
Jul 26, 2023
Jul 26, 2023

Repository files navigation

twittew's wecommendation awgowithm

t-twittew's wecommendation a-awgowithm i-is a set o-of sewvices and j-jobs that awe wesponsibwe f-fow sewving f-feeds of t-tweets and othew content acwoss aww twittew pwoduct suwfaces (e.g. ^^ fow you timewine, ^•ﻌ•^ s-seawch, expwowe, XD nyotifications). :3 fow an intwoduction t-to how the awgowithm w-wowks, (ꈍᴗꈍ) pwease wefew to ouw engineewing bwog. :3

awchitectuwe

p-pwoduct suwfaces at twittew a-awe buiwt on a s-shawed set of data, (U ﹏ U) modews, and softwawe fwamewowks. UwU the shawed components incwuded i-in this wepositowy awe wisted bewow:

type component descwiption
d-data tweetypie cowe tweet s-sewvice that h-handwes the weading a-and wwiting o-of tweet data. 😳😳😳
unified-usew-actions weaw-time stweam of usew actions o-on twittew. XD
usew-signaw-sewvice centwawized pwatfowm to wetwieve e-expwicit (e.g. wikes, o.O wepwies) and impwicit (e.g. (⑅˘꒳˘) pwofiwe visits, 😳😳😳 tweet cwicks) usew signaws. nyaa~~
m-modew simcwustews community detection a-and spawse embeddings i-into those c-communities. rawr
twhin dense knowwedge gwaph embeddings fow usews a-and tweets. -.-
twust-and-safety-modews m-modews fow detecting nysfw o-ow abusive content. (✿oωo)
weaw-gwaph m-modew to pwedict the wikewihood o-of a twittew usew intewacting w-with anothew usew. /(^•ω•^)
tweepcwed page-wank a-awgowithm fow cawcuwating twittew u-usew weputation. 🥺
wecos-injectow stweaming event pwocessow f-fow buiwding i-input stweams fow gwaphjet based sewvices.
gwaph-featuwe-sewvice sewves gwaph featuwes fow a diwected paiw of usews (e.g. how many o-of usew a's fowwowing w-wiked tweets fwom usew b). ʘwʘ
topic-sociaw-pwoof i-identifies topics w-wewated to i-individuaw tweets. UwU
wepwesentation-scowew compute scowes between paiws of entities (usews, XD t-tweets, (✿oωo) etc.) using embedding simiwawity. :3
softwawe fwamewowk navi h-high pewfowmance, (///ˬ///✿) machine w-weawning modew s-sewving wwitten i-in wust. nyaa~~
pwoduct-mixew softwawe f-fwamewowk f-fow buiwding feeds o-of content. >w<
timewines-aggwegation-fwamewowk fwamewowk f-fow genewating aggwegate featuwes in batch o-ow weaw time. -.-
wepwesentation-managew s-sewvice t-to wetwieve embeddings (i.e. (✿oωo) s-simcwusews a-and twhin). (˘ω˘)
twmw wegacy machine weawning fwamewowk buiwt on t-tensowfwow v1.

the pwoduct suwfaces cuwwentwy incwuded in this wepositowy awe the fow you timewine a-and wecommended nyotifications. rawr

fow you timewine

the d-diagwam bewow i-iwwustwates how m-majow sewvices and jobs intewconnect t-to constwuct a fow you timewine.

t-the cowe components o-of the fow you timewine incwuded in this wepositowy awe wisted bewow:

type component d-descwiption
candidate s-souwce seawch-index find and w-wank in-netwowk t-tweets. ~50% of tweets come fwom this candidate s-souwce. OwO
cw-mixew c-coowdination wayew fow fetching o-out-of-netwowk t-tweet candidates fwom undewwying compute sewvices. ^•ﻌ•^
usew-tweet-entity-gwaph (uteg) maintains an in memowy u-usew to tweet i-intewaction gwaph, UwU a-and finds candidates based on t-twavewsaws of t-this gwaph. (˘ω˘) this is buiwt on the gwaphjet f-fwamewowk. (///ˬ///✿) sevewaw othew gwaphjet based featuwes and candidate souwces awe wocated hewe. σωσ
fowwow-wecommendation-sewvice (fws) p-pwovides usews w-with wecommendations fow accounts to fowwow, /(^•ω•^) and t-tweets fwom those a-accounts. 😳
wanking wight-wankew wight wankew modew used by seawch i-index (eawwybiwd) to wank tweets. 😳
heavy-wankew nyeuwaw nyetwowk fow wanking candidate tweets. (⑅˘꒳˘) o-one of the main signaws used to sewect timewine t-tweets post c-candidate souwcing.
tweet mixing & fiwtewing home-mixew main sewvice u-used to constwuct a-and sewve the home timewine. 😳😳😳 buiwt on pwoduct-mixew. 😳
visibiwity-fiwtews wesponsibwe f-fow fiwtewing twittew content t-to suppowt wegaw compwiance, XD impwove pwoduct quawity, mya incwease u-usew twust, ^•ﻌ•^ pwotect wevenue thwough t-the use of h-hawd-fiwtewing, ʘwʘ visibwe pwoduct t-tweatments, ( ͡o ω ͡o ) and coawse-gwained downwanking. mya
timewinewankew w-wegacy sewvice w-which pwovides w-wewevance-scowed tweets fwom t-the eawwybiwd s-seawch index and uteg sewvice. o.O

wecommended n-nyotifications

t-the cowe components o-of wecommended nyotifications incwuded in t-this wepositowy awe wisted bewow:

t-type component d-descwiption
sewvice pushsewvice main wecommendation sewvice at t-twittew used to s-suwface wecommendations t-to ouw u-usews via nyotifications. (✿oωo)
wanking pushsewvice-wight-wankew w-wight wankew modew used by pushsewvice to wank tweets. :3 bwidges candidate genewation and heavy wanking b-by pwe-sewecting highwy-wewevant c-candidates fwom the initiaw h-huge candidate poow. 😳
pushsewvice-heavy-wankew muwti-task w-weawning modew to pwedict t-the pwobabiwities t-that the tawget u-usews wiww o-open and engage w-with the sent nyotifications. (U ﹏ U)

buiwd and test code

we incwude bazew buiwd fiwes fow most components, mya but nyot a top-wevew b-buiwd ow wowkspace f-fiwe. (U ᵕ U❁) we pwan t-to add a mowe compwete buiwd and t-test system in the futuwe. :3

contwibuting

we invite the community t-to submit g-github issues and puww wequests f-fow suggestions on impwoving the wecommendation a-awgowithm. mya we a-awe wowking on toows to manage these s-suggestions a-and sync changes to ouw intewnaw wepositowy. any secuwity concewns ow issues shouwd b-be wouted to o-ouw officiaw bug b-bounty pwogwam t-thwough hackewone. w-we hope to benefit fwom the c-cowwective intewwigence a-and expewtise of the gwobaw c-community i-in hewping us identify issues and s-suggest impwovements, OwO uwtimatewy weading to a b-bettew twittew. (ˆ ﻌ ˆ)♡

wead ouw bwog on t-the open souwce i-initiative hewe. ʘwʘ

Releases

No releases published

Packages

No packages published

Languages

  • Scala 62.5%
  • Java 21.9%
  • Starlark 6.0%
  • Python 4.1%
  • Thrift 2.4%
  • C++ 1.7%
  • Other 1.4%