How to set up local mongo database

Prerequisite

• Docker (macOS)
• Mongo shell command tool (instruction)
• Access to exchange-mongodb-backups-prd
• Google cloud SDK(https://cloud.google.com/sdk/install)

How to set up local mongoDB

• Stop all running container in Docker using

docker kill $(docker ps -q)

• Run mongoDB using Docker

docker run -d --name anyName --restart=always -p 27017:27017 mongo

• Download the latest backup data from here copy the URI of the file and use the following command to download it

gsutil cp gs://exchange-mongodb-backups-prd/ads-exchange-creative/ads-exchange-creative-1595548800.gz .

• Resotre downloaded gzip file to local mongoDB

mongorestore --gzip --archive=${BACKUP_FILE_GZ} --nsFrom “${DB_NAME}.*” --nsTo “${DB_NAME_RESTORE}.*”

for example

mongorestore --gzip --archive="ads-exchange-creative-1595548800.gz" --nsFrom "creatives" --nsTo "creatives"

How to use the script

After local mongoDB is installed and restored data from the downloaded file, using python3 to run the python script

python3 xxxx.py

There will be a progress bar to show how much it has done for now, usually it will take 10 minutes to finish. After it prints out the following info, the local database is ready to use

What does the script do?

• The script iterates all files in advertiser collection and add every files (original creative) to a new collection called allCreatives. Original creatives from advertiser collection stored in allCreatives in the following format

{
	"_id" : ObjectId("5f1a666a1c202f0d934dbfcf"),
	"creativeId" : "5909c6fd6bdc2a00115c2a75",
	"checksum" : "",
	"status" : "processed",
	"moderationStatus" : "approved",
	"organizationId" : "56cdf719b209521800a91f33",
	"gameId" : 500000004,
	"timeStamp" : "2019-12-03::12-02"
}

• After the above process, the script iterates all creatives in creatives collection and add every creative to allCreative Creatives from creatives collection stored in allCreative in the following format

{
	"_id" : ObjectId("5f1a666a1c202f0d934dbfcf"),
	"creativeId" : "5909c6fd6bdc2a00115c2a75",
	"checksum" : "",
	"status" : "SUCESS",
	"decision" : "GO",
	"organizationId" : "56cdf719b209521800a91f33",
	"gameId" : 500000004,
}