[Bug]: Logging does not update progress bar

[francois-rozet]

Describe the bug

The logs interface does not update tqdm progress bars.

Reproducing

With the following code,

import wandb
import tqdm
import time

run = wandb.init(project="tqdm-test")

for i in tqdm.tqdm(range(1000), ncols=88, ascii=True):
    run.log({"acc": i / 1000})
    time.sleep(10)

run.finish()

the progress bar looks like

  1%|6                                              | 13/1000 [02:10<2:44:31, 10.00s/it]

But the logs tab in the web interface always shows

1   0%|                                                          | 0/1000 [00:00<?, ?it/s] 

See https://wandb.ai/francois-rozet/tqdm-test/runs/rlqhqf9v/logs

[tcapelle]

This is weird, I remember I faced some of this back then. Have you tried printing something every iteration just to check?

[francois-rozet]

I just added a print(i) at each iteration. The progress bar still does not update (even though it does in my code).

1   0%|                                                          | 0/1000 [00:00<?, ?it/s]
2 0
3 1
4 2
5 3 

See https://wandb.ai/francois-rozet/tqdm-test/runs/4xekt4ma/logs

[ArtsiomWB]

Hey guys! Thank you for writing in.

I can repro this on my end as well.

I can check with our engineering team if this behavior is intentional or not. Some time ago, we had users writing in, asking to not display tqdm logs in the logs tab, because it created a lot of noise. I'll get back to you once I have an answer.

[ArtsiomWB]

Looks like this is intentional. Would you like me to send the feedback of seeing log by log coming from tqdm instead of just one line?

[francois-rozet]

The issue is not that it displays on a single line. The issue is that it does not update. My guess is that carriage returns (\r) are parsed incorrectly. The string <line1>\r<line2>\r<line3> printed by tqdm is likely truncated to <line1>, instead of <line3>.

[ArtsiomWB]

Gotcha, a couple of things to try here.

Could you please upgrade to the latest version of wandb by calling pip install wandb --upgrade, and then set the os.environ["WANDB_CONSOLE"] = "wrap" env var. After the two steps, I see that the tqdm shows up as 100 in my wandb logs.

[francois-rozet]

I updated to the latest version (19.6) and ran this code

import os

os.environ["WANDB_CONSOLE"] = "wrap"

import wandb
import tqdm
import time

run = wandb.init(project="tqdm-test")

for i in tqdm.tqdm(range(1000), ncols=88, ascii=True):
    run.log({"acc": i / 1000})
    time.sleep(1)

run.finish()

which produces

  9%|####5                                            | 93/1000 [01:33<15:13,  1.01s/it]

Unfortunately, this also gets stuck in W&B's web interface.

1   2%|7                                                | 15/1000 [00:15<16:26,  1.00s/it]

See https://wandb.ai/francois-rozet/tqdm-test/runs/jsy47jkc/logs

[ArtsiomWB]

Gotcha,

I see what you are talking about:

I'll bring this up to my engineering team. Thank you for reporting this.

[maxploter]

The issue causes an inconvenience, Is there any workaround?

[ArtsiomWB]

Hey Max, the fix for this should be setting the os.environ["WANDB_CONSOLE"] = "wrap" env var, but I am not seeing this helping on my end either. I'll re-escalate this concern to our SDK team.