Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PackerUpdate UI freezes in iTerm2 on MacOS #202

Open
gwerbin opened this issue Feb 8, 2021 · 69 comments
Open

PackerUpdate UI freezes in iTerm2 on MacOS #202

gwerbin opened this issue Feb 8, 2021 · 69 comments

Comments

@gwerbin
Copy link
Contributor

gwerbin commented Feb 8, 2021

PackerUpdate and PackerSync cause the Packer window to freeze when running within iTerm 2 on MacOS. It's unclear if the operation behind the UI completed successfully and the UI just crashed/froze, or if the underlying operation also crashed/froze. The only warning/error is a message that redrawtime had been exceeded. I tried setting redrawtime to something really high, like 20000, but I still get the error (and it triggers in less than 20 seconds).

When running the same update command in Neovim-Qt on the same machine, it works fine and the Packer window says "Everything up to date", although it takes several seconds to render any text in the Packer window at all. I do not get the redrawtime warning under Neovim-Qt.

The frozen update screen looks something like this:

           packer.nvim - updating 70 / 143 plugins           
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 
 ⟳ co1ncidence/sunset.vim: checking current commit...
 ⟳ zefei/simple-dark: checking current commit...
 ⟳ nvim-telescope/telescope-dap.nvim: checking current commit...
 ⟳ vim-scripts/pyte: checking current commit...
 ⟳ tomtom/tcomment_vim: checking current commit...
 ⟳ vim-scripts/proton: checking current commit...
 ⟳ wellle/targets.vim: checking current commit...
 ⟳ nvim-lua/popup.nvim: checking current commit...
 ⟳ vim-airline/vim-airline-themes: checking current commit...
 ⟳ nvim-telescope/telescope.nvim: checking current commit...
 ⟳ nvim-lua/plenary.nvim: checking current commit...
 ⟳ vim-airline/vim-airline: checking current commit...
 ⟳ axvr/photon.vim: checking current commit...
 ⟳ vim-scripts/phd: checking current commit...
 ⟳ lifepillar/pgsql.vim: checking current commit...
 ⟳ drewtempelmeyer/palenight.vim: checking current commit...
 ⟳ wbthomason/packer.nvim: checking current commit...
 ⟳ Th3Whit3Wolf/onebuddy: checking current commit...
 ⟳ Th3Whit3Wolf/one-nvim: checking current commit...
 ⟳ mhartington/oceanic-next: checking current commit...
 ⟳ nvim-treesitter/nvim-treesitter-textobjects: checking current commit...
 ⟳ nvim-treesitter/nvim-treesitter: checking current commit...
 ⟳ neovim/nvim-lspconfig: checking current commit...
 ⟳ theHamsta/nvim-dap-virtual-text: checking current commit...
 ⟳ mfussenegger/nvim-dap-python: checking current commit...
 ⟳ norcalli/nvim-colorizer.lua: checking current commit...
 ⟳ arcticicestudio/nord-vim: checking current commit...
 ⟳ zah/nim.vim: checking current commit...
 ⟳ preservim/nerdtree: checking current commit...
 ⟳ kassio/neoterm: checking current commit...
 ⟳ adelarsq/neoline.vim: checking current commit...
 ⟳ co1ncidence/mountaineer.vim: checking current commit...
 ⟳ co1ncidence/monokai-pro.vim: checking current commit...
 ⟳ wfxr/minimap.vim: checking current commit...
 ⟳ glepnir/lspsaga.nvim: checking current commit...
 ⟳ itchyny/lightline.vim: checking current commit...
 ⟳ JuliaEditorSupport/julia-vim: checking current commit...
 ⟳ vito-c/jq.vim: checking current commit...
 ⟳ nanotech/jellybeans.vim: checking current commit...
 ⟳ co1ncidence/javacafe.vim: checking current commit...
 ⟳ co1ncidence/itai.vim: checking current commit...
 ⟳ hkupty/iron.nvim: checking current commit...
 ⟳ Yggdroot/indentLine: checking current commit...
 ⟳ cocopon/iceberg.vim: checking current commit...
 ⟳ othree/html5.vim: checking current commit...
 ⟳ mfussenegger/nvim-dap: checking current commit...
 ⟳ YorickPeterse/happy_hacking.vim: checking current commit...
 ⟳ NieTiger/halcyon-neovim: checking current commit...
 ⟳ co1ncidence/gunmetal.vim: checking current commit...
 ⟳ sjl/gundo.vim: checking current commit...
 ⟳ npxbr/glow.nvim: checking current commit...
 ⟳ rhysd/git-messenger.vim: checking current commit...
 ⟳ jaredgorski/fogbell.vim: checking current commit...
 ⟳ bakpakin/fennel.vim: checking current commit...
 ⟳ editorconfig/editorconfig-vim: checking current commit...
 ⟳ dracula/vim: checking current commit...
 ⟳ jsit/disco.vim: checking current commit...
 ⟳ Shougo/denite.nvim: checking current commit...
 ⟳ nightsense/cosmic_latte: checking current commit...
 ⟳ nvim-lua/completion-nvim: checking current commit...
 ⟳ tjdevries/colorbuddy.vim: checking current commit...
 ⟳ ms-jpq/chadtree: checking current commit...
 ⟳ co1ncidence/bliss: checking current commit...
 ⟳ chriskempson/base16-vim: checking current commit...
 ⟳ ayu-theme/ayu-vim: checking current commit...
 ⟳ zacanger/angr.vim: checking current commit...
 ⟳ jdsimcoe/abstract.vim: checking current commit...
 ⟳ tmhedberg/SimpylFold: checking current commit...
 ⟳ kreskij/Repeatable.vim: checking current commit...
 ✓ vim-scripts/yowish: already up to date
 ✓ chrisbra/NrrwRgn: already up to date
 ✓ HerringtonDarkholme/yats.vim: already up to date
 ✓ overcache/NeoSolarized: already up to date
 ✓ othree/yajs.vim: already up to date
 ✓ Konfekt/FastFold: already up to date
 ✓ vlime/vlime: already up to date
 ✓ romainl/Apprentice: already up to date
 ✓ wellle/visual-split.vim: already up to date
 ✓ liuchengxu/vista.vim: already up to date
 ✓ lervag/vimtex: already up to date
 ✓ nightsense/vimspectr: already up to date
 ✓ chaoren/vim-wordmotion: already up to date
 ✓ wesQ3/vim-windowswap: already up to date
 ✓ tpope/vim-unimpaired: already up to date
 ✓ cespare/vim-toml: already up to date
 ✓ mswift42/vim-themes: already up to date
 ✓ kana/vim-textobj-user: already up to date
 ✓ kana/vim-textobj-indent: already up to date
 ✓ tpope/vim-surround: already up to date
 ✓ dstein64/vim-startuptime: already up to date
 ✓ mhinz/vim-startify: already up to date
 ✓ christoomey/vim-sort-motion: already up to date
 ✓ justinmk/vim-sneak: already up to date
 ✓ voldikss/vim-skylight: already up to date
 ✓ mhinz/vim-signify: already up to date
 ✓ kshenoy/vim-signature: already up to date
 ✓ tpope/vim-scriptease: already up to date
 ✓ tpope/vim-repeat: already up to date
 ✓ dhruvasagar/vim-prosession: already up to date
 ✓ junegunn/vim-peekaboo: already up to date
 ✓ rakr/vim-one: already up to date
 ✓ tpope/vim-obsession: already up to date
 ✓ noahfrederick/vim-noctu: already up to date
 ✓ LnL7/vim-nix: already up to date
 ✓ haya14busa/vim-metarepeat: already up to date
 ✓ jonathanfilip/vim-lucius: already up to date
 ✓ embear/vim-localvimrc: already up to date
 ✓ tommcdo/vim-lion: already up to date
 ✓ noahfrederick/vim-hemisu: already up to date
 ✓ jparise/vim-graphql: already up to date
 ✓ airblade/vim-gitgutter: already up to date
 ✓ tpope/vim-fugitive: already up to date
 ✓ voldikss/vim-floaterm: already up to date
 ✓ tommcdo/vim-exchange: already up to date
 ✓ tpope/vim-eunuch: already up to date
 ✓ Olical/vim-enmasse: already up to date
 ✓ elixir-editors/vim-elixir: already up to date
 ✓ easymotion/vim-easymotion: already up to date
 ✓ jeffkreeftmeijer/vim-dim: already up to date
 ✓ ap/vim-css-color: already up to date
 ✓ rbong/vim-crystalline: already up to date
 ✓ altercation/vim-colors-solarized: already up to date
 ✓ reedes/vim-colors-pencil: already up to date
 ✓ habamax/vim-colors-defminus: already up to date
 ✓ gyim/vim-boxdraw: already up to date
 ✓ benknoble/vim-auto-origami: already up to date
 ✓ Nequo/vim-allomancer: already up to date
 ✓ preservim/tagbar: already up to date
 ✓ kovetskiy/sxhkd-vim: already up to date
 ✓ inkarkat/vim-ingo-library: already up to date
 ✓ inkarkat/vim-AdvancedDiffOptions: already up to date
 ✓ srcery-colors/srcery-vim: already up to date
 ✓ chrisbra/unicode.vim: already up to date
 ✓ glepnir/spaceline.vim: already up to date
 ✓ mbbill/undotree: already up to date
 ✓ Th3Whit3Wolf/space-nvim: already up to date
 ✓ logico/typewriter: already up to date
 ✓ xero/sourcerer.vim: already up to date
 ✓ markonm/traces.vim: already up to date
 ✓ vim-scripts/sorcerer: already up to date
 ✓ jacoborus/tender.vim: already up to date
 ✓ kovisoft/slimv: already up to date
@wbthomason
Copy link
Owner

Thanks for the report! Unfortunately, my ability to debug this one is next to nil - I don't have access to a MacOS environment.

Did this start recently? In particular, had you updated packer or neovim shortly before first noticing this issue?

I'm almost wondering if this is an upstream bug. It sounds like it could be a problem with Neovim's rendering.

The redrawtime warning could point to an issue with syntax highlighting being inappropriately applied in the packer window, but that would seemingly trigger on other platforms too.

@akinsho
Copy link
Collaborator

akinsho commented Feb 8, 2021

Not sure if iTerm2 is important here but I run packer on macOS and Linux and haven't been having any issues running PackerSync or PackerUpdate. My build is NVIM v0.5.0-dev+nightly-434-gc95e797b83 i.e. from last Thursday/Friday.

@gwerbin
Copy link
Contributor Author

gwerbin commented Feb 8, 2021

Thanks for the report! Unfortunately, my ability to debug this one is next to nil - I don't have access to a MacOS environment.

Did this start recently? In particular, had you updated packer or neovim shortly before first noticing this issue?

I'm almost wondering if this is an upstream bug. It sounds like it could be a problem with Neovim's rendering.

The redrawtime warning could point to an issue with syntax highlighting being inappropriately applied in the packer window, but that would seemingly trigger on other platforms too.

This has been going since way back when I first reported the issue about Packer freezing during the Git password prompt. I've been meaning to report it for a while.

My current version info:

  • Neovim v0.5.0-dev+1000-g84d08358b, built with brew install --HEAD neovim
  • Packer commit c5b7f23e0b406767c1918d6888962fdb97f951e8
  • iTerm 3.4.3 with TERM=xterm-256color

On my Linux machine, I was not able to reproduce this in Urxvt:

  • Neovim v0.5.0-dev+e0a4399
  • Packer commit c5b7f23e0b406767c1918d6888962fdb97f951e8
  • rxvt-unicode 9.22 with TERM=rxvt-unicode

But I was able to reproduce on the Mac in Terminal.app.

I have also been having trouble lately with Terminfo not working right on my Mac. So I actually have a suspicion that the problem ultimately lies with whatever is wrong with my Terminfo database, and not iTerm specifically. Which would explain why it fails in all Mac terminals, succeeds in a Mac GUI, and succeeds in all Linux terminals & GUIs.

This might be a non-issue and/or an upstream issue. Need to investigate more when I have time.

Edit: Terminfo issue turned out to be local to my shell init script. Frozen Packer window issue persists in Terminal.app and iTerm2.

@mherna
Copy link

mherna commented Feb 12, 2021

@gwerbin I'm having the same issue on my mac, please let me know if you find a solution

@wbthomason
Copy link
Owner

@mherna: Also with iTerm2/Terminal.app? @akinsho, what terminal emulator do you use?

Have you had this problem with PackerInstall, or just update/sync?

Have you had this/similar problems with other Neovim plugins, e.g. telescope?

Finally, could you please update packer and post the contents of ~/.cache/nvim/packer.nvim.log after running an update/sync? I recently added more debug logging which should help us determine if this is a task hanging and blocking everything or the display not redrawing for some reason.

@akinsho
Copy link
Collaborator

akinsho commented Feb 12, 2021

I use kitty exclusively on my mac @wbthomason

@wbthomason
Copy link
Owner

Thanks - for reference, I also exclusively use kitty (on Linux) and cannot reproduce this.

@gegoune
Copy link

gegoune commented Feb 12, 2021

I am gonna risk being funny here but a good while ago I though I experienced something like that. It was all due to window being pretty small (in multiple planes) and for some reason packer/nvim was waiting for me to hit enter after showing some message, if I remember correctly it was about rplugin file being created/updated. It could be something like this perhaps?

@wbthomason
Copy link
Owner

Ah - Neovim (or Vim) will prompt for Enter by default if a certain number of message lines get printed all at once, which can happen after running UpdateRemotePlugins. Maybe that's happening here? I would be a bit surprised, though.

@gwerbin
Copy link
Contributor Author

gwerbin commented Feb 12, 2021

During one of these window freezes, I don't see any prompt appear, and I don't see output in :messages other than the 'redrawtime' exceeded one.

Also, the window is not really freezing. I can still interact with Nvim by moving the cursor, typing Ex commands, etc. But the information in the Packer window gets stuck part-way through updating.

It happens no matter how big the terminal window is.

I still wonder if this is an upstream Nvim issue relating to how control codes are processed by both of these Mac terminals. What mechanism does Packer use to redraw the window?

@wbthomason
Copy link
Owner

wbthomason commented Feb 12, 2021

Packer uses extmarks to track which lines correspond to which tasks, then sets modifiable=true for the display buffer, uses nvim_buf_set_lines to set the contents of the line, and sets modifiable=false again.

As far as I know, this is fairly standard? Looking at vim-packager, for example, it seems to take this general approach (though it does refresh only on a timer, whereas packer refreshes whenever there's new data). Perhaps Neovim cannot keep up with rapid updates on iTerm2, for some reason?

@mherna
Copy link

mherna commented Feb 12, 2021

@wbthomason yes, the issue also happens in terminal.app. telescope.nvim works well, and correct it happens when running PackerSync and PackerUpdate; PackerInstall works fine.

Here are the logs. It appears to get stuck waiting for plugin info is not ready yet.

[INFO  Fri Feb 12 12:37:24 2021] .../site/pack/packer/start/packer.nvim/lua/packer/clean.lua:78: Already clean!
[ERROR Fri Feb 12 12:37:25 2021] .../site/pack/packer/start/packer.nvim/lua/packer/async.lua:17: Error in coroutine: ...ite/pack/packer/start/packer.nvim/lua/packer/install.lua:26: attempt to call upvalue 'fmt' (a nil value)
[ERROR Fri Feb 12 12:37:25 2021] .../site/pack/packer/start/packer.nvim/lua/packer/async.lua:17: Error in coroutine: ...ite/pack/packer/start/packer.nvim/lua/packer/install.lua:26: attempt to call upvalue 'fmt' (a nil value)
[DEBUG Fri Feb 12 12:37:26 2021] ...site/pack/packer/start/packer.nvim/lua/packer/update.lua:88: Failed to update AndrewRadev/sideways.vim: {
  data = {
    data = {
      exit_code = 1,
      signal = 0
    },
    msg = "Error pulling updates for AndrewRadev/sideways.vim: Your configuration specifies to merge with the ref 'refs/heads/master'\nfrom the remote, but no such ref was fetched."
  },
  msg = "Error checking updated commit for AndrewRadev/sideways.vim: 19c5a21"
}
[INFO  Fri Feb 12 12:37:35 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet
[INFO  Fri Feb 12 12:37:40 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet
[INFO  Fri Feb 12 12:37:42 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet
[INFO  Fri Feb 12 12:37:43 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet
[INFO  Fri Feb 12 12:37:54 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet
[INFO  Fri Feb 12 12:37:59 2021] ...ite/pack/packer/start/packer.nvim/lua/packer/display.lua:381: Operations are still running; plugin info is not ready yet

@mherna
Copy link

mherna commented Feb 12, 2021

@wbthomason if running PackerUpdate it in any other terminal emulator may help solve the issue please let me know and I will try it.

@gwerbin
Copy link
Contributor Author

gwerbin commented Feb 12, 2021

How did you get those logs? I'd like to check mine as well.

I think I once had that "no such ref" problem in a plugin, and fixing it resolved the issue for a short while. Maybe there's something going wrong when Packer clones certain plugins from Github, causing it to freeze?

@mherna
Copy link

mherna commented Feb 12, 2021

@gwerbin nvim .cache/nvim/packer.nvim.log

@mherna
Copy link

mherna commented Feb 12, 2021

If it helps... I use @tjdevries neovim configuration

@gwerbin
Copy link
Contributor Author

gwerbin commented Feb 12, 2021

Unfortunately I don't see any such output in my case. I just see the "already clean!" message.

Would it help to post my compiled packer script?

@wbthomason
Copy link
Owner

@gwerbin: I recently added more debug logging; if you're not on the latest version of packer or haven't encountered this issue since you last updated packer, you may not have any useful output.

@mherna: That output actually points more toward a problem with git - the "info is not ready yet" message gets printed when you press on the packer display before all operations have finished.

It seems that either (as #200 might also imply) the git hanging issue is not as fixed as we thought in #91. I'm wondering if there's something different with the git-relevant environment variables (or your particular git config) on macOS that causes this?

It is also possible that there's an issue in the async module which causes the set of jobs to hang if one fails in this way, but I think we would see that more broadly. Still, it could be the cause.

@mherna
Copy link

mherna commented Feb 12, 2021

@wbthomason I was pressing the return key; that's why those messages got added to the logs.

The git version I was running was git version 2.24.3 (Apple Git-128) I used brew to install a newer version of git, and now I'm running git version 2.30.1, but the issue persists. I also checked my network packets, and once the packer window "freezes," no more packets are being sent or received.

I also reviewed my environment variables, and I have no variables related to git, which could affect Packer.nvim. Besides, I removed those plugins that were not found. My new :PackerUpdate logs are:

As you say, it is probably related to #200

[INFO  Fri Feb 12 15:39:10 2021] .../site/pack/packer/start/packer.nvim/lua/packer/clean.lua:78: Already clean!
[DEBUG Fri Feb 12 15:39:12 2021] ...site/pack/packer/start/packer.nvim/lua/packer/update.lua:80: Updated pwntester/octo.nvim: {
  err = {},
  messages = { "a91c61a Merge branch 'master' of github.com:pwntester/octo.nvim (47 seconds ago)", "ab97242 fix #86 (2 minutes ago)" },
  output = { "remote: Enumerating objects: 31, done.        \nremote: Counting objects:   3% (1/31)        \rremote: Counting objects:   6% (2/31)        \rremote: Counting objects:   9% (3/31)        \rremote: Counting objects:  12% (4/31)        \rremote: Counting objects:  16% (5/31)        \rremote: Counting objects:  19% (6/31)        \rremote: Counting objects:  22% (7/31)        \rremote: Counting objects:  25% (8/31)        \rremote: Counting objects:  29% (9/31)        \rremote: Counting objects:  32% (10/31)        \rremote: Counting objects:  35% (11/31)        \rremote: Counting objects:  38% (12/31)        \rremote: Counting objects:  41% (13/31)        \rremote: Counting objects:  45% (14/31)        \rremote: Counting objects:  48% (15/31)        \rremote: Counting objects:  51% (16/31)        \rremote: Counting objects:  54% (17/31)        \rremote: Counting objects:  58% (18/31)        \rremote: Counting objects:  61% (19/31)        \rremote: Counting objects:  64% (20/31)        \rremote: Counting objects:  67% (21/31)        \rremote: Counting objects:  70% (22/31)        \rremote: Counting objects:  74% (23/31)        \rremote: Counting objects:  77% (24/31)        \rremote: Counting objects:  80% (25/31)        \rremote: Counting objects:  83% (26/31)        \rremote: Counting objects:  87% (27/31)        \rremote: Counting objects:  90% (28/31)        \rremote: Counting objects:  93% (29/31)        \rremote: Counting objects:  96% (30/31)        \rremote: Counting objects: 100% (31/31)        \rremote: Counting objects: 100% (31/31), done.        \nremote: Compressing objects:  25% (1/4)        \rremote: Compressing objects:  50% (2/4)        \rremote: Compressing objects:  75% (3/4)        \rremote: Compressing objects: 100% (4/4)        \rremote: Compressing objects: 100% (4/4), done.        \nremote: Total 10 (delta 6), reused 10 (delta 6), pack-reused 0        \nFrom https://github.com/pwntester/octo.nvim\n   e8fb856..a91c61a  master     -> origin/master", "Updating e8fb856..a91c61a\nFast-forward\n lua/octo/reviews.lua | 4 ++++\n 1 file changed, 4 insertions(+)" },
  revs = { "e8fb856", "a91c61a" }
}

@akinsho
Copy link
Collaborator

akinsho commented Mar 11, 2021

Okay so turn of events but I'm now seeing this issue on my mac as well using kitty it doesn't always happen , but sometimes running PackerSync stalls and never completes. I see errors relating to number of opened files allowed similar to #149. I tried re-running the command a few times and I think when you close the window when it hangs it doesn't kill any running jobs so then hitting the command again spins up even more jobs which triggers #149.

@akinsho
Copy link
Collaborator

akinsho commented Mar 11, 2021

I just experimented with the solution from #149 i.e. I ran ulimit -S -n 200048 to increase the number of allowed files that can be opened and that seems to allow the command to run without issues. @mherna can you see if running that command in your shell then re-opening neovim and running packer update helps at all?

@wbthomason
Copy link
Owner

Maybe there should be a default job limit, rather than defaulting to starting everything at once? @mherna @akinsho, roughly how many plugins do you have?

@mherna
Copy link

mherna commented Mar 11, 2021

I have 130 plugins @wbthomason, @akinsho is correct setting ulimit -S -n 200048 worked wonders now PackerUpdate ran successfully.

@akinsho
Copy link
Collaborator

akinsho commented Mar 11, 2021

I have about 70 on my mac, I'm curious how many jobs packer is running to actually hit this limit in the first place. The things I found on this indicate macOS is a little stingy but still seems like the default is in the thousands so wondering how packer could hit this unless its running like 10-20+ jobs per plugin?

EDIT: also wondering what happens to all the jobs if you press q before they finish I'm assuming they try and continue in the back ground but this won't stop you from just running the command again?

@wbthomason
Copy link
Owner

@akinsho: Right, I too am surprised that packer would be running this many simultaneous jobs - each plugin should have at most one job running at a time. If you press q before they finish, the currently running jobs may continue but no new top-level jobs should start (the async pool will prevent new tasks from starting).

Does this ever happen in a fresh terminal, i.e. a shell process with no chance of previously opened files that are lingering, etc.?

@akinsho
Copy link
Collaborator

akinsho commented Mar 13, 2021

Not sure @wbthomason I use tmux and have windows and panes up for ages but do restart my Mac now and again. Not sure if it'd be different in a new shell can try and see when next I get a chance.

@cehoffman
Copy link

cehoffman commented Mar 16, 2021

The file limit does seem to be the issue. I was experiencing this as well and I have 82 plugins. Setting the higher limit with ulimit -S -n 200048 lets the PackerUpdate run to completion. I'm in a fairly typical environment as well with iTerm2 and several tmux panes. Setting it so high isn't required. My work computer isn't allowing that high a limit but 40000 works to fix the issue.

@akinsho
Copy link
Collaborator

akinsho commented Mar 18, 2021

@wbthomason I've tried a few times and this seems to eventually creep up, this article is quite a useful explainer but the TLDR is that it's a per application limit i.e. all of nvim's files are included in this not just packer but the limit is still 10,000 +. by default If it helps I've never experienced this issue prior to using packer so I don't think this is some other errant code in some part of nvim.

I wonder if some tidying up of resources is required between backer runs like job:close or something some of the luv jobs pure guesswork on my end. I can just add the command to my zshrc but as the article points out the limit is quite high and its indicative of a bug for it to be this high. I think the fact that it worsens the longer the shell/nvim is open for seems to me to indicate that the count is only increasing.

I wonder if the files could be visualised and I could use that to track the count over time

@smackesey
Copy link

smackesey commented Aug 29, 2021

I also have this problem. The Packer UI freezes vim every time on Sync/Update/Install. Some notes:

  • Freezes on <plugin name> checking updated commit... for some plugin, even with only one plugin listed.
  • Occurs in both VimR (an NVIM GUI) and in iTerm2.
  • Setting max_jobs (as @kyoh86 suggested) does not fix the problem.
  • Setting ulimit -S -n 200048 does not fix the problem.
  • Freezes even with PackerInstall
  • To quit VimR during the freeze, I don't have to Force Quit. Instead a regular quit (cmd-q) works, but there are several seconds of spinning beach ball before it quits.
  • Nothing printed to ~/.cache/nim/packer.nvim.log
  • commit 09cc2d (from Aug 26 2021)
  • macOS 11.15

Basically the plugin is completely unusable for me because of this mysterious freezing.

@ahmedelgabri
Copy link

I started to get the same issue too after adding some new plugins. Which took the number to 76 if I go to 73 plugins as @kyoh86 said, it works fine. Once I add an extra plugin I hit this issue.

  • Setting max_jobs does not fix the problem. Even if I set it to 1.
  • Setting ulimit -S -n 200048 does not fix the problem.
  • I'm using Kitty 0.23.1 on macOS 11.5.2 neovim 0.5.0 & packer 09cc2d6
  • tailing the log file shows that packer is always stuck at running tasks
[DEBUG Sun 29 Aug 12:20:09 2021 2204420155507] ...config/nvim/pack/packer/start/packer.nvim/lua/packer.lua:565: Running tasks

@akinsho
Copy link
Collaborator

akinsho commented Aug 29, 2021

Regarding set ulimit -S are people adding this to their shell startup scripts or just running it once? Since it will only apply for the lifetime of the shell, if it isn't re-executed every time you start one.

Another thing to note is that the UI will sometimes block if there is an error somewhere with a plugin which isn't ideal, but I have seen this before if there are issues with a plugin spec

@ahmedelgabri
Copy link

@akinsho I didn't add it to any shell startup files. I only ran it inside a shell and tested to see if it will work first or not, before deciding to make it permanent.

@mhanberg
Copy link

mhanberg commented Sep 1, 2021

I started to get the same issue too after adding some new plugins. Which took the number to 76 if I go to 73 plugins as @kyoh86 said, it works fine. Once I add an extra plugin I hit this issue.

* Setting `max_jobs` does not fix the problem. Even if I set it to `1`.

* Setting `ulimit -S -n 200048` does not fix the problem.

* I'm using Kitty `0.23.1` on macOS `11.5.2` neovim `0.5.0` & packer [09cc2d6](https://github.com/wbthomason/packer.nvim/commit/09cc2d615bbc14bca957f941052e49e489d76537)

* `tail`ing the log file shows that packer is always stuck at `running tasks`
[DEBUG Sun 29 Aug 12:20:09 2021 2204420155507] ...config/nvim/pack/packer/start/packer.nvim/lua/packer.lua:565: Running tasks

Chiming in to say I just experienced this as well. I dropped some plugins and now the update window seems to work again

@wbthomason
Copy link
Owner

@mhanberg/anyone else: I don't suppose you have the specs that you dropped to make this work? This is a mysterious bug; maybe we can isolate specific features, e.g. that are present in the troublesome specs? Though as @smackesey notes, apparently this (sometimes) happens with only 1 plugin spec, so maybe not.

I'm mostly mystified by the inconsistency; plenty of macOS users seem to have no trouble with packer (as far as I'm aware).

@ahmedelgabri
Copy link

@wbthomason In my case it didn't really matter which specs/plugins I drop. As long as the total number is <= 73 it works. I can constantly replicate the issue with nearly any 73 plugin combination.

@wbthomason
Copy link
Owner

Good to know, thanks. I would expect the ulimit or concurrent tasks limit fixes to work for that, but it seems they did not in your case. Maybe there's a race condition that becomes more likely with increasing numbers of plugins? But then I'd expect to see this on non-macOS platforms, unless it's a luv/libuv platform-specific bug...

@akinsho
Copy link
Collaborator

akinsho commented Sep 2, 2021

@ahmedelgabri it might be worth going through the post around ulimit on your own there are some comments in there that helped me debug the limit and the number of process my shell had running https://superuser.com/questions/433746/is-there-a-fix-for-the-too-many-open-files-in-system-error-on-os-x-10-7-1

gotgenes added a commit to gotgenes/dotfiles that referenced this issue Sep 16, 2021
The UI started freezing with more plugins.

See wbthomason/packer.nvim#202
@smackesey
Copy link

Just reviving this issue and reporting a bit of new info-- I updated to macOS Monterey and was hoping maybe this problem would disappear, but nothing has changed.

@ahmedelgabri
Copy link

For me it seems like max_jobs = 70 is the max I can set and makes the issue goes away.

@gotgenes
Copy link

FWIW I use the the nvim-treesitter plugin which will launch more processes with TSUpdate command. I found I had to further reduce max_jobs to 60 to allow for that command to also succeed.

@beauwilliams
Copy link

I am having the same problem too. Packer has not been able to update my packages for a good 2 months or so. It just freezes.

@beauwilliams
Copy link

This fixes my issue. Setting max jobs to 4. I am on MacOSX.
#456 (comment)

@jryio
Copy link

jryio commented Dec 1, 2021

Update: After modifying lunarvim to take max_jobs = 4 I can confirm that @beauwilliams has a working solution for my configuration below. Setting max_jobs = 50 still causes freezes as in #456


I am also experiencing a total UI lock and freeze running on macOS

OS Version: macOS Monterey 12.0.1
Laptop: Apple M1 Max chip
Terminal: Alacritty (alacritty 0.9.0 (fed349a)) installed via Homebrew
Neovim:

NVIM v0.6.0
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by brew@Monterey

Packer Commit: 7f62848f3a92eac61ae61def5f59ddb5e2cc6823

@ghost
Copy link

ghost commented Dec 22, 2021

just hit this problem on arch linux with neovim 0.6, all mentioned solutions above did not work out

@towry
Copy link

towry commented Jun 10, 2022

just hit this problem on arch linux with neovim 0.6, all mentioned solutions above did not work out

Add max_jobs=50, then PackerClean and PackerSync works.

@marcelarie
Copy link

How can I fix this, where should I put max_jobs ?

@wilkinson4
Copy link

How can I fix this, where should I put max_jobs ?

require('packer').startup({
    function(use)
        -- Packer can manage itself
        use {'wbthomason/packer.nvim'}
    end,
    config = {display = {open_cmd = 'leftabove 75vnew \\[packer\\]'}, max_jobs = 10}
})

corasaurus-hex added a commit to corasaurus-hex/lvim that referenced this issue Jul 1, 2022
The default config makes PackerInstall/Sync freeze up unless you set this config value [as recommended in this packer.nvim issue](wbthomason/packer.nvim#202).
ChristianChiarulli referenced this issue in ChristianChiarulli/nvim Jul 26, 2022
WilliamHsieh added a commit to WilliamHsieh/dotfiles that referenced this issue Nov 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests