IncrementalCompact re-iteration is broken

[maleadt]

Normally it should be possible to (partially) iterate an object, and iterate anew using a different iterator. This is not the case with IncrementalCompact.

Set-up, on latest master:

julia> foo(i) = i == 1 ? 1 : 2
foo (generic function with 1 method)

julia> ir = only(Base.code_ircode(foo, (Int,)))[1]
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
  2 ─      return 1                                                                         │
  3 ─      return 2                                                                         │


julia> compact = Core.Compiler.IncrementalCompact(ir)
──────────────────────────────────────────────────────────────────────────────────────────────────
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
  2 ─      return 1                                                                         │
  3 ─      return 2                                                                         │

Let's iterate a bit:

julia> x = Core.Compiler.iterate(compact)
(Pair{Pair{Int64, Int64}, Any}(1 => 1, :(_2 === 1)), (2, 1))

julia> compact
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
──────────────────────────────────────────────────────────────────────────────────────────────────
  └──      goto #3 if not %1                                                                │
  2 ─      return 1                                                                         │
  3 ─      return 2                                                                         │


julia> x = Core.Compiler.iterate(compact, x[2])
(Pair{Pair{Int64, Int64}, Any}(2 => 2, :(goto %3 if not %1)), (3, 2))

julia> compact
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
──────────────────────────────────────────────────────────────────────────────────────────────────
  2 ─      return 1                                                                         │
  3 ─      return 2                                                                         │

The same, but with a new iterator:

julia> x = Core.Compiler.iterate(compact)
(Pair{Pair{Int64, Int64}, Any}(3 => 3, :(return 1)), (4, 1))

julia> compact
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
  2 ─      return 1                                                                         │
──────────────────────────────────────────────────────────────────────────────────────────────────
  3 ─      return 2                                                                         │


julia> x = Core.Compiler.iterate(compact, x[2])
(Pair{Pair{Int64, Int64}, Any}(4 => 4, :(return 2)), (5, 1))

julia> x = Core.Compiler.iterate(compact, x[2])

julia> compact
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
  2 ─      return 1                                                                         │
  3 ─      return 2                                                                         │
──────────────────────────────────────────────────────────────────────────────────────────────────

So this second iteration resumed where the previous iterator left off, which is wrong, but it also breaks the resulting IR's CFG:

julia> Core.Compiler.complete(compact)
1 1 ─ %1 = (_2 === 1)::Bool                                                                 │╻ ==
  └──      goto #3 if not %1                                                                │
  !!! ─      return 1                                                                       │
  !!! ─      return 2                                                                       │

The !!!s don't occur if I only iterate using a single iterator.

[vtjnash]

Normally it should be possible to (partially) iterate an object, and iterate anew using a different iterator

This is sometimes true, but stateful iterators exist and aren't required to obey it

[maleadt]

Stateful iterators exist and aren't required to obey it

Can you give an other example? I take it that would mean it's OK for the second iterator to resume at the point the first left off. I can understand that, but the fact that the completed IR is broken afterwards still seems like a bug.

[vtjnash]

It is rare to have an iterator that is also mutating, but that is the explicit purpose of this iterator

[maleadt]

I'm not saying it shouldn't mutate, I'm saying that it shouldn't corrupt the IR when using multiple of these stateful iterators. I think this is a legit issue, because of the state is initialized (it disregards the running bb counter). I agree it's rare though (esp. because it's still easy to corrupt the IR when using multiple of these iterators), but it seems like a simple change to fix this at least.