return used tls from @threads for easier use of task_local_storage()

[IanButterworth]

In thinking about #48542 I wondered maybe @threads could return the tls of all the used tasks, which would make task_local_storage() easier to use with @threads.

Something like

julia> function sum_multi(a::Vector{T}) where T
           tlss = Threads.@threads for i in eachindex(a)
               buf = get(task_local_storage(), :buf, zero(T))::T
               task_local_storage()[:buf] = buf + a[i]
           end
           return sum(tls -> tls[:buf]::T, tlss)
       end
sum_multi (generic function with 1 method)

julia> sum_multi(1:1_000_000)
500000500000

(Maybe this toy example could be reduced further with smarter get usage?)

[IanButterworth]

To make this friendly and not too verbose to repl usage, a minimal show method for Vector{IdDict{Any, Any}} could be added

[vtjnash]

I think there is already a different macro for parallel reductions that can be more efficient than this (parallel reductions)? I forget what the name changed to. It used to be @parallel in v0.0.

[MasonProtter]

@vtjnash are you thinking of what's now @distributed [reducer] for ...? We've never had something like that for multithreading in Base, only multiprocessing as far as I'm aware.

[vtjnash]

Yeah, sounds right

[vchuravy]

I think the right strategy here is to implement something akin to Hyper Reducers in Cilk (http://www.fftw.org/~athena/papers/hyper.pdf)

@tkf was experimenting with similar constructs, and I think it would be worthwhile providing a richer infrastructure in Base,
but I don't think returning/exposing TLS is the right strategy.

[vtjnash]

How about simply:

Suggested change

	return map(Base.get_task_tls, tasks)
	return tasks

[IanButterworth]

Is there other useful things inside the tasks?

[IanButterworth]

One thing that's nicer is that the show method for a vector of tasks is a lot tidier than a vector of dict. So running a threaded for loop in the repl would be tidier.

[IanButterworth]

Another idea would be to expose a way to handle the buffer objects without the user having to manipulate tasks/TLSs

xs, ys = Threads.@threads x = zeros(10,10) y = zeros(2,2) for i in eachindex(a)
 	# do something with x, y, a[i] etc.
end

# optionally do something with xs & ys which are both of length threadpoolsize()

Edit: tried out here #50052

[vchuravy]

I would prefer that over exposing the TLS objects in that way. Kinda reminds me of Floops reducers.

I am not super convinced by the syntax, but the common ask seems to be that one wants some buffer shared by iterations on that tasks and then somehow returned.

[jonas-schulze]

How about having the local keyword create a task-local variable?

Edit: Or some new Julia keyword task_local akin to thread_local from C++.

https://en.cppreference.com/w/cpp/language/storage_duration

[IanButterworth]

What would an example of that look like? I think I get idea but not sure how initialization would be indicated

[IanButterworth]

It seems to me that using task_local_storage() with @threads (and returning tls in some form from the macro) would rarely be a good fit because task_local_storage() access has to be in the loop body, thus you introduce dict operations every iteration.

#50052 seems like a better fit for @threads use cases.

I'm tempted to close this in favor of that

[jonas-schulze]

What would an example of that look like? I think I get idea but not sure how initialization would be indicated

Threads.@threads for i in eachindex(a)
    # only initialized once; not reset between different `i`
    # similar to `static` from C/C++
    local x = zeros(10, 10)
    local y = zeros(2, 2)

    # do something with x, y, a[i] etc.
end